The popularity of sports and the associated commercial benefits have encouraged broadcasters to generate and circulate an enormous volume of sports videos across online digital platforms. Effective management and processing of such extensive content is challenging. This brings a demand for the development of effective and efficient summarization techniques to handle large video collections while maintaining viewer engagement and optimizing storage and transmission requirements. This study introduces an automated system for summarizing videos of multiple sports genres through excitement-based event detection. The proposed approach analyzes the audio stream of sports videos to identify exciting events, which are then used to create concise video summaries. We propose a novel feature space, trimmed mean absolute deviated-local ternary patterns (TMAD-LTP), to better capture the distinctive traits of exciting segments within audio stream. We used our TMAD-LTP features to train a binary Support Vector Machine classifier to distinguish between excited and non-excited frames. The video frames corresponding to the exciting audio frames are identified as keyframes. The prior and later frames around these keyframes are appended to produce meaningful video skims that are then arranged sequentially to generate the highlights. Experimental evaluation on a diverse set of cricket and soccer videos demonstrates the impressive performance of the proposed method over contemporary approaches by achieving an average accuracy of 97.71%. These impressive results validate the efficacy of our approach for generating more useful sports highlights.