The learning activity and education process have been moving towards heavy multimedia consumption and online platforms as the means of content delivery. With so many online video courses, video summarisation has become a handy tool. It captures the input video’s important information by selecting a subset of the video frames or shots and generating a summary of that video. 

Researchers from Shiv Nadar University, Sonia Khetarpaul, Lakshay Jain, Kush Goyal, and P Vishnu Tej, developed a Deep Learning model for analyzing lecture videos. The video summarisation model consists of automatically generating video frame summaries, which are of two types - static summaries and dynamic summaries. Static summaries are a series of keyframes, while dynamic summaries are a series of shots.

In the rapidly evolving landscape of the internet, the pivotal role of lecture videos in education has become increasingly prominent. However, leveraging these videos for effective learning presents its own set of challenges. One notable limitation is the difficulty in using video lectures for quick review. Furthermore, locating specific information within a video can be a daunting task. While video scrubbing facilitates visual exploration, it falls short when efficiently skimming the audio content. 

Often, crucial explanations and contextual information accompany the visuals, making it imperative to find a more effective solution. To address these challenges, an approach involves the integration of slideshows that encapsulate the essence of lectures through succinct text and illustrative photos.

Tools and approach

The researcher used various tools and datasets for lecture video summarisation. VT-SSum is a benchmark dataset with spoken language for video transcript segmentation and summarisation. The lecture videos used have been collected from NPTEL(National Programme on Technology Enhanced Learning). The website provides lecture video recordings of various courses in numerous domains. CSO Classifier, a new unsupervised approach for automatically classifying research papers according to the Computer Science Ontology (CSO), a comprehensive ontology of research areas in the field of Computer Science. The CSO Classifier takes as input the keyword and returns a selection of research concepts drawn from the ontology.

Furthermore, the model has numerous features that make it efficient, such as video splitting, frame extraction, audio extraction, speech-to-text conversion, key phrase extraction, heading extraction, text summarisation, and Ontology-based key phrase extraction. The researchers used different tools to develop each of these features.

Laborious Task

After thoroughly analysing the results, the researchers concluded that video summarisation is a laborious task that has been automated with the help of deep Learning techniques. The quality of summarisation depends on the transcript, speech extraction precision, and the model’s accuracy. The transcript obtained depends upon the precision of the speech-to-text API. The CSO classifier contains a comprehensive ontology of research areas in Computer Science.

Based on a research by Shiv Nadar University.

Image: Unsplash