Unsupervised Video Summarization with Attentive Conditional Generative Adversarial Networks

Xufeng He, Yang Hua, Tao Song, Zongpu Zhang, Zhengui Xue, Ruhui Ma, Neil Robertson, Haibing Guan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Citations (Scopus)
327 Downloads (Pure)

Abstract

With the rapid growth of video data, video summarization technique plays a key role in reducing people's efforts to explore the content of videos by generating concise but informative summaries. Though supervised video summarization approaches have been well studied and achieved state-of-the-art performance, unsupervised methods are still highly demanded due to the intrinsic difficulty of obtaining high-quality annotations. In this paper, we propose a novel yet simple unsupervised video summarization method with attentive conditional Generative Adversarial Networks (GANs). Firstly, we build our framework upon Generative Adversarial Networks in an unsupervised manner. Specifically, the generator produces high-level weighted frame features and predicts frame-level importance scores, while the discriminator tries to distinguish between weighted frame features and raw frame features. Furthermore, we utilize a conditional feature selector to guide GAN model to focus on more important temporal regions of the whole video frames. Secondly, we are the first to introduce the frame-level multi-head self-attention for video summarization, which learns long-range temporal dependencies along the whole video sequence and overcomes the local constraints of recurrent units, e.g., LSTMs. Extensive evaluations on two datasets, SumMe and TVSum, show that our proposed framework surpasses state-of-the-art unsupervised methods by a large margin, and even outperforms most of the supervised methods. Additionally, we also conduct the ablation study to unveil the influence of each component and parameter settings in our framework.
Original languageEnglish
Title of host publicationProceedings of the 27th ACM International Conference on Multimedia
PublisherAssociation for Computing Machinery (ACM)
Pages2296–2304
ISBN (Print)978-1-4503-6793-6
DOIs
Publication statusPublished - Oct 2019

Fingerprint

Dive into the research topics of 'Unsupervised Video Summarization with Attentive Conditional Generative Adversarial Networks'. Together they form a unique fingerprint.

Cite this