Published February 28, 2023 | Version CC BY-NC-ND 4.0
Journal article Open

A Review on Key Features and Novel Methods for Video Summarization

Creators

  • 1. Faculty of Computing, Sathyabama University, Chennai (Tamil Nadu), India.
  • 2. Principal, Swarnandhra College of Engineering and Technology, Narasapur (A.P), India.

Contributors

Contact person:

  • 1. Faculty of Computing, Sathyabama University, Chennai (Tamil Nadu), India.

Description

Abstract: In this paper, we discuss techniques, algorithms, evaluation methods used in online, offline, supervised, unsupervised, multi-video and clustering methods used for Video Summarization/Multi-view Video Summarization from various references. We have studied different techniques in the literature and described the features used for generating video summaries with evaluation methods, supervised, unsupervised, algorithms and the datasets used.We have covered the survey towards the new frontier of research in computational intelligence technique like ANN (Artificial Neural Network) and other evolutionary algorithms for VS using both supervised and unsupervised methods. We highlight on single, multi-video summarization with features like video, audio, and semantic embeddings considered for VS in the literature. A careful presentation is attempted to bring the performance comparison with Precision, Recall, F-Score, and manual methods to evaluate the VS.

Notes

Published By: Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP) © Copyright: All rights reserved.

Files

F37370811622.pdf

Files (920.2 kB)

Name Size Download all
md5:e4cbaf7569d89ca0b276f36ef768873b
920.2 kB Preview Download

Additional details

Related works

Is cited by
Journal article: 2249-8958 (ISSN)

References

  • Open video project.
  • E. ASADI AND N. M. CHARKARI, Video summarization using fuzzy c-means clustering, in Electrical Engineering (ICEE), 2012 20th Iranian Conference on, IEEE, 2012, pp. 690–694.
  • X. CHE, H. YANG, AND C. MEINEL, Automatic online lecture highlighting based on multimedia analysis, IEEE Transactions on Learning Technologies, 11 (2018), pp. 27–40.
  • S. E. F. DE AVILA, A. P. B. LOPES, A. DA LUZ JR, AND A. DE ALBU- QUERQUE ARAÚJO, Vsumm: A mechanism designed to produce static video summaries and a novel evaluation method, Pattern Recognition Letters, 32 (2011), pp. 56–68.
  • M. ELFEKI, A. SHARGHI, S. KARANAM, Z. WU, AND A. BORJI, Multi-view egocentric video summarization, arXiv preprint arXiv:1812.00108, (2018).
  • E. ELHAMIFAR AND M. C. D. P. KALUZA, Online summarization via submodular and convex optimization., in CVPR, 2017, pp. 1818–1826.
  • Z. ELKHATTABI, Y. TABII, AND A. BENKADDOUR, Video summarization: techniques and applications, World Academy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering, 9 (2015), pp. 928–933.
  • J. FAJTL, H. S. SOKEH, V. ARGYRIOU, D. MONEKOSSO, AND P. REMAGNINO, Summarizing videos with attention, arXiv preprint arXiv:1812.01969, (2018).
  • H. FAROUK, K. ELDAHSHAN, AND A. A. E. ABOZEID, Effective and efficient video summarization approach for mobile devices, International Journal of Interactive Mobile Technologies (iJIM), 10 (2016), pp. 19–26.
  • L. FENG, Z. LI, Z. KUANG, AND W. ZHANG, Extractive video summarizer with memory augmented neural networks, in 2018 ACM Multimedia Conference on Multimedia Conference, ACM, 2018, pp. 976–983.
  • Y. FU, Y. GUO, Y. ZHU, F. LIU, C. SONG, AND Z.-H. ZHOU, Multi-view video summarization, IEEE Transactions on Multimedia, 12 (2010), pp. 717–729.
  • Y. GONG AND X. LIU, Video summarization and retrieval using singular value decomposition, Multimedia Systems, 9 (2003), pp. 157–168.
  • M. GYGLI, H. GRABNER, AND L. VAN GOOL, Video summarization by learning submodular mixtures of objectives, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3090–3098.
  • A. HANJALIC AND H. ZHANG, An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis, IEEE Transactions on circuits and systems for video technology, 9 (1999), pp. 1280–1289.
  • R. HANNANE, A. ELBOUSHAKI, AND K. AFDEL, Mskvs: Adaptive mean shift- based keyframe extraction for video summarization and a new objective verifi- cation approach, Journal of Visual Communication and Image Representation, (2018).
  • Y. HE, C. GAO, N. SANG, Z. QU, AND J. HAN, Graph coloring-based surveillance video synopsis, Neurocomputing, 225 (2017), pp. 64–79.
  • T. HUSSAIN, K. MUHAMMAD, A. ULLAH, Z. CAO, S. W. BAIK, AND V. H. C. DE ALBUQUERQUE, Cloud-assisted multiview video summarization using cnn and bidirectional lstm, IEEE Transactions on Industrial Informatics, 16 (2019), pp. 77– 86.
  • Z. JI, Y. MA, Y. PANG, AND X. LI, Query-aware sparse coding for web multi- video summarization, Information Sciences, (2018).
  • Z. JI, K. XIONG, Y. PANG, AND X. LI, Video summarization with attention-based encoder-decoder networks, arXiv preprint arXiv:1708.09545, (2017).
  • Z. JI, Y. ZHANG, Y. PANG, AND X. LI, Hypergraph dominant set based multi- video summarization, Signal Processing, 148 (2018), pp. 114–123.
  • J. KAVITHA AND P. A. J. RANI, Static and multiresolution feature extraction for video summarization, Procedia Computer Science, 47 (2015), pp. 292–300.
  • P. KOUTRAS AND P. MARAGOS, Susinet: See, understand and summarize it, arXiv preprint arXiv:1812.00722, (2018).
  • J. LI, Z. LIAO, C. ZHANG, AND J. WANG, Event detection on online videos using crowdsourced time-sync comment, in Cloud Computing and Big Data (CCBD), 2016 7th International Conference on, IEEE, 2016, pp. 52–57.
  • M. MA, S. MET, J. HOU, S. WAN, AND Z. WANG, Video summarization via temporal collaborative representation of adjacent frames, in Intelligent Signal Processing and Communication Systems (ISPACS), 2017 International Symposium on, IEEE, 2017, pp. 164–169.
  • I. MADEMLIS, A. TEFAS, AND I. PITAS, Summarization of human activity videos using a salient dictionary, in Image Processing (ICIP), 2017 IEEE International Conference on, IEEE, 2017, pp. 625–629.
  • B. MAHASSENI, M. LAM, AND S. TODOROVIC, Unsupervised video summarization with adversarial lstm networks, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, 2017.
  • S. MEI, G. GUAN, Z. WANG, S. WAN, M. HE, AND D. D. FENG, Video summarization via minimum sparse reconstruction, Pattern Recognition, 48 (2015), pp. 522–533.
  • J. MENG, S. WANG, H. WANG, J. YUAN, AND Y.-P. TAN, Video summarization via multi-view representative selection, IEEE Trans. on Image Processing, (2018), pp. 2134–2145.
  • J. MOHAN AND M. S. NAIR, Dynamic summarization of videos based on descrip- tors in space-time video volumes and sparse autoencoder, IEEE Access, 6 (2018), pp. 59768–59778.
  • K. MUHAMMAD, J. AHMAD, Z. LV, P. BELLAVISTA, P. YANG, AND S. W. BAIK, Efficient deep cnn-based fire detection and localization in video surveillance applications, IEEE Transactions on Systems, Man, and Cybernetics: Systems, (2018), pp. 1–16.
  • K. MUHAMMAD, T. HUSSAIN, AND S. W. BAIK, Efficient cnn based summarization of surveillance videos for resource-constrained devices, Pattern Recognition Letters, (2018).
  • P. MUNDUR, Y. RAO, AND Y. YESHA, Keyframe-based video summarization using delaunay clustering, International Journal on Digital Libraries, 6 (2006), pp. 219–232.
  • A. PACKIALATHA AND A. CHANDRASEKAR, Effective video summarization using eigen based classification, Transylvanian Review, (2016).
  • V. PARAMANANTHAM AND D. S. SURESHKUMAR, Multi view video summarization using rnn and surf based high level moving object feature frames, Inter- national Journal of Engineering Research in Computer Science and Engineering (IJERCSE), 9 (2022).
  • M. PAUL AND M. M. SALEHIN, Spatial and motion saliency prediction method using eye tracker data for video summarization, IEEE Transactions on Circuits and Systems for Video Technology, (2018).
  • D. PURWANTO, Y.-T. CHEN, W.-H. FANG, AND W.-C. WU, Video summarization: How to use deep-learned features without a large-scale dataset, in 2018 9th International Conference on Awareness Science and Technology (iCAST), IEEE, 2018, pp. 220–225.
  • M. ROCHAN AND Y. WANG, Learning video summarization using unpaired data, arXiv preprint arXiv:1805.12174, (2018).
  • A. SHARGHI, J. S. LAUREL, AND B. GONG, Query-focused video summa- rization: Dataset, evaluation, and a memory network based approach, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2127–2136.
  • M. SRINIVAS, M. M. PAI, AND R. M. PAI, An improved algorithm for video summarization–a rank based approach, Procedia Computer Science, 89 (2016), pp. 812–819.
  • N. SRIVASTAVA, E. MANSIMOV, AND R. SALAKHUDINOV, Unsupervised learn- ing of video representations using lstms, in International conference on machine learning, 2015, pp. 843–852.
  • K. SUN, J. ZHU, Z. LEI, X. HOU, Q. ZHANG, J. DUAN, AND G. QIU, Learning deep semantic attributes for user video summarization, in Multimedia and Expo (ICME), 2017 IEEE International Conference on, IEEE, 2017, pp. 643–648.
  • W. TAYLOR AND F. Z. QURESHI, Real-time video summarization on commodity hardware, in Proceedings of the 12th International Conference on Distributed Smart Cameras, ACM, 2018, p. 16.
  • S. S. THOMAS, S. GUPTA, AND V. K. SUBRAMANIAN, Smart surveillance based on video summarization, in IEEE Region 10 Symposium (TENSYMP), 2017, IEEE, 2017, pp. 1–5.
  • D. S. K. VINSENT PARAMANANTHAM, A real time video summarization for youtube videos and evaluation of computational algorithms for their time and storage reduction, International Journal on Recent and Innovation Trends in Computing and Communication, 6 (2018), pp. 176–186.
  • J. WU, S.-H. ZHONG, J. JIANG, AND Y. YANG, A novel clustering method for static video summarization, Multimedia Tools and Applications, 76 (2017), pp. 9625–9641.
  • H. YANG, B. WANG, S. LIN, D. WIPF, M. GUO, AND B. GUO, Unsupervised extraction of video highlights via robust recurrent auto-encoders, in Proceedings of the IEEE international conference on computer vision, 2015, pp. 4633–4641.
  • X. YANG AND Z. WEI, Genetic keyframe extraction for soccer video, Procedia Engineering, 23 (2011), pp. 713–717.
  • L. YAO, A. TORABI, K. CHO, N. BALLAS, C. PAL, H. LAROCHELLE, AND COURVILLE, Describing videos by exploiting temporal structure, in Proceed- ings of the IEEE international conference on computer vision, 2015, pp. 4507– 4515.
  • Y. YUAN, T. MEI, P. CUI, AND W. ZHU, Video summarization by learning deep side semantic embedding, IEEE Transactions on Circuits and Systems for Video Technology, 29 (2017), pp. 226–237.
  • K. ZHANG, W.-L. CHAO, F. SHA, AND K. GRAUMAN, Video summarization with long short-term memory, in European conference on computer vision, Springer, 2016, pp. 766–782.
  • K. ZHANG, K. GRAUMAN, AND F. SHA, Retrospective encoders for video summarization, in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 383–399.
  • S. ZHANG, Y. ZHU, AND A. K. ROY-CHOWDHURY, Context-aware surveillance video summarization., IEEE Trans. Image Processing, 25 (2016), pp. 5469–5478.
  • Y. ZHANG, X. LIANG, D. ZHANG, M. TAN, AND E. P. XING, Unsupervised object-level video summarization with online motion auto-encoder, arXiv preprint arXiv:1801.00543, (2018).
  • Y. ZHANG, R. TAO, AND Y. WANG, Motion-state-adaptive video summarization via spatiotemporal analysis, IEEE Transactions on Circuits and Systems for Video Technology, 27 (2017), pp. 1340–1352.
  • B. ZHAO, L. FEI-FEI, AND E. P. XING, Online detection of unusual events in videos via dynamic sparse coding, in CVPR 2011, IEEE, 2011, pp. 3313–3320.
  • B. ZHAO, X. LI, AND X. LU, Hsa-rnn: Hierarchical structure-adaptive rnn for video summarization, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7405–7414.
  • B. ZHAO AND E. P. XING, Quasi real-time summarization for consumer videos, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 2513–2520.
  • K. ZHOU, Y. QIAO, AND T. XIANG, Deep reinforcement learning for un- supervised video summarization with diversity-representativeness reward, arXiv preprint arXiv:1801.00054, (2017).
  • Deep reinforcement learning for unsupervised video summarization with diversity-representativeness reward, in Thirty-Second AAAI Conference on Artificial Intelligence, 2018.

Subjects

ISSN: 2249-8958 (Online)
https://portal.issn.org/resource/ISSN/2249-8958#
Retrieval Number: 100.1/ijeat.F37370811622
https://www.ijeat.org/portfolio-item/F37370811622/
Journal Website: www.ijeat.org
https://www.ijeat.org
Publisher: Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP)
https://www.blueeyesintelligence.org