Extending Action Recognition in the Compressed Domain

Research output: Chapter in Book/Report/Conference proceedingConference contribution


As the internet continues to extend its reach into every facet of society, video is becoming one of the most common mediums for the communication of ideas and information. Working with video in the compressed domain saves resources by avoiding decompression and allows for faster processing due to smaller input streams where redundancy is avoided. Most prior work on compressed video processing has focused on MPEG-4 part-2 codec which is dated in part due to its non-optimized compression ratio. For example, H.264 has a compression ratio 2x greater than MPEG-4 part-2 codec with improved quality. Due to the increasing prevalence of the more effective H.264 codec for video content, designing a network to infer directly on H.264 compressed video is essential. Hence, we propose a new video analytics architecture that uses only two streams of data from the compressed domain as compared to the three or more generally used for compressed recognition. The proposed architecture, coined Extended Codec Recognition Network (ECRN), is the first approach to our knowledge to support action recognition on both MPEG4 part-2 and H.264 compressed video. It is computationally efficient and achieves competitive accuracy to methods performing recognition solely on MPEG-4 part-2 streams. The ability to achieve competitive accuracy using a modern video codec creates the potential to extend compressed action recognition to a wide range of applications.

Original languageEnglish (US)
Title of host publicationProceedings - 36th International Conference on VLSI Design, VLSID 2023 - held concurrently with 22nd International Conference on Embedded Systems, ES 2023
PublisherIEEE Computer Society
Number of pages6
ISBN (Electronic)9798350346787
StatePublished - 2023
Event36th International Conference on VLSI Design, VLSID 2023 - Hyderabad, India
Duration: Jan 8 2023Jan 12 2023

Publication series

NameProceedings of the IEEE International Conference on VLSI Design
ISSN (Print)1063-9667


Conference36th International Conference on VLSI Design, VLSID 2023

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Electrical and Electronic Engineering

Cite this