Joint event detection and description in continuous video streams

Huijuan Xu; Boyang Li; Vasili Ramanishka; Leonid Sigal; Kate Saenko

doi:10.1109/WACVW.2019.00011

Joint event detection and description in continuous video streams

Huijuan Xu, Boyang Li, Vasili Ramanishka, Leonid Sigal, Kate Saenko

Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

33 Scopus citations

Abstract

Dense video captioning involves first localizing events in a video and then generating captions for the identified events. We present the Joint Event Detection and Description Network (JEDDi-Net) for solving this task in an end-to-end fashion, which encodes the input video stream with three-dimensional convolutional layers, proposes variable- length temporal events based on pooled features, and then uses a two-level hierarchical LSTM module with context modeling to transcribe the event proposals into captions. We show the effectiveness of our proposed JEDDi-Net on the large-scale ActivityNet Captions dataset.

Original language	English (US)
Title of host publication	Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	25-26
Number of pages	2
ISBN (Electronic)	9781728113920
DOIs	https://doi.org/10.1109/WACVW.2019.00011
State	Published - Feb 8 2019
Event	19th IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019 - Waikoloa Village, United States Duration: Jan 7 2019 → Jan 11 2019

Publication series

Name	Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019

Conference

Conference	19th IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019
Country/Territory	United States
City	Waikoloa Village
Period	1/7/19 → 1/11/19

All Science Journal Classification (ASJC) codes

Computer Science Applications
Computer Vision and Pattern Recognition

Access to Document

10.1109/WACVW.2019.00011

Cite this

Xu, H., Li, B., Ramanishka, V., Sigal, L., & Saenko, K. (2019). Joint event detection and description in continuous video streams. In Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019 (pp. 25-26). Article 8638336 (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACVW.2019.00011

Xu, Huijuan ; Li, Boyang ; Ramanishka, Vasili et al. / Joint event detection and description in continuous video streams. Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 25-26 (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019).

@inproceedings{d201f979e01447f385ae455eaadb9403,

title = "Joint event detection and description in continuous video streams",

abstract = "Dense video captioning involves first localizing events in a video and then generating captions for the identified events. We present the Joint Event Detection and Description Network (JEDDi-Net) for solving this task in an end-to-end fashion, which encodes the input video stream with three-dimensional convolutional layers, proposes variable- length temporal events based on pooled features, and then uses a two-level hierarchical LSTM module with context modeling to transcribe the event proposals into captions. We show the effectiveness of our proposed JEDDi-Net on the large-scale ActivityNet Captions dataset.",

author = "Huijuan Xu and Boyang Li and Vasili Ramanishka and Leonid Sigal and Kate Saenko",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 19th IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019 ; Conference date: 07-01-2019 Through 11-01-2019",

year = "2019",

month = feb,

day = "8",

doi = "10.1109/WACVW.2019.00011",

language = "English (US)",

series = "Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "25--26",

booktitle = "Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019",

address = "United States",

}

Xu, H, Li, B, Ramanishka, V, Sigal, L & Saenko, K 2019, Joint event detection and description in continuous video streams. in Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019., 8638336, Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019, Institute of Electrical and Electronics Engineers Inc., pp. 25-26, 19th IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019, Waikoloa Village, United States, 1/7/19. https://doi.org/10.1109/WACVW.2019.00011

Joint event detection and description in continuous video streams. / Xu, Huijuan; Li, Boyang; Ramanishka, Vasili et al.
Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 25-26 8638336 (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Joint event detection and description in continuous video streams

AU - Xu, Huijuan

AU - Li, Boyang

AU - Ramanishka, Vasili

AU - Sigal, Leonid

AU - Saenko, Kate

PY - 2019/2/8

Y1 - 2019/2/8

N2 - Dense video captioning involves first localizing events in a video and then generating captions for the identified events. We present the Joint Event Detection and Description Network (JEDDi-Net) for solving this task in an end-to-end fashion, which encodes the input video stream with three-dimensional convolutional layers, proposes variable- length temporal events based on pooled features, and then uses a two-level hierarchical LSTM module with context modeling to transcribe the event proposals into captions. We show the effectiveness of our proposed JEDDi-Net on the large-scale ActivityNet Captions dataset.

AB - Dense video captioning involves first localizing events in a video and then generating captions for the identified events. We present the Joint Event Detection and Description Network (JEDDi-Net) for solving this task in an end-to-end fashion, which encodes the input video stream with three-dimensional convolutional layers, proposes variable- length temporal events based on pooled features, and then uses a two-level hierarchical LSTM module with context modeling to transcribe the event proposals into captions. We show the effectiveness of our proposed JEDDi-Net on the large-scale ActivityNet Captions dataset.

UR - http://www.scopus.com/inward/record.url?scp=85063062041&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063062041&partnerID=8YFLogxK

U2 - 10.1109/WACVW.2019.00011

DO - 10.1109/WACVW.2019.00011

M3 - Conference contribution

AN - SCOPUS:85063062041

T3 - Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019

SP - 25

EP - 26

BT - Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 19th IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019

Y2 - 7 January 2019 through 11 January 2019

ER -

Xu H, Li B, Ramanishka V, Sigal L, Saenko K. Joint event detection and description in continuous video streams. In Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 25-26. 8638336. (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019). doi: 10.1109/WACVW.2019.00011

Joint event detection and description in continuous video streams

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Cite this