Beyond remote control: Exploring natural gesture inputs for smart TV systems

Huiyue Wu, Liuqingqing Yang, Shengqian Fu, Xiaolong Luke Zhang

Research output: Contribution to journalArticlepeer-review

10 Scopus citations


This paper reports on work that explores natural gesture inputs from end-users for television use in a typical living room setting. First, we derive a set of 19 user-defined freehand gestures for regular TV control tasks. This study helps us to determine the gestures preferred by TV viewers and reduce the risk of developing an unusable, ineffective system. Then, based on this user-defined gesture set, we propose a unified framework to address specific problems in a complex real-world TV viewing environment, including 1) the automatic exclusion of many meaningless daily actions by TV viewers, 2) the capability to recognize both static and dynamic gestures simultaneously, as well as one- and two-handed gestures simultaneously, and 3) the continuous recognition of multiple dynamic gestures in the air (e.g., a channel switching gesture for channel 127). Experimental results show that our approach allows users to interact with TV-based applications more flexibly and effectively, with improved user experience and user satisfaction. Finally, we highlight the implications of our work for the design and development of related freehand gesture applications.

Original languageEnglish (US)
Pages (from-to)335-354
Number of pages20
JournalJournal of Ambient Intelligence and Smart Environments
Issue number4
StatePublished - 2019

All Science Journal Classification (ASJC) codes

  • Software


Dive into the research topics of 'Beyond remote control: Exploring natural gesture inputs for smart TV systems'. Together they form a unique fingerprint.

Cite this