Screenomics: A Framework to Capture and Analyze Personal Life Experiences and the Ways that Technology Shapes Them

Byron Reeves, Nilam Ram, Thomas N. Robinson, James J. Cummings, C. Lee Giles, Jennifer Pan, Agnese Chiatti, Mj Cho, Katie Roehrick, Xiao Yang, Anupriya Gagneja, Miriam Brinberg, Daniel Muise, Yingdan Lu, Mufan Luo, Andrew Fitzgerald, Leo Yeykelis

Research output: Contribution to journalArticlepeer-review

86 Scopus citations


Digital experiences capture an increasingly large part of life, making them a preferred, if not required, method to describe and theorize about human behavior. Digital media also shape behavior by enabling people to switch between different content easily, and create unique threads of experiences that pass quickly through numerous information categories. Current methods of recording digital experiences provide only partial reconstructions of digital lives that weave–often within seconds–among multiple applications, locations, functions, and media. We describe an end-to-end system for capturing and analyzing the “screenome” of life in media, i.e., the record of individual experiences represented as a sequence of screens that people view and interact with over time. The system includes software that collects screenshots, extracts text and images, and allows searching of a screenshot database. We discuss how the system can be used to elaborate current theories about psychological processing of technology, and suggest new theoretical questions that are enabled by multiple timescale analyses. Capabilities of the system are highlighted with eight research examples that analyze screens from adults who have generated data within the system. We end with a discussion of future uses, limitations, theory, and privacy.

Original languageEnglish (US)
Pages (from-to)150-201
Number of pages52
JournalHuman-Computer Interaction
Issue number2
StatePublished - 2021

All Science Journal Classification (ASJC) codes

  • Applied Psychology
  • Human-Computer Interaction


Dive into the research topics of 'Screenomics: A Framework to Capture and Analyze Personal Life Experiences and the Ways that Technology Shapes Them'. Together they form a unique fingerprint.

Cite this