Prediction in the Presence of Response-Dependent Missing Labels

Hyebin Song, Garvesh Raskutti, Rebecca Willett

Research output: Chapter in Book/Report/Conference proceedingConference contribution


In various settings, limitations of sensing technologies or other sampling mechanisms result in missing labels, where the likelihood of a missing label is an unknown function of the data. For example, satellites used to detect forest fires cannot sense fires below a certain size threshold. In such cases, training datasets consist of positive and pseudo-negative observations (true negatives or undetected positives with small magnitudes). We develop a new methodology and non-convex algorithm which jointly estimates the magnitude and occurrence of events, utilizing prior knowledge of the detection mechanism. We provide conditions under which our model is identifiable. We prove that even though our approach leads to a non-convex objective, any local minimizer has an optimal statistical error (up to a log term) and the projected gradient descent algorithm has geometric convergence rates. We demonstrate on both synthetic data and a California wildfire dataset that our method outperforms existing state-of-the-art approaches.

Original languageEnglish (US)
Title of host publication2021 IEEE Statistical Signal Processing Workshop, SSP 2021
PublisherIEEE Computer Society
Number of pages5
ISBN (Electronic)9781728157672
StatePublished - Jul 11 2021
Event21st IEEE Statistical Signal Processing Workshop, SSP 2021 - Virtual, Rio de Janeiro, Brazil
Duration: Jul 11 2021Jul 14 2021

Publication series

NameIEEE Workshop on Statistical Signal Processing Proceedings


Conference21st IEEE Statistical Signal Processing Workshop, SSP 2021
CityVirtual, Rio de Janeiro

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering
  • Applied Mathematics
  • Signal Processing
  • Computer Science Applications


Dive into the research topics of 'Prediction in the Presence of Response-Dependent Missing Labels'. Together they form a unique fingerprint.

Cite this