HDBTracker: Monitoring the aggregates on dynamic hidden web databases

Weimo Liu, Saad Bin Suhaim, Saravanan Thirumuruganathan, Nan Zhang, Gautam Das, Ali Jaoua

Research output: Contribution to journalConference articlepeer-review

2 Scopus citations


Numerous web databases, e.g., amazon.com, eBay.com, are "hidden" behind (i.e., accessible only through) their restrictive search and browsing interfaces. This demonstration showcases HDBTracker, a web-based system that reveals and tracks (the changes of) userspecified aggregate queries over such hidden web databases, especially those that are frequently updated, by issuing a small number of search queries through the public web interfaces of these databases. The ability to track and monitor aggregates has applications over a wide variety of domains - e.g., government agencies can track COUNT of openings at online job hunting websites to understand key economic indicators, while businesses can track the AVG price of a product over a basket of e-commerce websites to understand the competitive landscape and/or material costs. A key technique used in HDBTracker is RS-ESTIMATOR, the first algorithm that can efficiently monitor changes to aggregate query answers over a hidden web database.

Original languageEnglish (US)
Pages (from-to)1569-1572
Number of pages4
JournalProceedings of the VLDB Endowment
Issue number13
StatePublished - 2014
EventProceedings of the 40th International Conference on Very Large Data Bases, VLDB 2014 - Hangzhou, China
Duration: Sep 1 2014Sep 5 2014

All Science Journal Classification (ASJC) codes

  • Computer Science (miscellaneous)
  • General Computer Science


Dive into the research topics of 'HDBTracker: Monitoring the aggregates on dynamic hidden web databases'. Together they form a unique fingerprint.

Cite this