Aggregate estimation over dynamic hidden web databases

Weimo Liu, Saravanan Thirumuruganathan, Nan Zhang, Gautam Das

Research output: Contribution to journalConference articlepeer-review

13 Scopus citations


Many databases on the web are "hidden" behind (i.e., accessible only through) their restrictive, form-like, search interfaces. Recent studies have shown that it is possible to estimate aggregate query answers over such hidden web databases by issuing a small number of carefully designed search queries through the restrictive web interface. A problem with these existing work, however, is that they all assume the underlying database to be static, while most realworld web databases (e.g., Amazon, eBay) are frequently updated. In this paper, we study the novel problem of estimating/tracking aggregates over dynamic hidden web databases while adhering to the stringent query-cost limitation they enforce (e.g., at most 1,000 search queries per day). Theoretical analysis and extensive realworld experiments demonstrate the effectiveness of our proposed algorithms and their superiority over baseline solutions (e.g., the repeated execution of algorithms designed for static web databases).

Original languageEnglish (US)
Pages (from-to)1107-1118
Number of pages12
JournalProceedings of the VLDB Endowment
Issue number12
StatePublished - 2014
EventProceedings of the 40th International Conference on Very Large Data Bases, VLDB 2014 - Hangzhou, China
Duration: Sep 1 2014Sep 5 2014

All Science Journal Classification (ASJC) codes

  • Computer Science (miscellaneous)
  • General Computer Science


Dive into the research topics of 'Aggregate estimation over dynamic hidden web databases'. Together they form a unique fingerprint.

Cite this