A Prototype Hybrid Prediction Market for Estimating Replicability of Published Work

Tatiana Chakravorti, Robert Fraleigh, Timothy Fritton, Michael McLaughlin, Vaibhav Singh, Christopher Griffin, Anthony Kwasnica, David Pennock, C. Lee Giles, Sarah Rajtmajer

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We present a prototype hybrid prediction market and demonstrate the avenue it represents for meaningful human-AI collaboration. We build on prior work proposing artificial prediction markets as a novel machine learning algorithm. In an artificial prediction market, trained AI agents (bot traders) buy and sell outcomes of future events. Classification decisions can be framed as outcomes of future events, and accordingly, the price of an asset corresponding to a given classification outcome can be taken as a proxy for the systems confidence in that decision. By embedding human participants in these markets alongside bot traders, we can bring together insights from both. In this paper, we detail pilot studies with prototype hybrid markets for the prediction of replication study outcomes. We highlight challenges and opportunities, share insights from semi-structured interviews with hybrid market participants, and outline a vision for ongoing and future work.

Original languageEnglish (US)
Title of host publicationHHAI 2023
Subtitle of host publicationAugmenting Human Intellect - Proceedings of the 2nd International Conference on Hybrid Human-Artificial Intelligence
EditorsPaul Lukowicz, Sven Mayer, Janin Koch, John Shawe-Taylor, Ilaria Tiddi
PublisherIOS Press BV
Number of pages10
ISBN (Electronic)9781643683942
StatePublished - Jun 22 2023
Event2nd International Conference on Hybrid Human-Artificial Intelligence, HHAI 2023 - Munich, Germany
Duration: Jun 26 2023Jun 30 2023

Publication series

NameFrontiers in Artificial Intelligence and Applications
ISSN (Print)0922-6389
ISSN (Electronic)1879-8314


Conference2nd International Conference on Hybrid Human-Artificial Intelligence, HHAI 2023

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence

Cite this