TY - GEN
T1 - A Prototype Hybrid Prediction Market for Estimating Replicability of Published Work
AU - Chakravorti, Tatiana
AU - Fraleigh, Robert
AU - Fritton, Timothy
AU - McLaughlin, Michael
AU - Singh, Vaibhav
AU - Griffin, Christopher
AU - Kwasnica, Anthony
AU - Pennock, David
AU - Giles, C. Lee
AU - Rajtmajer, Sarah
N1 - Publisher Copyright:
© 2023 The Authors.
PY - 2023/6/22
Y1 - 2023/6/22
N2 - We present a prototype hybrid prediction market and demonstrate the avenue it represents for meaningful human-AI collaboration. We build on prior work proposing artificial prediction markets as a novel machine learning algorithm. In an artificial prediction market, trained AI agents (bot traders) buy and sell outcomes of future events. Classification decisions can be framed as outcomes of future events, and accordingly, the price of an asset corresponding to a given classification outcome can be taken as a proxy for the systems confidence in that decision. By embedding human participants in these markets alongside bot traders, we can bring together insights from both. In this paper, we detail pilot studies with prototype hybrid markets for the prediction of replication study outcomes. We highlight challenges and opportunities, share insights from semi-structured interviews with hybrid market participants, and outline a vision for ongoing and future work.
AB - We present a prototype hybrid prediction market and demonstrate the avenue it represents for meaningful human-AI collaboration. We build on prior work proposing artificial prediction markets as a novel machine learning algorithm. In an artificial prediction market, trained AI agents (bot traders) buy and sell outcomes of future events. Classification decisions can be framed as outcomes of future events, and accordingly, the price of an asset corresponding to a given classification outcome can be taken as a proxy for the systems confidence in that decision. By embedding human participants in these markets alongside bot traders, we can bring together insights from both. In this paper, we detail pilot studies with prototype hybrid markets for the prediction of replication study outcomes. We highlight challenges and opportunities, share insights from semi-structured interviews with hybrid market participants, and outline a vision for ongoing and future work.
UR - http://www.scopus.com/inward/record.url?scp=85171430967&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85171430967&partnerID=8YFLogxK
U2 - 10.3233/FAIA230093
DO - 10.3233/FAIA230093
M3 - Conference contribution
AN - SCOPUS:85171430967
T3 - Frontiers in Artificial Intelligence and Applications
SP - 300
EP - 309
BT - HHAI 2023
A2 - Lukowicz, Paul
A2 - Mayer, Sven
A2 - Koch, Janin
A2 - Shawe-Taylor, John
A2 - Tiddi, Ilaria
PB - IOS Press BV
T2 - 2nd International Conference on Hybrid Human-Artificial Intelligence, HHAI 2023
Y2 - 26 June 2023 through 30 June 2023
ER -