TY - GEN
T1 - Dynamic Contract Design for Heterogenous Workers in Crowdsourcing for Quality Control
AU - Qiu, Chenxi
AU - Squicciarini, Anna Cinzia
AU - Rajtmajer, Sarah Michele
AU - Caverlee, James
N1 - Funding Information:
This work was supported in part by AFOSR grant FA9550-15-1-0149.
Publisher Copyright:
© 2017 IEEE.
PY - 2017/7/13
Y1 - 2017/7/13
N2 - Crowdsourcing sites heavily rely on paid workers to ensure completion of tasks. Yet, designing a pricing strategies able to incentivize users' quality and retention is non trivial. Existing payment strategies either simply set a fixed payment per task without considering changes in workers' behaviors, or rule out poor quality responses and workers based on coarse criteria. Hence, task requesters may be investing significantly in work that is inaccurate or even misleading. In this paper, we design a dynamic contract to incentivize high-quality work. Our proposed approach offers a theoretically proven algorithm to calculate the contract for each worker in a cost-efficient manner. In contrast to existing work, our contract design is not only adaptive to changes in workers' behavior, but also adjusts pricing policy in the presence of malicious behavior. Both theoretical and experimental analysis over real Amazon review traces show that our contract design can achieve a near optimal solution. Furthermore, experimental results demonstrate that our contract design 1) can promote high-quality work and prevent malicious behavior, and 2) outperforms the intuitive strategy of excluding all malicious workers in terms of the requester's utility.
AB - Crowdsourcing sites heavily rely on paid workers to ensure completion of tasks. Yet, designing a pricing strategies able to incentivize users' quality and retention is non trivial. Existing payment strategies either simply set a fixed payment per task without considering changes in workers' behaviors, or rule out poor quality responses and workers based on coarse criteria. Hence, task requesters may be investing significantly in work that is inaccurate or even misleading. In this paper, we design a dynamic contract to incentivize high-quality work. Our proposed approach offers a theoretically proven algorithm to calculate the contract for each worker in a cost-efficient manner. In contrast to existing work, our contract design is not only adaptive to changes in workers' behavior, but also adjusts pricing policy in the presence of malicious behavior. Both theoretical and experimental analysis over real Amazon review traces show that our contract design can achieve a near optimal solution. Furthermore, experimental results demonstrate that our contract design 1) can promote high-quality work and prevent malicious behavior, and 2) outperforms the intuitive strategy of excluding all malicious workers in terms of the requester's utility.
UR - http://www.scopus.com/inward/record.url?scp=85027266521&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85027266521&partnerID=8YFLogxK
U2 - 10.1109/ICDCS.2017.187
DO - 10.1109/ICDCS.2017.187
M3 - Conference contribution
AN - SCOPUS:85027266521
T3 - Proceedings - International Conference on Distributed Computing Systems
SP - 1168
EP - 1177
BT - Proceedings - IEEE 37th International Conference on Distributed Computing Systems, ICDCS 2017
A2 - Lee, Kisung
A2 - Liu, Ling
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 37th IEEE International Conference on Distributed Computing Systems, ICDCS 2017
Y2 - 5 June 2017 through 8 June 2017
ER -