Object identification with Pay-As-You-Go crowdsourcing

The conventional crowdsourcing paradigm requires an explicit task description and payment scheme. Requesters can then easily determine whether the crowdsourced results are satisfactory, and workers will have a fairly clear expectation of the monetary reward once the task is accomplished. However, such a paradigm becomes problematic when it is applied to Object Identification (OI) tasks. First, for OI tasks, it is difficult for requesters to evaluate whether sufficient numbers of objects have been found by an individual worker, warranting payment. Second, the same objects can be detected by many workers and ending up being unnecessary workload and inefficient performance. In this paper, we design a new crowdsourcing paradigm for OI tasks. Designing such a paradigm is challenging. Firstly, an easily-detected object can be found by multiple workers, which leads to an unfair situation that the requester has to make extra payments for the duplication. Secondly, there is usually a time limit to finish the overall crowdsourcing process, which demands efficient assignment strategy. To address these challenges, we propose solutions to achieve fairness by a Pay-As-You-Go (PAYG) mechanism and efficiency by a new worker-assignment scheme, Adaptive Worker Assignment (AWA). Extensive experiments are conducted to demonstrate the advantages of this new paradigm.