IProWA: A Novel Probabilistic Graphical Model for Crowdsourcing Aggregation

Tianqi Wang, Houping Xiao, Fenglong Ma, Jing Gao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations


Crowdsourcing has become a popular way to obtain a large volume of labeled data due to its low cost and high efficiency. Usually the crowdsourcing process enables redundancy in the collected labels in order to ensure the correctness of item labels. However, workers on the crowdsourcing platform may make mistakes on some items, leading to inconsistent labels. In this case, it is important to aggregate these noisy labels and obtain the true labels of the items. The correctness of the item label provided by a worker depends on both the worker's ability and the property of the item. However, most of the existing models consider the effect of workers' abilities but ignore that of the item properties. In this paper, we propose a novel crowdsourcing aggregation method (IProWA) which incorporates the modeling of not only worker expertise level but also item property. In particular, items are represented by a K dimensional vector (i.e., item parameter), where K is the number of possible categories and each dimension represents a category. The proposed model transforms the true label estimation into the estimation of item parameters as it connects the true label and the parameters of an item. In worker modeling, it models the different category propensities among different worker groups. Experimental results show that the performance of the proposed model is comparable to that of the state-of-the-art baselines and the learned item parameters can help interpret the property of that item.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE International Conference on Big Data, Big Data 2019
EditorsChaitanya Baru, Jun Huan, Latifur Khan, Xiaohua Tony Hu, Ronay Ak, Yuanyuan Tian, Roger Barga, Carlo Zaniolo, Kisung Lee, Yanfang Fanny Ye
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781728108582
StatePublished - Dec 2019
Event2019 IEEE International Conference on Big Data, Big Data 2019 - Los Angeles, United States
Duration: Dec 9 2019Dec 12 2019

Publication series

NameProceedings - 2019 IEEE International Conference on Big Data, Big Data 2019


Conference2019 IEEE International Conference on Big Data, Big Data 2019
Country/TerritoryUnited States
CityLos Angeles

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Networks and Communications
  • Information Systems
  • Information Systems and Management


Dive into the research topics of 'IProWA: A Novel Probabilistic Graphical Model for Crowdsourcing Aggregation'. Together they form a unique fingerprint.

Cite this