Comparative case studies of open source software peer review practices

Jing Wang, Patrick C. Shih, Yu Wu, John M. Carroll

Research output: Contribution to journalArticlepeer-review

16 Scopus citations


Context The power of open source software peer review lies in the involvement of virtual communities, especially users who typically do not have a formal role in the development process. As communities grow to a certain extent, how to organize and support the peer review process becomes increasingly challenging. A universal solution is likely to fail for communities with varying characteristics. Objective This paper investigates differences of peer review practices across different open source software communities, especially the ones engage distinct types of users, in order to offer contextualized guidance for developing open source software projects. Method Comparative case studies were conducted in two well-established large open source communities, Mozilla and Python, which engage extremely different types of users. Bug reports from their bug tracking systems were examined primarily, complemented by secondary sources such as meeting notes, blog posts, messages from mailing lists, and online documentations. Results The two communities differ in the key activities of peer review processes, including different characteristics with respect to bug reporting, design decision making, to patch development and review. Their variances also involve the designs of supporting technology. The results highlight the emerging role of triagers, who bridge the core and peripheral contributors and facilitate the peer review process. The two communities demonstrate alternative designs of open source software peer review and their tradeoffs were discussed. Conclusion It is concluded that contextualized designs of social and technological solutions to open source software peer review practices are important. The two cases can serve as learning resources for open source software projects, or other types of large software projects in general, to cope with challenges of leveraging enormous contributions and coordinating core developers. It is also important to improve support for triagers, who have not received much research effort yet.

Original languageEnglish (US)
Pages (from-to)1-12
Number of pages12
JournalInformation and Software Technology
StatePublished - Nov 1 2015

All Science Journal Classification (ASJC) codes

  • Software
  • Information Systems
  • Computer Science Applications


Dive into the research topics of 'Comparative case studies of open source software peer review practices'. Together they form a unique fingerprint.

Cite this