Capturing narratives of graduate engineering attrition through online forum mining

This research paper presents methods by which researchers can harvest data from social media forums as a way to gain insight on sensitive issues or populations. In the present research, we are interested in studying doctoral attrition, which is a complex and multifaceted phenomenon that poses practical significance to funding agencies, advisors, and students themselves. Sampling non-completers is difficult, and researchers generally find it difficult to collect nationwide narratives of attrition. This paper presents a novel method for studying attrition using the publicly-available online forum Reddit.com to collect first-hand accounts and authentic narratives of attrition. These often-anonymous online discussions offer a unique view into the authentic thoughts of engineering graduate students considering leaving their program, throughout the decision-making process. This paper proposes a method to efficiently collect and parse open-source information into coherent narratives across "posts" or "threads" of conversation using data mining tools. The underlying methodology developed is based on achieving a holistic view of the discourse patterns and authentic narratives surrounding attrition, which in turn allows researchers to capture meaningful, authentic, and credible emergent themes unbiased by social response. We present a short summary of results to show the dominant narratives of attrition achieved through this method; however, the main focus of this paper is to present the method itself, which has the potential to be extended and modified to aid in other large data mining efforts to answer other research questions related to sensitive topics.