Can Lexical Sophistication and Cohesion Automatically Differentiate Student Engagement in Socio-technical Platforms?

This work aims to better analyze student engagement in socio-technical platforms by investigating whether the language students produce in online discussions is an indication of their cognitive engagement in collaborative activities. Primarily, this study evaluates whether a combination of linguistic features related to lexical sophistication and cohesion can capture students' cognitive engagement levels in an online course. We downloaded and annotated posts from the online platform for an undergraduate information sciences and technology course to create the human-coded dataset. Then, we assessed the lexical sophistication and cohesion of human-annotated posts and used lexical sophistication and cohesion indices in multivariate analysis of variance (MANOVA). A subsequent analysis using discriminant function analysis (DFA) suggested that the discriminant functions obtained from the human-annotated posts indicate a distinction between cognitive engagement categories. While the DFA model developed using cohesion indices shows a clear separation between cognitive engagement categories, the model built on lexical sophistication indices provides a partial separation. Study results suggest a promising approach for the application of linguistic features to support the categorization of discourse based on cognitive engagement.