TY - GEN
T1 - Experimenting with latent semantic analysis and latent dirichlet allocation on automated essay grading
AU - Hoblos, Jalaa
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/12/14
Y1 - 2020/12/14
N2 - The demand of scoring natural language responses has created a need for new computational tools that can be applied to automatically grade student essays. Systems for automatic essay assessment have been commercially available since 1990's. However, the progress in the field was obstructed by a lack of qualitative information regarding the effectiveness of such systems. Most of the research in automatic essay grading has been associated with English writing due to its widespread use and the availability of more learner collection and language processing software for the language. In addition, there is large number of commercial software for grading programming assignments automatically. In this work, we investigate document semantic similarity based on Latent Semantic Analysis (LSA) and on Latent Dirichlet Allocation (LDA). We use an open-source Python software, Gensim, to develop and implement an essay grading system able to compare an essay to an answer-key and assign to it a grade based on semantic similarity between the two. We test our tool on variable-size essays and conduct experimental tests to compare the results obtained from human grader (professor) and those obtained from the automatic grading system. Results show high correlation between the professor grades and the grades assigned by both modeling techniques. However, LSA-based modeling showed more promising results than the LDA-based method.
AB - The demand of scoring natural language responses has created a need for new computational tools that can be applied to automatically grade student essays. Systems for automatic essay assessment have been commercially available since 1990's. However, the progress in the field was obstructed by a lack of qualitative information regarding the effectiveness of such systems. Most of the research in automatic essay grading has been associated with English writing due to its widespread use and the availability of more learner collection and language processing software for the language. In addition, there is large number of commercial software for grading programming assignments automatically. In this work, we investigate document semantic similarity based on Latent Semantic Analysis (LSA) and on Latent Dirichlet Allocation (LDA). We use an open-source Python software, Gensim, to develop and implement an essay grading system able to compare an essay to an answer-key and assign to it a grade based on semantic similarity between the two. We test our tool on variable-size essays and conduct experimental tests to compare the results obtained from human grader (professor) and those obtained from the automatic grading system. Results show high correlation between the professor grades and the grades assigned by both modeling techniques. However, LSA-based modeling showed more promising results than the LDA-based method.
UR - http://www.scopus.com/inward/record.url?scp=85100892118&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85100892118&partnerID=8YFLogxK
U2 - 10.1109/SNAMS52053.2020.9336533
DO - 10.1109/SNAMS52053.2020.9336533
M3 - Conference contribution
AN - SCOPUS:85100892118
T3 - 2020 7th International Conference on Social Network Analysis, Management and Security, SNAMS 2020
BT - 2020 7th International Conference on Social Network Analysis, Management and Security, SNAMS 2020
A2 - Guetl, Christian
A2 - Saleh, Imad
A2 - Caravolo, Paolo
A2 - Jararweh, Yaser
A2 - Benkhelifa, Elhadj
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 7th International Conference on Social Network Analysis, Management and Security, SNAMS 2020
Y2 - 14 December 2020 through 16 December 2020
ER -