WE WRITE AUTOMATED SCORING: USING CHATGPT FOR SCORING IN LARGE-SCALE WRITING RESEARCH PROJECTS

Kausalai Kay Wijekumar; Debra McKeown; Shuai Zhang; Pui Wa Lei; Nikolaus Hruska; Pablo Pirnay-Dummer

doi:10.4324/9781003397618-12

WE WRITE AUTOMATED SCORING: USING CHATGPT FOR SCORING IN LARGE-SCALE WRITING RESEARCH PROJECTS

Kausalai Kay Wijekumar
, Debra McKeown
, Shuai Zhang
, Pui Wa Lei
, Nikolaus Hruska
, Pablo Pirnay-Dummer

Educational Psychology, Counseling, and Special Education

Research output: Chapter in Book/Report/Conference proceeding › Chapter

4 Scopus citations

Abstract

This chapter describes an approach to developing and testing a novel automated scoring model that leveraged ChatGPT to evaluate upper elementary-grade persuasive writing within the context of an efficacy evaluation of the We Write intervention. In addition to describing this scoring approach, this chapter presents results comparing the performance of ChatGPT and human scores. The chapter concludes with a discussion of findings and directions for future research related to the application of ChatGPT and large language models to the automated scoring of writing.

Original language	English (US)
Title of host publication	The Routledge International Handbook of Automated Essay Evaluation
Publisher	Taylor and Francis
Pages	178-194
Number of pages	17
ISBN (Electronic)	9781040033241
ISBN (Print)	9781032502564
DOIs	https://doi.org/10.4324/9781003397618-12
State	Published - Jan 1 2024

All Science Journal Classification (ASJC) codes

General Arts and Humanities
General Social Sciences
General Psychology
General Computer Science

Access to Document

10.4324/9781003397618-12

Cite this

@inbook{42ab70e5b02343c19e3af54e84d0ad64,

title = "WE WRITE AUTOMATED SCORING: USING CHATGPT FOR SCORING IN LARGE-SCALE WRITING RESEARCH PROJECTS",

abstract = "This chapter describes an approach to developing and testing a novel automated scoring model that leveraged ChatGPT to evaluate upper elementary-grade persuasive writing within the context of an efficacy evaluation of the We Write intervention. In addition to describing this scoring approach, this chapter presents results comparing the performance of ChatGPT and human scores. The chapter concludes with a discussion of findings and directions for future research related to the application of ChatGPT and large language models to the automated scoring of writing.",

author = "Wijekumar, \{Kausalai Kay\} and Debra McKeown and Shuai Zhang and Lei, \{Pui Wa\} and Nikolaus Hruska and Pablo Pirnay-Dummer",

note = "Publisher Copyright: {\textcopyright} 2024 selection and editorial matter, Mark D. Shermis and Joshua Wilson; individual chapters, the contributors.",

year = "2024",

month = jan,

day = "1",

doi = "10.4324/9781003397618-12",

language = "English (US)",

isbn = "9781032502564",

pages = "178--194",

booktitle = "The Routledge International Handbook of Automated Essay Evaluation",

publisher = "Taylor and Francis",

address = "United States",

}

TY - CHAP

T1 - WE WRITE AUTOMATED SCORING

T2 - USING CHATGPT FOR SCORING IN LARGE-SCALE WRITING RESEARCH PROJECTS

AU - Wijekumar, Kausalai Kay

AU - McKeown, Debra

AU - Zhang, Shuai

AU - Lei, Pui Wa

AU - Hruska, Nikolaus

AU - Pirnay-Dummer, Pablo

PY - 2024/1/1

Y1 - 2024/1/1

N2 - This chapter describes an approach to developing and testing a novel automated scoring model that leveraged ChatGPT to evaluate upper elementary-grade persuasive writing within the context of an efficacy evaluation of the We Write intervention. In addition to describing this scoring approach, this chapter presents results comparing the performance of ChatGPT and human scores. The chapter concludes with a discussion of findings and directions for future research related to the application of ChatGPT and large language models to the automated scoring of writing.

AB - This chapter describes an approach to developing and testing a novel automated scoring model that leveraged ChatGPT to evaluate upper elementary-grade persuasive writing within the context of an efficacy evaluation of the We Write intervention. In addition to describing this scoring approach, this chapter presents results comparing the performance of ChatGPT and human scores. The chapter concludes with a discussion of findings and directions for future research related to the application of ChatGPT and large language models to the automated scoring of writing.

UR - https://www.scopus.com/pages/publications/85195766558

UR - https://www.scopus.com/pages/publications/85195766558#tab=citedBy

U2 - 10.4324/9781003397618-12

DO - 10.4324/9781003397618-12

M3 - Chapter

AN - SCOPUS:85195766558

SN - 9781032502564

SP - 178

EP - 194

BT - The Routledge International Handbook of Automated Essay Evaluation

PB - Taylor and Francis

ER -

WE WRITE AUTOMATED SCORING: USING CHATGPT FOR SCORING IN LARGE-SCALE WRITING RESEARCH PROJECTS

Abstract

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this