Automated essay scoring using transformer models

Mayer, Christian ; Ludwig, Sabrina

Document Type: Conference presentation
Year of publication: 2022
Conference title: JURE 2022
Location of the conference venue: Porto, Portugal
Date of the conference: 18.-22.07.2022
Related URLs:
Publication language: English
Institution: Business School > Berufs- u. Wirtschaftspädagogik (Rausch 2016-)
Business School > Wirtschaftspädagogik II (Seifried 2012-)
Subject: 004 Computer science, internet
330 Economics
370 Education
Keywords (English): artificial intelligence , assessment methods and tools , problem solving , vocational education
Abstract: Automated Essay Scoring (AES) of open-ended text responses is becoming increasingly popular in the education sector since it significantly reduces the effort of human scoring. Natural language processing that relies on machine learning is particularly promising for text classification and AES. Previous AES approaches are mostly based on a bag-of-words (BOW) approach or recurrent neural networks (RNNs). However, with the introduction of "transformer" models by Vaswani et al. (2017), these models have taken a leading role in virtually all areas of language processing. We applied the latest transformer-based approaches for natural language processing (NLP) based on machine learning algorithms to assess student textual answers. For doing so, we used a dataset of 2,088 email responses written by learners to a problem-solving task and manually labeled in terms of politeness. We show that this approach outperforms a traditional bag of words (BOW) approach for politeness classification. We argue that for AES tasks such as politeness classification, the transformer-based approach has clear benefits, while a BOW approach suffers from the fact that it does not consider word order and reduces the words to their stem.

Metadata export


+ Search Authors in

+ Page Views

Hits per month over past year

Detailed information

You have found an error? Please let us know about your desired correction here: E-Mail

Actions (login required)

Show item Show item