artificial intelligence , assessment methods and tools , problem solving , vocational education
Abstract:
Automated Essay Scoring (AES) of open-ended text responses is becoming increasingly popular in the education sector since it significantly reduces the effort of human scoring. Natural language processing that relies on machine learning is particularly promising for text classification and AES. Previous AES approaches are mostly based on a bag-of-words (BOW) approach or recurrent neural networks (RNNs). However, with the introduction of "transformer" models by Vaswani et al. (2017), these models have taken a leading role in virtually all areas of language processing.
We applied the latest transformer-based approaches for natural language processing (NLP) based on machine learning algorithms to assess student textual answers. For doing so, we used a dataset of 2,088 email responses written by learners to a problem-solving task and manually labeled in terms of politeness. We show that this approach outperforms a traditional bag of words (BOW) approach for politeness classification. We argue that for AES tasks such as politeness classification, the transformer-based approach has clear benefits, while a BOW approach suffers from the fact that it does not consider word order and reduces the words to their stem.
Search Authors in
