ACES: Automated Academic Essay Scoring Using a Natural Language Processing-Based Regression Mechanism

Abstract

Academic essays are essential testing instruments that evaluate the students’ ability to organize thoughts and synthesize information. However, grading them is an exhausting and cumbersome process that requires considerable manpower. It may be prone to errors, and there are also serious concerns about fairness, such that an essay graded B+ today may be graded B- tomorrow by the same checker. Therefore, the author proposes ACES, an essay scoring mechanism that employs natural language processing (NLP) to address the issue at hand. NLP is a sub-field of artificial intelligence (AI) concerned with granting computers the ability to understand texts in much the same way humans can. With essay scoring reformulated as a regression problem, ACES takes the essay answer as the input, converts it to a vector representation of numbers in the embedding space, and feeds it to the neural network model (which serves as the approximate regression function) to predict its score as the output. In this paper, the author successfully implements four versions of ACES that employ different embedding sources and neural network models, with the ACES variant that considers context and word frequency information performing the best (i.e., ACES-BERT).