Search

Overview of Automated Evaluation Systems for Second Language Writing: A Study of Domestic and International Research

Author(s)

Qianxue Zhao

Affiliation(s)

School of Humanities and Social Sciences, Xi'an Polytechnic University, Xi'an, Shaanxi, China

Abstract

The first automated essay scoring system was developed 50 years ago. Automated writing evaluation systems are evolving into feature-rich platforms, moving beyond simple scoring mechanisms. This paper reviews and analyzes the current state of English writing automated evaluation systems through a survey of related literature and studies. It introduces common English writing evaluation systems and websites, explaining their basic principles, technical characteristics, and application directions. Additionally, it reviews recent case studies of automated scoring systems that provide feedback. This critical literature review aims to extract insights and suggests a dynamic combination of human and machine evaluation for second language writing assessment.

Keywords

Automated Essay Scoring System; Second Language Writing; Research and Application

References

[1]Page, E.B. Grading essays by computer: Progress report. In Educational Testing Service (Ed.), Proceedings of the Invitational Conference on Testing Problems. New York City: Princeton, NJ: Educational Testing Service, 1966:87 -10. [2]Taghipour, K., & Ng, H.T. A neural approach to automated essay scoring. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016: 1882-1891. [3]Dong, F., Zhang, Y., & Yang, J. Attention-based recurrent convolutional neural network for automatic essay scoring. Proceedings of the 21st Conference on Computational Natural Language Learning (CONLL 2017), 2017: 153-162. [4]Ridley, R., He, L., Dai, X., Huang, S., & Chen, J. Automated cross-prompt scoring of essay traits. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35: 13745-13753. [5]ALharbi, W. AI in the foreign language classroom: A pedagogical overview of automated writing assistance tools. Education Research International, 2023: 1-15. [6]Wei, P., Wang, X., & Dong, H. The impact of automated writing evaluation on second language writing skills of Chinese EFL learners: a randomized controlled trial. Frontiers in Psychology, 2023, 14: 1249991. [7]Zhao, J., Li, Y., & Feng, W. Investigating the validity and reliability of a comprehensive essay evaluation model of integrating manual feedback and intelligent assistance. International Journal of Emerging Technologies in Learning (Online), 2023, 18(4): 248-261. [8]Mohsen, M. A., & Abdulaziz, A. The effectiveness of using a hybrid mode of automated writing evaluation system on EFL students’ writing. Teaching English with Technology, 2019, 19(1): 118-131. [9]Burstein, J. The E-rater scoring engine: Automated essay scoring with natural language processing. Shermis, M. D., & Burstein, I. Automated essay scoring: a cross-disciplinary perspective. Mahwah, NJ: Lawrence Erlbaum Associates, 2003. [10]Yao, D. Automated writing evaluation for ESL learners: A case study of Pigai system. Journal of Asia TEFL, 2021, 18(3): 949-958. [11]Huang, Y., & Wilson, J. Using automated feedback to develop writing proficiency. Computers and Composition, 2021, 62. [12]Wang, I. X., Wu, X., Coates, E., Zeng, M., Kuang, J., Liu, S., ... & Park, J. Neural Automated Writing Evaluation with Corrective Feedback. arXiv preprint arXiv:2402.17613, 2024. [13]Xu, J., & Zhang, S. Understanding AWE feedback and English writing of learners with different proficiency levels in an EFL classroom: A sociocultural perspective. The Asia-Pacific Education Researcher, 2022, 31(4): 357-367. [14]Ke, Z., Carlile, W., Gurrapadi, N., & Ng, V. Learning to Give Feedback: Modeling Attributes Affecting Argument Persuasiveness in Student Essays. IJCAI, 2018: 4130-4136. [15]Ngo, T. T. N., Chen, H. H. J., & Lai, K. K. W. The effectiveness of automated writing evaluation in EFL/ESL writing: A three-level meta-analysis. Interactive Learning Environments, 2022: 1–18. [16]Li Yanling, Tian Xiachun. Reliability Study of iWrite 2.0 Online English Composition Scoring. Modern Educational Technology, 2018, 28(2): 75-80. [17]Bai Lifang. Validity Study and Usage Suggestions of Automated Scoring Systems in Formative Testing. Contemporary Foreign Language Education, 2023(0): 17-28. [18] Hu Xuewen. The Impact of Online Composition Self-Revision on College Students’ English Writing Results. Foreign Language Educational Technology, 2015(3): 45-49. [19] Cao Ting. The Impact of Feedback from Automated Composition Scoring Systems on the Syntactic Complexity of Students at Different Levels: A Case Study of Pigai. Examination and Evaluation (College English Research Edition), 2018(5): 94-101. [20] Li Guangfeng. A Study on the Impact of Multiple Feedback Based on Automated Evaluation Systems on English Composition Revision. Foreign Language Teaching, 2019, 40(4): 72-76. [21] Li Yihua. An Investigative Analysis of Learners’ Acceptance of Teachers' Electronic Feedback in English Writing. Language Education, 2016, 4(1): 34-41. [22] Li Guangfeng, Wang Nannan. Research on Feedback Effectiveness: Influencing Factors and Teaching Strategies. Theory and Practice of Education, 2018, 38(19): 57-60.