Disruptions in education are not only good; they’re essential to drive innovation. From Covid-19 to gamification, education and assessment – in general – have seen a significant shift in how it does business.
And then along came AI. Although nothing new, its implications for educating and assessing students are nothing short of revolutionary, especially with the advent of chatbots like ChatGPT. Sure, the risk of cheating is much greater, yet this concern can be eliminated if we have robust safeguards to prevent students from exploiting AI for dishonest purposes. Once these are in place, we can then turn our attention to the boundless potential of AI – making institutions more efficient, reliable, and cost-effective.
In this blog, we explore the upsides to AI in e-assessment – from its humble beginnings as a simple tutor to its capabilities to construct entire high-stakes exams from item generation to essay marking.
Contents
The Emergence of AI in E-Assessment
Integrating AI in education and assessment has been a steady and evolutionary process. In the late 20th century, AI started to dip its toe in education with PLATO, an early AI platform that offered a range of lessons from reading to college-level engineering, as far back as the 1960s. In the 70’s and 80’s, AI tutors began to emerge to help students alongside human teachers. In the 90’s, auto-scoring essays became possible, allowing for punctuation, grammar, and spelling errors to be recognized.
In 2000 one of the key advancements was the development of natural language processing (NLP) algorithms, which have made it possible to auto-grade essays and open-ended questions, thereby making the assessment process more efficient. AI has also made possible the dynamic generation of assessment items, creating a virtually infinite pool of questions to assess students, which we will cover in this blog.
Chatbots and AI in E-Assessment: An Introduction to ChatGPT
In our last blog, we discussed possible pitfalls of AI chatbots such as ChatGPT, the biggest concern being students’ ability to cheat on exams or plagiarize assignments using AI. But to ignore its positive possibilities would be a disservice to education.
ChatGPT is a powerful AI model developed by OpenAI that uses machine learning to generate human-like text based on input from a student – in either question format or a writing prompt. In education, it can function as an advanced chatbot capable of facilitating interactive learning and assessment. Its most significant strength lies in its ability to mimic human-like conversation, enabling personalized learning experiences. Students can converse with the AI, ask questions, and receive tailored responses that suit their learning pace and style. This makes it a versatile tool for e-assessments, offering the opportunity for open-ended questions and complex problem-solving exercises that go beyond traditional multiple-choice questionnaires.
Let’s explore several exciting opportunities for e-assessment:
AI-Driven Dynamic Item Generation
Dynamic Item Generation is a method that instantly creates test questions, or ‘assessment items,’ according to certain rules and templates. Traditionally, creating a varied set of questions involves a detailed process, including designing the exam content, getting it checked by experts in the subject, testing it out, hiring psychometricians, among many other processes. All these steps are to ensure the test is valid, reliable, and fair, however, because this process is so complex and involved, it can also be quite expensive.
AI, using smart machine learning methods and language understanding, can create banks of questions for exams. These systems work by processing large volumes of educational content related to the exam subject. They analyze patterns in question types, structure, and content, and use this data to generate new, original questions. AI can also adjust each question’s difficulty level and topic based on predefined criteria or learning objectives, creating a dynamic, adaptable item bank. With AI, you can create varied sets of questions, making life easier for teachers and saving time, effort, and money for institutions.
Auto-Grading for Essays and Open-ended Questions
One of the significant hurdles in traditional e-assessment has been grading essays and open-ended responses. Manual grading is time-consuming and subject to inconsistencies and human error. AI comes to the rescue here as well. Using sophisticated Natural Language Processing and machine learning algorithms, AI can assess and grade open-ended responses accurately and efficiently. AI can analyze various aspects of a response, such as relevance to the question, coherence of ideas, grammar, and vocabulary, providing a comprehensive evaluation. This streamlines the grading process and provides students with detailed feedback to enhance their learning.
Exam Construction
We’ve touched on the capabilities of AI to create individual questions and even grade them, but what if we take it a step further? What if AI could craft complete exams?
AI-assisted exam construction uses AI to generate, organize, and structure exam questions. It involves using machine learning algorithms to parse a large dataset of educational content and creating questions based on the analyzed data. This can encompass a wide range of question types, from multiple choice to short answer and essay questions. These algorithms are designed to identify the core concepts, themes, and structures within the subject matter and generate questions that assess a student’s understanding of these elements. Plus, it doesn’t just make one-size-fits-all questions, it can change up the difficulty level of the questions, making them harder or easier depending on what each student or class needs, allowing for personalized learning experiences.
Risks and Considerations
Despite its immense potential, integrating AI into e-assessment also brings several risks and ethical considerations. We must consider the fairness and bias in AI algorithms, privacy concerns surrounding the data collected for AI analysis, and the accessibility of AI-assisted tools for all students. Not to mention, the technology is only as good as the data it’s trained on. We can reinforce existing educational inequalities with careful management and diverse educational content.
The key to mitigating these concerns lies in rigorous algorithm testing, transparency in AI decisions, and strict data privacy measures. Future blogs will delve deeper into these mitigation strategies. Still, it’s worth mentioning that understanding and addressing these issues will be crucial to realizing the full potential of AI in e-assessment.
AI holds vast potential to reshape the e-assessment landscape. It offers a chance to create a more personalized, adaptive, and efficient testing process, enabling teachers to support their students’ learning journeys better. Yet, while we forge ahead in our AI exploration, we must remain aware of the potential risks and work proactively to mitigate them. With careful consideration and the right strategies in place, we stand to make incredible strides in education. The future of e-assessment with AI is not just exciting – it’s promising, bringing us closer to a world where each student gets the personalized educational experience they deserve.
Sources:
The intelligent essay assessor: Applications to educational technology
Foltz, Peter. Laham, Darrell. Landauer, T. 1999/04/02
What is Natural Language Processing? Intro to NLP in Machine Learning
https://www.gyansetu.in/blogs/what-is-natural-language-processing//
Plato
https://physics.illinois.edu/people/history/PLATO
Applying modern psychometric techniques to melodic discrimination testing: item response theory, computerized adaptive testing, and automatic item generation. Harrison, P.M., Collins, T., & Müllensiefen, D. (2017)
