GenAI Tools: Capable, But Not a Substitute for Human Expertise

Analysis: GenAI tools are already effectively handling many topics traditionally taught in university settings.

In an unprecedented test, I fed university exam questions to two prominent GenAI models, ChatGPT-4o mini and DeepSeek-R1, without any additional information or prompts. The results were startling. ChatGPT-4o mini achieved a 90% score, while DeepSeek-R1 scored 94%, matching or surpassing the average performance of human students in a UCC engineering exam.

Elaine Burke, tech journalist and host of the For Tech’s Sake podcast, reviews ChatGPT-4o on RTÉ Radio 1’s Brendan O’Connor Show.

For context, the average score of 61 UCC engineering students who took the exam in May 2024 was 63.8%. These students met stringent entry requirements, including advanced mathematics proficiency and a 520-point Leaving Cert score. This comparison underscores the remarkable capability of current GenAI models.

While these GenAI tools excelled in undergraduate-level questions, their performance on more advanced engineering topics was mixed. Errors and logical inconsistencies emerged, underscoring the limitations of these models in tackling highly specialized problems. GenAIs are powerful tools but should not replace human expertise, especially in critical fields.

Explore the transformative impact of DeepSeek AI on RTÉ Radio 1’s Drivetime.

A recent study from the École Polytechnique Fédérale de Lausanne (EPFL) demonstrated that ChatGPT could answer at least 65.8% of examination questions correctly across 50 diverse technical and natural science courses. Similar findings have emerged in other professional fields, with GenAIs passing key examinations in law and medicine.

Implications for University Assessment Methods

The implications of GenAI’s capabilities on university assessment are profound. Traditional end-of-semester exams, while stress-inducing, ensure academic integrity. These exams are invigilated, reducing the risk of cheating. However, as universities adopt more continuous assessment and online learning, the vulnerability to GenAI increases.

Online assessments, once heralded for flexibility and convenience, now face significant challenges. GenAIs can convincingly perform tasks like writing essays, coding, and solving mathematical problems. The integrity of online university programs is under serious scrutiny.

To safeguard academic integrity in the GenAI era, supervised, in-person assessments, akin to traditional exams, are becoming necessary again. Degrees awarded should reflect genuine knowledge and understanding, not just proficiency in using AI tools.

Attempting to police or ban the use of GenAI is futile.

Efforts to restrict GenAI usage are futile. The technology’s advancements and ubiquity make such measures impractical. Instead, institutions should encourage responsible use. Transparency about AI tool usage should be promoted, and the fundamentals of the subject matter should be emphasized. Teaching critical thinking is crucial for students to discern the reliability of AI-generated content.

Read: ChatGPT or CheatGPT? Irish Students Weigh In on the Latest AI

Online education offers numerous benefits, including flexibility and the ability to reach a wider audience. However, ensuring the integrity of these programs in the GenAI era will require innovative yet thorough assessment methods.

Follow RTÉ Brainstorm on WhatsApp and Instagram for more stories and updates

The views expressed here are those of the author and do not represent or reflect the views of RTÉ

We invite you to share your thoughts on this evolving landscape of education and technology. Leave a comment below, subscribe to our newsletter for regular updates, or share this article on your social media platforms.