Testing our 24-month GenAI Academic Integrity Predication: Did we get it right?

Two and a half years ago, the release of ChatGPT-3.5 sent shockwaves through educational institutions. The realisation that a freely available AI could pass university-level assessments triggered widespread concern. In response to this upheaval, we launched one of the largest multi-institutional collaborative studies to assess the real-world impact of generative AI across all major assessment types. This study broke many records.

Our initial findings confirmed that ChatGPT-3.5 could indeed pass many assessments, particularly when students had a sense of what the expected output should look like. But the real threat wasn’t what it could do then; it was what was coming. At the time, we wrote:

“We need to be prepared for a very different environment in the next twelve to twenty-four months.”

So, how accurate was that prediction at the 12-month mark?

Within 12 months, ChatGPT-3.5 had already begun to feel like a relic. Newer, more powerful models, and a growing army of alternatives were rapidly changing the landscape. Our follow-up study put these new tools to the test, and the results were clear: not only had the capabilities improved, but it had also become easier for students with minimal understanding to pass assessments. While performance varied by subject, the overall academic integrity risk threshold had undeniably been breached.

We responded by equipping the education community with the Generative AI Assessment Security & Opportunity Matrix, alongside practical procedures for conducting risk assessment audits. We categorised assessment strategies into short-term and long-term options, urging institutions to act sooner rather than later. Many heeded the call. Our two studies combined have reached 55,000 downloads.

Now, at the 24-month mark, it’s time to ask again: How accurate?

Our third and final updated 2025 study…… This time, we didn’t publish a third study because the 2024 data already told us what we needed to know. The message was simple and sobering: all assessments are now at risk. If academic integrity matters to your institution, business as usual is no longer an option.

If we repeated the study today, one finding would stand out above all: most of the short-term security measures we previously recommended are already obsolete. The newest generation of GenAI tools is far more capable than anything we tested just a year ago. Tasks with known, specific, or structured answers can now be completed by students with almost no real understanding. The ability to work across text, documents, code, images, voice, and video, while still imperfect, is advancing faster than our ability to respond. If P's means degrees, the latest generation can help most students get there. Moving forward requires shifting to the recommended long-term options.

In our own day-to-day practice, we’ve come to rely on these tools ourselves. They let us do more, faster. Unsurprisingly, students have integrated them too, nearly all of them. We’ve run numerous controlled experiments. One includes educating students on professional conduct, AI ethics, reinforcing the importance of trust, and providing assessments explicitly framed as off-limits for AI assistance. Despite those efforts, most still used it. More interesting was that, despite being told not to use it, a good proportion used it and declared its use. It's hardly surprising as they're already using these tools in so many parts of their lives, so steering clear of them during assessments is a big ask.

The most honest position now?

"The only trust to have now, is to trust students will use it if given the opportunity"

Twenty-four months later, our prediction holds. If your department or institution has yet to implement meaningful changes, it is almost certain you are facing an academic integrity problem, whether you realise it or not. Denial won’t delay the consequences, it will just make the change harder.

What are we doing about it?

Well, that is why we formed the AAIEEC. We have created one of the largest collaborations to look at many aspects of how we learn to live and adapt to a world of GenAI. We are building a wealth of resources and research insights to help the community move forward.

By Sasha Nikolic, Peter Neal, Rezwanul Haque, Scott Daniel, Sarah Grundy, Marina Belkina, Sarah Lyden, and Ghulam M Hassan

27 May 2025

Testing our prediction