AI-proof project-based assessments by making them context-specific

GenAI’s ability to produce content in the blink of an eye is a particularly sharp double-edged sword for language teaching. At our institution, many students now rely on AI to complete their writing assessments – vital components of testing linguistic proficiency – meaning that the 1,000-word discursive essays previously used to assess proficiency have now become unviable.

Literature about adopting GenAI in teaching suggests that modifying assessment tasks is one way to confront student over-reliance on GenAI. An emphasis on creative and context-specific assessments to engage students on a personal level prompted us to design a project-based learning (PBL) assessment that culminates in a combined written group proposal and individual speaking test. 

Why project-based learning?

Projects tend to favour a learner-driven instructional model, in which students collaborate with peers, identify problems, engage in investigations, set implementation plans and reflect on their experiences within real-world contexts. A group project can give students an opportunity to engage with university life by tasking them with proposing improvements to the learning experience, student support systems and campus environment.

For example, students might propose the introduction of beehives and flower gardens around campus to improve local biodiversity and create a more attractive learning environment. At the beginning of the process, they explore and photograph their campus in its current state, considering areas to place the beehives. They investigate student and staff needs via surveys and contextually relevant sources, learning that many of their peers have phobias, are allergic or simply sceptical. 

Throughout the semester, teachers can guide students in brainstorming and critically assessing their ideas before rationalising the proposed implementation through a collaborative piece of writing, for example, a 2,000-word proposal. A curriculum that adopts this approach sees teachers as facilitators and students as responsible for resolving institution-specific issues that AI tools have limited or no knowledge of.

• Spotlight guide: Bringing GenAI into the university classroom
• Keep calm and carry on: ChatGPT doesn’t change a thing for academic integrity
• So, you want to use ChatGPT in the classroom this semester?

Adapt the assessment criteria

GenAI’s ability to polish pieces of text reduces the importance of linguistic accuracy as an evaluation criterion in this context. A project-based assessment can place less emphasis on the written element and assign greater weight to an individual spoken defence of the proposal.

First, assess students on their ability to collaboratively produce a logically convincing and linguistically cohesive written proposal. Assess students’ persuasive communication strategies: for example, the extent to which their ideas are convincingly developed using collected survey data or secondary source evidence.  

Then, conduct a speaking test in which students must individually defend their written proposal against examiner questions. By asking general questions about the project-based learning experience (such as: “What challenges did you face during the project?”) and specific questions related to their written work (“Why did you choose to place the beehives there?”, for example), students must reflect independently on the project and how to communicate it effectively. 

The writing and speaking assessment descriptors can then use Common European Framework of Reference for Languages-aligned criteria to evaluate how well students use relevant language skills. This approach ensures that we assess students not only on their written output but, more importantly, on their ability to articulate ideas verbally in real-time.  

Reward contextual awareness

This task requires students to have contextual knowledge about their institution and academic style, therefore, GenAI cannot effectively complete this task. Both the student and the examiner know that beehives cannot be located on the third floor of our central building, but the GenAI they feel so compelled to use does not.

Ultimately, a curriculum and assessment redesigned to discourage over-reliance on GenAI shifts students’ attention from using GenAI to complete written tasks to instead actively engaging with a tangible and highly context-specific issue. We do not punish the use of AI, merely curate the task to limit its use.  

Unlike the timed exam, we see this approach as a positive step towards effective engagement with GenAI in higher education. By staying up to date, reactionary and flexible, we can adapt our project-based learning assessments to offer effective ways to assess students’ language proficiency. 

Oliver Jarvest and Ying Zhou are English for academic purposes lecturers and Simon Sheridan is director of the English Language Centre at Xi’an Jiaotong Liverpool University in China.

If you’d like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.