Karim Abbas
← All work 2024Final-year project

University course assistant

Retrieval-augmented generation is now a core part of my day job, and this is where I first built it. A general chatbot can explain a topic, but it does not know what your lecturer covered. This project gave GPT-3.5 access to a module’s lecture material and fine-tuned it on course-style questions, then checked whether students got better answers.

  • Python
  • GPT-3.5
  • Retrieval-augmented generation
  • Fine-tuning
  • NLP
Code Ask for the report

The question

Students have module-specific questions and lecturers cannot answer them one at a time. Off-the-shelf models answer fluently but without the lecture context, so the answers can be generic or subtly wrong for the course.

What I built

  • Retrieval. Lecture material is indexed so the most relevant passages are pulled into each answer.
  • Fine-tuning. GPT-3.5 was fine-tuned on academic questions in the style of the course.
  • Evaluation. I compared it with the standard model using automatic scores such as ROUGE, and collected feedback from university students.

What I found

The enhanced assistant gave more specific answers than the standard model and stayed closer to the source lecture. Students who tried it said it would help their learning. I presented it at the University of Leeds Digital AI Summit, and the project was awarded a First (73%).

Caveats

  • It was tested on one module with a small group of students, so the results may not carry over to other courses.
  • The full metric tables are in the report, which I am happy to share.