Abstract
AI mentor METIS outperforms GPT-5 and Claude Sonnet 4.5 in supporting undergraduate research writing across multiple stages, with higher student scores and improved document-grounded outputs, though challenges remain in tool routing and stage classification.
Many students lack access to expert research mentorship. We ask whether an AI mentor can move undergraduates from an idea to a paper. We build METIS, a tool-augmented, stage-aware assistant with literature search, curated guidelines, methodology checks, and memory. We evaluate METIS against GPT-5 and Claude Sonnet 4.5 across six writing stages using LLM-as-a-judge pairwise preferences, student-persona rubrics, short multi-turn tutoring, and evidence/compliance checks. On 90 single-turn prompts, LLM judges preferred METIS to Claude Sonnet 4.5 in 71% and to GPT-5 in 54%. Student scores (clarity/actionability/constraint-fit; 90 prompts x 3 judges) are higher across stages. In multi-turn sessions (five scenarios/agent), METIS yields slightly higher final quality than GPT-5. Gains concentrate in document-grounded stages (D-F), consistent with stage-aware routing and groundings failure modes include premature tool routing, shallow grounding, and occasional stage misclassification.
Community
Students have immense research potential, but enough mentors for them. What if we could design an AI system to mentor them?
We introduce METIS (Mentoring Engine for Thoughtful Inquiry & Solutions), a stage-aware research mentor.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- From Pilots to Practices: A Scoping Review of GenAI-Enabled Personalization in Computer Science Education (2025)
- An Agentic AI Framework for Training General Practitioner Student Skills (2025)
- Socratic Students: Teaching Language Models to Learn by Asking Questions (2025)
- Can Consumer Chatbots Reason? A Student-Led Field Experiment Embedded in an"AI-for-All"Undergraduate Course (2025)
- An Experience Report on a Pedagogically Controlled, Curriculum-Constrained AI Tutor for SE Education (2025)
- SocraticAI: Transforming LLMs into Guided CS Tutors Through Scaffolded Interaction (2025)
- Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactions (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper