Persona-MCMC: Iterative LLM Persona Refinement for Interpretable Student Modeling
Classical student models such as Item Response Theory summarize a student with a number. That number cannot say what the student knows, and it cannot be used for new questions. Persona-MCMC represents each student instead with a persona: a short, human-readable description of their strengths and weaknesses. A large language model then uses the persona to predict how the student will answer questions it has never seen.
To find a good persona, one LLM agent proposes small edits to the persona and another predicts the student's answers with it. Each edit is accepted or rejected using a Metropolis-Hastings-style rule based on predictive accuracy. Finally, an aggregation agent merges the top personas into one. On real student responses to math questions (NeurIPS 2020 Education Challenge), Persona-MCMC outperformed IRT and in-context learning baselines and produced personas that are more specific and topic-aware.
Method: Multi-agent LLM prompting (GPT-4.1-mini); Metropolis-Hastings-inspired acceptance with annealed temperature; top-K persona aggregation; comparison against 1PL IRT and in-context learning (ICL).