Assessment of expert decisions in graduate quantum mechanics

Abstract

[This paper is part of the Focused Collection in Investigating and Improving Quantum Education through Research.] One of the greatest weaknesses of physics education research is the paucity of research on graduate education. While there are a growing number of investigations of graduate student degree progress and admissions, there are very few investigations of learning at the graduate level. Additionally, existing studies of learning in physics graduate programs frequently focus on content knowledge rather than professional skills such as problem solving. Given that over 90% of physics Ph.D. graduates report solving technical problems regularly in the workplace, we sought to develop an assessment to measure how well graduate programs are training students to solve problems. Using a framework that characterizes expert-like problem-solving skills as a set of decisions to be made, we developed and validated such an assessment in graduate quantum mechanics (QM) following recently developed design frameworks for measuring problem solving and best practices for assessment validation. We collected validity evidence through think-aloud interviews with practicing physicists and physics graduate students, as well as written solutions provided by physics graduate and undergraduate students. The assessment shows strong potential in differentiating novice and expert problem solving in QM and showed reliability in repeated testing with similar populations. These results show the promise of measuring expert decision making in graduate QM and provide baseline measurements for future educational interventions to more effectively teach these skills.