Designing an Exam that Makes the Grade
Somehow we’ve passed the midpoint of the semester, so it’s already time to start thinking about… finals? Evaluating our students’ learning is one of the most important tasks we undertake in our teaching, and it’s also one of the most difficult. Exams are common tools for measuring student achievement, but crafting a good one takes considerable skill and attention.
When you give an exam, you’re gathering data on your students’ progress, so the test should be both valid and reliable. When we’re “testing what we want to be testing,” as Ory and Ryan (1993) put it, we start with a clear idea of what successful learning would look like: what should students know or be able to do when they finish the course? Then we figure out how students can best give us evidence of that learning. If your test is not well aligned with your goals, “you will not be able to evaluate your students’ progress with any accuracy,” warn Clegg and Cashin.
A good exam should also be fair, so to determine what you need to test, you’ll have to consider how much time and priority you allotted to particular concepts this semester. The most important ideas deserve the bulk of attention, student effort and class time, so they merit a corresponding proportion of exams. As the research on the “testing effect” reminds us, if we test primarily on minutiae, our students may leave the course remembering those details, rather than the foundational concepts (Mastascusa, 2011). Another important consideration is the proportion of each exam that tests higher-order thinking, as opposed to recall.
To make sure your exam is balanced, and reflects your priorities for learning, you might want to draw up a Table of Specifications, where you identify the major concepts and their weight, assigning each concept the number and difficulty of questions that will adequately measure students’ level of mastery.
If you want your students to be able to apply concepts in a variety of ways, you’ll needquestions that demand that sort of thinking. Many test banks tend to rely on recognition questions, rather than analysis, so you will probably need to develop your own. (We’re happy to help you write great questions!) Of course, your students will also need to practice this thinking in class, repeatedly: we can’t expect them to pick up new reasoning skills the day of the exam.
Tests and quizzes don’t just measure learning; they should also be powerful learning opportunities. We’re asking students to retrieve and apply the ideas they need to take away from our courses, so ideally our exams are leading students through the thinking we want them to practice. They can keep learning after midterms by reviewing their tests afterward, and reflecting on how they prepared for them. (For ideas, see our previous tip on exam wrappers.)
Practical guidelines and suggestions, plus a checklist for creating effective tests, are available here. CMU also offers great tips on designing effective exams. And as always, we’re available to help: feel free to bring us your draft exams, or sign up for our Exam Design Workshop.
UPCOMING…
Exam Design Workshop
Wednesday, November 13 | 10:00am-12:00 | DIF 432 | Sign up to attend
Test what you want to be testing. During this session, we’ll work on designing exams that are not only accurate measures of student learning in your course, but also learning opportunities themselves.