If the courses you teach include exams, you may be working on preparing the first one around this time of the semester. Many faculty use exams to evaluate how much students have learned at particular points in a course, but designing an exam that does so effectively can be easier said than done.
When you give an exam, you’re gathering data on your students’ progress, so you want your exam to be both valid and reliable. First, as Ory and Ryan put it, you have to make sure you’re “testing what you want to be testing”—that is, what your students should know and be able to do. If your test is not aligned with your learning goals for students, “you will not be able to evaluate your students’ progress with any accuracy,” explain Clegg and Cashin. This means you need for the tasks on the exam—the questions, problems, etc.—to elicit the kind of thinking that you’re trying to test.
For example, you’ll need to consider what proportion of each exam will test higher-order thinking as opposed to recall. Many test banks tend to focus on recognition questions rather than analysis. If you want your students to be able to apply concepts in a variety of ways, you’ll need questions that demand that sort of thinking. Of course, your students will also need to practice this thinking in class, repeatedly. We can’t expect them to pick up new reasoning skills during an exam.
It’s useful to think about how much time and priority you allot to particular concepts in the time leading up to the exam. The most important ideas deserve the bulk of our attention, student effort, and class time, so they merit a corresponding proportion of exams. As Mastascua et al. point out, research on the “testing effect” shows that if we test primarily on minutiae, our students may leave the course remembering those details, rather than the foundational concepts.
You may also be thinking about how to make it less tempting to consult AI for correct answers. Some faculty are switching to paper exams, and even blue books or oral exams. In-class, open-book, open-note exams that ask students to apply, analyze, and synthesize tend to be resistant to AI distortion, and also tend to be useful for assessing higher-order thinking. Varied question types, including some short answers, can be useful, as can multi-part questions and reflection questions. Very context-specific, personalized or grounded questions are more AI resistant. Two-stage exams, where students take an exam first individually and then as a group, can inhibit reliance on AI assistance at the same time as they help students talk through their ideas and practice collaboration.
Tests and quizzes don’t just measure performance; they can be powerful learning opportunities. We’re asking students to retrieve and apply the ideas they need to take away from our courses, so ideally we’re leading them through the thinking we want them to practice. Students can also learn from exams by completing exam wrappers. These activities help students reflect on their preparation and performance on an exam, identify gaps in their current knowledge and skills, and strategize how they can fill those gaps and improve their performance on the next exam.
If you’re working on an exam right now, practical guidelines and suggestions for effective tests are available here. You can also explore this more in-depth guide to writing good multiple-choice questions. CMU offers broadly applicable tips on designing effective exams. And as always, we’re available to help: feel free to bring us your draft exams if you want feedback! We can also help you think about alternative forms of assessment. You can reach out for a consultation by emailing us at pro-teaching@fsu.edu. We look forward to working with you!