Valid Tests
Evaluating our students’ learning is one of the most important tasks we undertake in our teaching, and it’s also one of the most difficult. Exams are a common tool for measuring student achievement, but they’re not easy to write.
When you give an exam, you’re gathering data on your students’ progress, so the test should be both valid and reliable. When we’re “testing what we want to be testing,” as Ory and Ryan put it, we start with a clear idea of what successful learning would look like—what should students know or be able to do by this point in the class?—and figure out how students can best give us evidence of that learning. If your test is not well aligned with your goals, “you will not be able to evaluate your students’ progress with any accuracy,” warn Clegg and Cashin.
It’s also important to think about how much time and priority you allotted to particular concepts so far this semester. The most important ideas deserve the bulk of our attention, student effort, and class time, so they merit a corresponding proportion of exams. As the research on the “testing effect” reminds us, if we test primarily on minutiae, our students may leave the course remembering those details, rather than the foundational concepts (Mastascusa, 2011).
Another important consideration is the proportion of each exam that tests higher-order thinking, as opposed to recall. Many test banks tend to rely on recognition questions, rather than analysis. If you want your students to be able to apply concepts in a variety of ways, you’ll need questions that demand that sort of thinking. Of course, your students will also need to practice this thinking in class, repeatedly: we can’t expect them to pick up new reasoning skills the day of the exam.
Tests and quizzes don’t just measure learning; they should also be powerful learning opportunities. We’re asking students to retrieve and apply the ideas they need to take away from our courses, so ideally our exams are leading students through the thinking we want them to practice. They can keep learning by reviewing the test afterward, and reflecting on how they prepared for it. (For ideas, see our previous tip on exam wrappers.)
Practical guidelines and suggestions, plus a checklist for creating effective tests, are available here. CMU also offers great tips on designing effective exams. And as always, we’re available to help: feel free to bring us your draft exams, or sign up for our Exam Design Workshop!
Upcoming Events
Exam Design Workshop
Tues., Oct. 23rd | 2:00-4:00 p.m. | DIF 432 | Sign up to attend
Test what you want to be testing. During this session, we’ll work on designing exams that are not only accurate measures of student learning in your course, but also learning opportunities themselves.
Open Textbook Workshop
Thurs., Oct. 25rd | 10:00 a.m.-12:00 p.m. | Bradley Reading Room, Strozier Library
The Office of the Provost is sponsoring a workshop to introduce faculty to open textbooks and the benefits they can bring to student learning, faculty pedagogical practice, and social justice on campus. Participating faculty will be invited to engage with an open textbook in their discipline by writing a brief review, for which they will be eligible to receive a $200 stipend. Interested faculty are invited to apply by Friday, October 12th.
If you have questions about this workshop or open textbooks, please contact Devin Soper, Scholarly Communications Librarian, at 850-645-2600 or dsoper@fsu.edu. You can also visit the Open & Affordable Textbook Initiative website for more information about open education initiatives at FSU.