“I’m really cynical when I answer these questions and I think they’re trying to trick you,” Meredith Mills, a junior in journalism, said about multiple choice questions. “They’re testing how well you read, versus how much you know or understand.”
In a majority of Ohio State classes, multiple choice is utilized by instructors in favor of open-response questions because they are the most efficient way of testing students. But although a well-written multiple choice exam can test depth of understanding, students are often given poorly written questions, which only test superficial knowledge, said Bruce W. Tuckman, a professor in the School of Education Policy and Leadership and director of the Walter E. Dennis Learning Center.
“The easiest questions to write are ones that try to fit a definition to a term – pure memory questions. That’s why a lot of tests are bad,” Tuckman said, adding that many instructors have not been taught to write questions properly.
For students, the implications of poorly written tests are large.
“An awful lot depends on it,” Tuckman said. “It’s a student’s grade … an awful lot is dependent on people being able to write tests.”
Jerome D’Agostino, a professor in the School of Education Policy and Leadership, agrees that multiple choice questions can be ineffective if not written properly.
“I don’t know about OSU tests in particular, but research shows that yes, multiple choice tends to encourage instructors to test for superficial knowledge,” he said. “It’s not like all multiple choice questions are doomed for testing only surface knowledge, but it really takes a lot of work and a lot of skill to test for the deeper cognitive skills, so that’s why you don’t often see that.”
Tuckman said that an instructor must write questions that test more than memorized facts to assess accurately a student’s understanding of the course material.
“You have to write comprehension questions and not knowledge questions,” he said. “With knowledge questions, if a student memorizes the right words, they can pick up on the right answer.”
Tuckman uses a question he wrote for a recent exam as an example.
“I had a question … where students are given four characteristics of problem solving and they’re asked to identify which of the four doesn’t fit but not told what the criterion is …. Three of the answer choices represent a way that experts would think to solve problems and one represented a way that experts wouldn’t think, novices would. This type of question requires understanding, you can’t just memorize material,” he said, adding “These questions are harder to write.”
Another issue for students taking multiple-choice exams is that instructors will often use generic questions provided by the textbook.
“Probably about three quarters of the tests college students have to take come out of test files that someone puts together,” Tuckman said. “Especially for undergraduates, most of the textbooks come with questions.”
These textbook questions are often poorly written.
“It’s been my experience that most of the questions that come with the instructor materials, those items tend to be inferior in quality,” D’Agostino said. “They’re often created as an afterthought.”
The generic textbook questions can also be confusing.
“When you instruct, you use your own language, so text questions are not in the instructor’s words and students get confused,” D’Agostino said. “If the instructor’s not just teaching out of the textbook they should definitely not just pull items from the instructor’s manual.”
However, the solution to poorly written exams is not simply to mandate open-response questions.
“The advantage of open-ended questions is they can ask for a deeper level of processing and ask students to justify their response and reasoning,” D’Agostino said. He added that grading open-ended exams can be complicated.
“The trouble with these essay tests is that you’re really at the mercy of the person who’s grading it,” Tuckman said.
If there is more than one person grading exams there can be discrepancies in the scoring, D’Agostino added.
Multiple-choice exams have many advantages, especially if they are well-written.
“Although they encourage instructors to test for surface knowledge, you can test a larger domain by using more questions,” D’Agostino said. “They tend to be more objective and can be scored more efficiently … I wouldn’t be blaming instructors so much.”
Briony Clare can be reached at [email protected].
