If you missed the chance to attend the “writing effective multiple choice questions” workshops, here are the top tips that were shared.
In this installment, we will take a look at some techniques to determine the quality of your questions.
For more information, check out the resources from the RED Group here:
http://tar576.wix.com/assessment#!resources/c1nv8
Sorting Your Scores - Determining High and Low Scorers
- The first step is to list all of your exam scores from highest to lowest.
- In a small group, divide the class in half to form a high and low group.
- In a large group, take the top 27% and bottom 27% to form these groups.
Determining Question Difficulty
High Scores Correct + Low Scores Correct
Total Responses
- For each question, count how many students answered it correctly from the high group, and then from the low group.
- Add the correct answers from the high and low groups together.
- Divide that number by the total number of students who answered the question.
- What does the result mean?
- .25 or less: This was a hard question
- .75 or more: This was an easy question
Determining the Discrimination Index
High Scores Correct - Low Scores Correct
Total Responses x 0.5
The discrimination index is a way to tell if your question is good or wonky.
Ideally, all of the high group gets questions right, and the low group gets them wrong.
If only your low group gets a question right, it indicates the question is not good.
Ideally, all of the high group gets questions right, and the low group gets them wrong.
If only your low group gets a question right, it indicates the question is not good.
- Take the number of correct high scorers and subtract the number of correct low scorers.
- Divide by half of the total number of answers.
- What does the result mean?
- A score of 0 to 1 is good - more high scorers than low got the question right
- A score of -1 to 0 is bad - more low scores than high got the question right