Heather Griffen investigated the context sensitivity of the Force Concept Inventory. The Force Concept Inventory and a context-modified test were given to 654 students enrolled in introductory physics classes at the University of Arkansas. Context changes had a statistically significant effect on student responses in 6 of 10 questions. The numerical impact on overall student performance is not sufficient to affect normal use of the Force Concept Inventory as a diagnostic instrument. Examination of responses by the same student allows calculation of the fraction of the students that consistently apply correct conceptual knowledge, the fraction of students that consistently apply misconceptions, and fraction of students whose knowledge is uncertain. Click here for her thesis. . Part of this research was published in Phys. Rev. ST Phys. Educ. Res. 3, 010102 (2007)
Ed Corcoran investigated the statistical distribution of pretest scores. A student’s state of knowledge entering a science class is variable. Some students have no prior knowledge of the material, where other students have substantial previous knowledge. The distribution of these states of knowledge is extracted by fitting test data with combinations of the binomial distribution. Since the pretest scores of both the Force Concepts Inventory and the Conceptual Survey of Electricity and Magnetism contain a substantial number of questions that appear to be answered simply by random guessing the multiple choice pretest overestimates the real state of student knowledge leading to a bias in the conceptual gain and the normalized conceptual gain towards low values. This bias can be removed from the conceptual gain by correcting the pretest score for the effect of guessing and constructing an effective present score. This effective pretest is an estimate of what the student would score on a non-multiple choice test.
Click here for the thesis. This work was refined and a poster was presented by John Stewart at the Summer 2007 AAPT meeting. Click here for the poster.