Économie et Statistique n° 424-425: Measuring Adult Skills with the "Information and daily life" survey (Information et vie quotidienne: IVQ)
Development of an Overall Score for an Adaptive Test
The construction of an overall skill indicator from answers to test exercises requires special statistical methods that are more complex than the mere tabulation of correct answers. Such methods are particularly useful when the persons assessed have not all taken the same tests. That is the case with the 2004 Information and Daily Life Survey (Information et Vie Quotidienne: IVQ), in which the exercises were adapted to the respondent’s skill level in order to increase motivation, particularly for respondents who had left school a long time earlier. The results of the first test were used to set the difficulty of subsequent questions. This “orientation” procedure substantially improves collection conditions and data quality, but it generates special difficulties for estimating an overall skill indicator valid for the entire population. How can we compare results for groups that have not taken the same tests? Our article suggests various methods to allow for the orientation procedure. We tested them on fictitious data to assess their robustness and limits. This information loss caused by orientation is rather modest but not null—particularly for a significant number of persons situated around the threshold levels where they are channelled toward the hard tests or the easy tests. These results suggest some adjustments that should be made in the test structure for the next edition of the Survey.