How do the collection mode and questionnaire used affect the European indicators in the French Labour Force Survey ?

As part of the future Framework Regulation for the Production of European Statistics on Persons and Households (Integrated European Social Statistics - IESS), aimed at harmonising and standardising social statistics surveys, the questionnaire for the French Labour Force Survey (EEC) is slated to be re-designed. The expected changes to the questionnaire module dedicated to position on the labour market (Labour Status module) will probably impact measurements on the main labour market indicators, first and foremost, employment and unemployment rates. Participating in the task force in charge of developing the European questionnaire model, INSEE responded favourably to the call for proposals issued by Eurostat in 2015 to conduct a pilot test of "At work" questions on this future new module. The Institute thus implemented an experimental survey, the main objective of which was to test for a "questionnaire effect" on the employment rate and the unemployment rate in particular, but also to analyse the connection between changes to questions and the resulting estimates.

With reflections ongoing as to how to implement the multi-mode approach in its surveys (especially the Internet), INSEE chose to conduct this test online without any interviewer intervention. 40,000 households were surveyed exclusively online : half received the current questionnaire from the Labour Force Survey ; the other half received a version proposed by Eurostat. Comparing responses to the two versions, researchers were able to identify a "questionnaire effect" as regards the main labour market indicators. To illustrate, online, the unemployment rate calculated based on the current questionnaire was 12%, as compared to 11% based on the version proposed by Eurostat.

Once the "questionnaire effect" was analysed, the data collected online were then compared and contrasted with data collected at the same time in face-to-face situations, through the French Labour Force Survey. This "mode effect" was analysed and broken down into a "selection effect" and a "measurement effect", as recommended in the literature. Analysing respondent profiles revealed a "selection effect" in the web survey. Specifically, households with high incomes have a greater propensity to respond via the Internet. The same applies to households that own their place of residence, and households residing in urban municipalities. People aged 65 or over, on the other hand, are less represented among respondents to the online survey than are young people. Once adjustments had been made for this "selection effect", the gap in unemployment rate was 2 points : it rose to 12% when calculated based on data collected online, versus 10% in face-to-face situations. Comparison of data adjusted for the "selection effect" also revealed an over-representation of the unemployed as defined by the ILO among the online respondents.

In conclusion, this test made it possible to demonstrate that a change in questionnaire or the integration of the Internet as a data collection mode could have an impact on the main labour market indicators. However, it also revealed potential positive impacts of a change in collection mode, which could be a facilitator for certain respondent profiles.