In 2000, SESSI, which was the Ministry of Industry’s Statistical Department at that time, started collecting data from businesses via the internet with its monthly branch survey, before extending it to other surveys in the industrial sector. With the merger of part of SESSI and INSEE in 2008, the data collection infrastructure became the property of INSEE. INSEE already had its own system, since 2004, for a dozen frequent and regular business surveys. Gradually, the rapid spread of the internet within businesses and the desire to reduce collection costs have encouraged other producer services, from INSEE or ministerial statistical departments, to digitise their questionnaires. By seeking to respond as quickly as possible to the aspirations of producers and the pressing demand of businesses, we ended up with a juxtaposition of heterogeneous tools. This situation led INSEE to seek to reduce the number of web data collection infrastructures (see the article by Jean-Marc Béguin on the emergence of innovation in statistical production) in order to facilitate responses from businesses, while also reducing the IT maintenance costs of the various platforms.

At the same time, during the French national conferences on administrative simplification in April 2011, businesses raised their difficulties with regard to the statistical burden. Among the 80 measures resulting from these conferences was the objective of “Digitising 100% of Official Statistical Surveys” and centralising questionnaires on a single site from 2013. Everyone agreed – and the business interviews confirmed this – that this method of responding was less expensive for businesses than using paper questionnaires. At that time, the availability of data collection by internet was still insufficient: the surveys for which it was technically possible represented 73% of the volume of questionnaires sent by INSEE to businesses, but only 51% of the surveys of the entire Official Statistical System. Therefore, the aim was less to reduce the statistical burden in the traditional sense of the term (“reduce the volume of surveys”) than to reduce the response burden by simplifying collection systems for businesses.

This was the context in which the Coltrane (COLlecte TRANsversale d’Enquêtes – Transversal Collection of Surveys) project was started in 2010. Its main objectives were to:

The project already had a broad scope in terms of its schedule. Its original objectives did not include being able to meet the needs of household surveys (or the population census). However, the idea remained that all potentially common functions between the two data collection areas developed during the project should eventually be shared.

One particular feature sets Coltrane apart from existing platforms (Saraiva et alii, 2016; Ouvrir dans un nouvel ongletFazio et alii, 2015; Ouvrir dans un nouvel ongletThieme et alii, 2015): this feature is the close links that it has had, from the outset, with the RMéS statistical metadata repository project (Bonnans, 2019) and, in particular, the stated objective of using the metadata that describe the surveys questionnaires to directly generate the collection tools, hence the term “active” metadata to describe the system.

INSEE has adopted the GSBPM to model its statistical processes and the DDI standard (Ouvrir dans un nouvel ongletData Documentation Initiative) to formally describe the life cycle of the data, in particular the “questionnaire” objects. This structural choice made it impossible to use one of the collection software packages on the market, as they were not yet adapted to these standards.

The automatic generation of collection media, i.e. of the screens and programs that allow responses to be retrieved, based on their DDI description, forms the basis for the expected productivity gains. It was initially developed as part of a Coltrane sub-project called Eno (see the article on the Eno generator by Heïdi Koumarianos and Éric Sigaud). The Eno generator makes it possible, based on the specification of the questionnaire, to automate tasks previously carried out manually: development of the collection medium by a computer engineer and then testing and acceptance by the statistician; it makes it possible to ensure that several different collection methods (including paper) correspond to the same general description. Therefore, unlike in the past, the designer no longer makes the same number of specifications as there are types of collection media. A single specification is sufficient and it guarantees the homogeneity of the questioning whatever the method.

Initiated with Coltrane and its sub-project Eno, the activation of metadata for collection has been enhanced with the Pogues tool (see the article on the Pogues tool by Franck Cotton and Thomas Dubois), which enables the statistician to design the questionnaire template and to generate the template in DDI automatically, without using an expert in that standard.

At present, the Pogues/Eno/Coltrane system seeks to place the user-designer at the centre of the process. For Coltrane, it was decided from the outset to opt for modular development, with the successive production of “operations”. An operation was a complete and coherent functional unit, for which an application was delivered in production and was the subject of complete acceptance by a group of users. The introduction of an operation into production required the collection of a pilot internet survey with a set of functionalities (sometimes version specific) which could then be used for subsequent pilot surveys.

Three operations were launched during the first phase of the project, during which it was planned to develop all of the functions originally planned.

The second phase, also known as the generalisation or “mass integration” phase, began in September 2016. Once most of the components of the Coltrane service offer had been developed, it consisted of gradually transferring the thirty or so INSEE surveys into the system, especially those that used the old SESSI infrastructure (such as the Monthly Industry Surveys [Enquêtes Mensuelles de Branche, or EMB]).

The start of this second phase was more difficult than initially planned. In fact, the integration of the first surveys revealed weaknesses in the collection platform. In order to resolve them, it was necessary to optimise certain developments and strengthen the collection infrastructure in production so as to be sure that it would be able to support the expected burden once up and running. The mass integration of the surveys into the system was therefore carried out within the planned schedule, but at that time the platform did not offer all the functions envisaged during the project.

The platform provides services to survey designers during the Build phase of the collection media. Coltrane sends the responses collected via the platform, in a standard format, to the survey management applications (Figure 1). In order to avoid “discouraging” respondents, few consistency or partial non-response checks are implemented in Coltrane. This lack of obstructive checks also makes it possible to assume that Coltrane does not currently induce any trend due to internet collection compared to “paper” collection.

The surveys edit tools, which are outside Coltrane’s functional scope, make it possible to monitor the collection, in particular to identify non-responding units to be reactivated, to control the quality of responses and to adjust them if necessary. Therefore, through management applications, Coltrane also provides services to survey management teams during the Collection phases. We will see further how Coltrane directly equips the survey clerks in contact with the businesses.

However, the first users of Coltrane are, in fact, the businesses responding to the surveys hosted on the platform.

Figure 1. Coltrane, on the Borderline Between Statisticians and Businesses

When it is selected for the sample, the surveyed business receives a letter (or even an email) notifying it that it has been selected to answer a questionnaire. This letter, in addition to providing a brief presentation of the survey, contains the elements required to connect to the platform: link to open, username and password assigned to the natural person who will answer the survey and who is called the “Contact”, so that:

On first accessing the authentication portal (Figure 2), the Contact must personalise their password. In case of loss of the password, the Contact can request a new one online. The authentication portal uses a shared identification service, developed outside the project.

Once authenticated, the Contact is directed to a response portal (Figure 2) and, more specifically, to a tab named “Mes enquêtes” (“My Surveys”), a single access point for responding to all surveys managed by Coltrane (surveys from INSEE or any ministerial statistical office that has adopted Coltrane):

Figure 2. What the Surveyed Business Views with the Coltrane Platform

When a Contact navigates through a questionnaire, their answers are automatically recorded each time the page is changed. In this way, the Contact can leave the questionnaire at any time and return to it whenever they wish: the Contact is then given the option of returning directly to the last page viewed or starting over from the beginning. On reaching the end of the survey, the Contact can choose to send their response or simply save it to return to it later. At the time of sending, the Contact is given the option of downloading a summary of their response in “PDF” format.

From the response portal, the Contact can of course enter or update their personal information (first name, last name, email address, telephone number, position, postal address, etc.) using the “Mon compte” (“My Account”) tab. The Contact may also personalise their password and, in particular, if the Contact has received multiple usernames (one per survey, for example), they can “group them together” and have a single account to access and respond to questionnaires for several surveyed units or several surveys. On a practical level, this is a great advantage of the portal.

Throughout their navigation on the collection platform, the Contact within the business may access frequently asked questions and online assistance via a form. This assistance is contextualised, i.e. depending on where the form comes from, it is pre-completed to a greater or lesser extent and can be sent to two types of stakeholders:

This set of services offered through Coltrane ultimately makes it possible to improve the image of official statistics, particularly for large companies that receive a significant number of statistical survey questionnaires. First, their learning curve for completing questionnaires and navigating the collection platform are reduced. Second, they may be encouraged to respond to surveys that are in progress and that they had forgotten about.

While there has been no significant increase in response rates since the introduction of Coltrane, the system at least makes it possible to maintain the existing rates at fairly high levels.

In addition to the portals intended for the surveyed businesses, the Coltrane project has established several tools for survey designers to facilitate the creation of collection media, on the one hand, and to manage the collection itself, on the other.

The Eno collection media generator was developed as part of the Coltrane project. The project team also made a significant contribution to the establishment of Pogues (see the article by Franck Cotton and Thomas Dubois), a tool for designing the DDI questionnaire template. Using these tools, the statistician designs the questionnaire template and views it in a “blank” and non-personalised version, and approves the final version.

They must then use the management application for their survey to provide the Coltrane platform with the survey sample and its characteristics (in XML format). For each surveyed unit, this so-called “personalisation” file contains:

Coltrane generates and hosts all the web questionnaires for the units to be surveyed, based on the blank, non-personalised questionnaire template provided by Eno and the personalisation file provided by the external survey management application. At the start of the collection period, Coltrane therefore generated the same number of blank (but personalised) questionnaires as there were units to survey.

Once the collection period for a survey is open, Coltrane extracts the responses that have been collected since the previous extraction, with the frequency chosen by the survey manager (in practice, up to 4 times a day) then sends them (in XML format) to the survey edit tool. Some survey managers have also chosen to retrieve the “proof of submission” PDF files.

In contrast, Coltrane does not offer a post-collection response archiving service. The responses of the surveyed units are only stored on the platform for three months after the end of the collection period; this allows survey clerks to access the raw data throughout the entire data validation phase. They are then deleted, with the onus on the survey manager to save them in their own environment, where necessary.

To successfully carry out the collection for a survey, it is essential to contact the businesses regularly, to remind them of their obligation to respond to labelled official statistical surveys, for example. Usually, there are four letter templates: start of the collection period, reminder, formal notice and non-response report. Coltrane also provides a letter template for additions to the panel and another to thank outgoing panellists.

During the Coltrane project, INSEE developed a module to create “ready-to-print” letters. This “Mail Module” is now available for any other application with similar needs. For surveys collected using Coltrane:

As before, Coltrane relies on an XML personalisation file (list and personalisation elements of the units for which the letter must be sent, type of letter) provided by the survey management application, to produce a file that it then sends to the Mail Module.

The Mail Module uses the templates (start of the collection period, for example) and the Coltrane file to edit personalised letters in PDF format, as well as information useful for managing the sending of the letters.

However, Coltrane is not a generic collection management tool. Thus:

The letters may be accompanied by a paper questionnaire, depending on the collection strategy adopted. Experience shows, for example, that to increase the internet response rate, it is preferable not to include a paper questionnaire at the start of the collection period: the internet response rate for the ACEMO survey thus increased from 20% to 80% in the year when the DARES decided to no longer send a “paper” questionnaire in the initial mailing. In contrast, it is preferable to send a paper questionnaire at the time of formal notice to avoid any later response by the business. It must also be possible to send a paper questionnaire when the respondent requests. To do so, the procedure is the same as for a usual letter, except that the template shown in the Mail Module is generated by Eno, as was the web collection medium. Coltrane will kind of “direct mail” the questionnaire template.

At the start of the collection periods or at the time of reminders, Coltrane also makes it possible to supplement the sending of a letter with an email, for those contacts with an email address, requesting that they respond to the survey.

The logic is the same as for sending letters: using pre-defined letter templates and an XML personalisation file provided by the survey management application, Coltrane generates a message that is distributed via an INSEE automatic email sending tool. This message includes information on the survey concerned (name, objectives, expected return date, etc.), extracts from legal texts relating to the surveys and, above all, the link to access the authentication portal, a reminder of the contact’s username and a forgotten password link, so as to reduce assistance requests.

By convention, the sender of these emails is the collection office for the survey, which allows the respondent to have easy access to assistance.

In addition, in less than a year it will also be possible to send the questionnaire in PDF format by email to a business that so wishes.

This service offer will, of course, be all the more effective if the survey project manager has an up-to-date contact database containing as many valid email addresses as possible (invalid email addresses are also reported to the survey designers who use the “email sending” service).

Coltrane is more than just a user-friendly collection platform that facilitates the work of survey designers. It is also a contact repository that will be used by survey clerks who process responses and are in contact with surveyed businesses.

To that end, when a survey is integrated into the system, Coltrane imports massively the survey contacts already known (in particular when internet collection was already in use). Then, in order to bring this repository to life, the survey clerks use a “Contact Management” application: to create a contact, to amend its characteristics, to give the contact the right to respond to a survey or withdraw that right, etc. In addition, updates to the contact’s details, made by the respondents directly (via the “My Account” section of the Coltrane interface) are recorded in real time in this repository.

When a clerk amends a characteristic of a contact responding to several surveys, the managers of the other surveys are notified by email. The application is also accessible through the INSEE-Contact hubs to assist the respondent (loss of access code, for example).

Furthermore, the clerks of a survey can view the questionnaires of their respondents regardless of their status (already sent to INSEE or not). In particular, this makes it possible to help businesses that have difficulty answering or to verify the sending or simply the saving of a questionnaire.

In 2019, Coltrane was up and running with the integration of all business surveys carried out by INSEE (with the exception of those collected through the deposit/withdrawal of questionnaires) and by the DARES, offering collection via the internet. The number of surveys carried out using this system has increased steadily over the years.

The objective is to maximise the use of Coltrane for Official Statistical System surveys, so that businesses that respond to multiple surveys need use only one infrastructure. If this is achieved, more than a million questionnaires will be processed using Coltrane annually. In 2019, 900,000 questionnaires will have been accessible on the platform (Figure 3) and the figure of 500,000 returned questionnaires should be achieved for the first time. In addition, since 2015, more than 615,000 contacts have been registered in the repository.

Figure 3. Number of Questionnaires Offered on Coltrane since 2017 by Survey Type

This significant volume creates robustness requirements on the site, the activity of which is not uniform throughout the year, or even during a day (Figure 4 shows daily collection peaks in the late morning and late afternoon).

Figure 4. Maximum Number of Connections Per Hour Between 08:00 and 19:00 in 2019 on the Coltrane Website

Reading note: in 2019, up to 1400 connections were recorded between 11:00 and 12:00 on the Coltrane response portal. This is also the highest number ever achieved in one hour in one day in 2019.

The implementation of Coltrane has resulted in the industrialisation, pooling and professionalisation of collection.

The Eno generator provides standardised media, implementing the rules of best practice for internet questionnaires (Ouvrir dans un nouvel ongletMorrison, 2010; Snijkers, 2013; Dillman 2014). These rules are managed at INSEE by a specialised unit within the Methodology Directorate, which monitors these subjects internationally. This work makes it possible to have collection media that meet the standards in force for both paper and online media. It therefore leads to a homogenisation of practices and of the ergonomics of the media for the users of this system.

Collection is thus more robust, as this specialisation makes it possible to learn more from experience and to better understand the collection method effects or the possible impacts of the ergonomics on responses.

However, this standardisation is sometimes not well received by some designers who view this requirement as a reduction in their room for manoeuvre, in terms of questionnaire design or question formulation. In fact, the standardisation imposed by this new system often leads to the abandonment of old formatting choices, which allowed space savings and a reduction in the costs associated with mass mailing in paper format, but were not always accompanied by the required legibility.

To help them enter this new approach, the designers are guided by documentation presenting the possibilities of the Eno generator, best practices in internet questionnaires and a questionnaire creation protocol. In addition, they are supported in their journey by experts specialising in the profession of collection. Thanks to the services offered by the Mail Module, they also avoid the often tedious task of designing accompanying letters.

Ultimately, this makes it possible to offer more homogeneous collection media and consistent letters; a service that renders the questionnaires more accessible to respondents. The selection of common ergonomics is also a major asset for more easily identifying questionnaires as coming from the official statistics body.

This sharing is only a first step. There is another project in progress, aimed at standardising management applications for thematic surveys and sharing post-collection edit tools. It fits directly within the Pogues/Eno/Coltrane system and will complete the task of limiting the burden on the designer of describing the questionnaire: the same occurrence will be common to all phases of the survey, from Collection to Processing, in the sense of the GSBPM.

The Coltrane platform is dedicated to business surveys, which have their own specific characteristics. However, its application to household surveys is not necessarily a given (see the article by Jean-Marc Béguin on the emergence of innovation in statistical production). Nevertheless, the modular nature of Coltrane and the fact that these modules are now proven through years of use makes it possible to envisage some form of reuse, even if only partial. Thus, as part of INSEE’s work on the multimodal collection of household surveys, it is envisaged that the “collection by internet” portion will be carried out through a dedicated portal, and that this portal will reuse a significant number of investments already made for Coltrane. This will not only make it possible to pool, and therefore reduce, the maintenance resources of both applications, thereby increasing flexibility, but also, given the strategic importance, to give people specialist training on the technologies used and thus secure the entire INSEE digitised collection process.

Coltrane could also benefit, symmetrically, from developments made for this “household” portal and thus increase its service offer. In particular, the work on responsive design, which was taken into account by Coltrane in 2019, or the work on the public site promoting household surveys, the principle of which could be adapted to the needs of business surveys, are worthy of mention.

In addition to the potential for sharing, eventually, with household surveys, other functions are being examined in order to further improve the services offered to respondents and designers.

For businesses, the team is examining a tool to help codification or self-completion. This concerns the questions asked to businesses that must lead to the selection of a code in a classification, sometimes with very many modalities (such as NACE, for example). Currently, the contact of the surveyed unit enters a label, which is then coded in the survey management application (therefore, outside Coltrane). An online codification support tool would be able to facilitate the response to this type of question.

Firstly, the implementation of self-completion to enter the commune code in the contact address is envisaged. To apply it to a questionnaire, it will first be necessary to measure the collection mode effect it generates, so as to be able to correct it. At present, in the absence of input assistance and obstructive checks in a response portal, the hypothesis that there is no impact by the collection mode between an internet questionnaire and a paper questionnaire seems plausible (although it has not been formally tested). However, with the implementation of the self-completion process, this hypothesis fails as this tool is being developed with the hope of improving (and therefore modifying) response quality.

A second area to help with input is envisaged for responding to tables containing a significant number of boxes (the value of which is taken from the business’s information system, incidentally). In this case, the idea would be to propose importing a file rather than entering each box in the table. For large businesses that are systematically surveyed and for repeated surveys, writing a small program to make it possible to produce the file in question would most likely be profitable.

For designers, new functions associated with collection management will be proposed to survey managers in order to make them even more autonomous. Thus, a new interface will be developed so that they are able to enter the metadata required to generate and use the collection media (name and objectives of the survey and start, expected return and end dates for the campaign), metadata which manage the display of data in the response portal themselves without having to use Coltrane’s maintenance teams. These functions will be opened up to designers outside INSEE.

These developments are all the more important as the Coltrane platform is now the standard for internet business surveys. It is thus imposed on designers, so it is logical that in return for this constraint, Coltrane facilitates their work through a sufficiently broad range of services.