Measurement Invariance of Survey Data across Face-to-face and Video Interview Modes

Abstract

The COVID-19 pandemic posed challenges to the traditional mode of administration of surveys. In most European countries it was not feasible to conduct standardized in-person interviews due to restrictions. These events accelerated the trend towards alternative modes of survey data collection, such as live video interviewing. In this paper, we investigate whether survey data collected with two different interviewing modes, face-to-face and video interview, are comparable. Measurement invariance is a prerequisite to make meaningful comparisons across groups. Using data from the European Social Survey Round 10, we assess if the measurement invariance condition is met for two concepts, social trust and attitudes towards immigration, in six European countries: Estonia, Finland, Iceland, Italy, the Netherlands and Norway. The results are encouraging: Full metric invariance is established for all countries for social trust and for three out of six countries (Iceland, Italy and Norway) for attitudes towards immigration. For those cases, analysts can aggregate data across modes to investigate relationships among variables, e.g., using regression analysis. Scalar invariance holds in fewer cases. For the concept social trust, full scalar invariance is established for Finland, Italy, the Netherlands and Norway while for the concept attitudes towards immigration, it is established for Iceland and Italy. For those cases, data can be aggregated across modes to compare observed means at the country-language level. Generally, where measurement equivalence is not reached, we recommend excluding observations from the video interview mode before analysis.