Good will hunting. Predicting response quality using motivation in longitudinal surveys

Valentin Brunel (1) , Jean-Baptiste Portelli

1 CDSP - Centre de données socio-politiques de Sciences Po (Sciences Po, CNRS)

Observers of survey methodology have pointed out declining response rates and decreasing quality of survey as perhaps the most important challenges facing data collection. In order to better understand this phenomenon, we wish to study the impact of respondents’ motivation on response rates and response quality. This study focuses on two longitudinal annual studies collected by the ELIPSS Panel in France from 2013 to 2018. This probability-based longitudinal panel of roughly 3000 respondents collects monthly answers to different social-science surveys. The “ELIPSS Annual survey” collects socio-demographic information. Estimations of data quality and its evolution through time are made using data from those Annual surveys, data from detailed digital practices (in “ELIPSS Digital Practices” studies), response rates in all other surveys and duration of response to each survey. Our research question is : how does motivation influence the chances of staying longer in the panel and of giving quality answers ? Initial motivation is captured in a series of indicators collected during recruitment for the survey. Those indicators are both open-ended questions (with free answers) and grid questions with limited choices. Ongoing motivation is captured throughout Annual surveys by short questions on the study itself and respondent’s appreciation and dedication. All these indicators are used through different methods like a statistic score in order to predict participation (chances of leaving the panel), as well as answer quality (chances of not answering to certain questions, or answering ‘i don’t know’ or ‘i refuse to answer’, as well as duration of response). Two main hypotheses will be tested : that this score of motivation in the panel is correlated with higher participation and better response quality, and secondly, that creating multiple categories of motivation can help us notice differential participation rates and response quality. This communication will present the results of our models in predicting the influence of motivation on survival in the panel and quality of survey answers. We hope it will contribute to the efforts aiming at maximising response rates and data quality. For longitudinal survey, asking about initial motivation when entering the panel could thus become a tool for panel management.

Mots clés: motivation; attrition; longitudinal

Domaines: Méthodes et statistiques

Valentin Brunel, Jean-Baptiste Portelli. Good will hunting. Predicting response quality using motivation in longitudinal surveys. European Survey Research Association, Jul 2019, Zagreb, Croatia. ⟨hal-02874071⟩