Methodological Research

Why we test our methods

Methodological research is the study of how research and evaluation are carried out to ensure this work is effective. The Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) conducts methodological research for three main reasons:

1. To develop and refine methods for evaluating questions
2. To examine how well questions measure what they are intended to measure (construct validity)
3. To increase understanding of how respondents understand, think about, and answer questions (the question response process) and why respondents answer survey questions incorrectly (response error)

There are many evaluation methods. Every method has its own advantages and disadvantages. Researching these methods helps CCQDER ensure the methods we use produce the highest quality evaluation results.

Promoting methodological research

Question evaluation has been a standard practice within the federal statistical system for decades. More broadly, question evaluation is often misunderstood and could be more widely and frequently used.

Working groups

CCQDER is part of several working groups focused on methodological research and question evaluation.

Interagency Response Error Group (IREG) is a U.S. federal interagency group devoted to communicating about sources of measurement error in questionnaires and other data collection instruments.

QUALPOR is an affinity group of the American Association of Public Opinion Research (AAPOR) which shares best practices and discusses new approaches to qualitative research design.

Questionnaire Evaluation Standards (QUEST) is an international working group made up of question evaluators from federal agencies in other countries. QUEST members promote evaluation standards, discuss challenges, and present findings. They also share their recent experiences with questionnaire development and testing.