But beginning in 2014, the Census Bureau redesigned the income questions in response to findings that the survey has misclassified and generally underreported income—in particular, sources of retirement income.

The survey redesign did capture more income, especially pension income, according to EBRI.

However, it also significantly lowered the survey’s estimates of retirement plan participation among those most likely to participate.

Those downward estimates contradicted results of other surveys on retirement plan participation, which instead indicated an upward trend.

Because, when the survey’s questionnaire was changed, it “was lengthened and new information was given to respondents when asking the additional income-defining questions,” EBRI’s report said, “[c]onsequently, more nonresponses could result, and if the respondents did answer, a higher likelihood of incorrect information could be collected due to respondent fatigue and confusion from new questions.”

EBRI’s report said that, while the redesign of the CPS questionnaire “achieved one of its primary goals of capturing more income—especially pension income,” the changes that were made affected results from unchanged areas of the survey.

The report concluded, “Unless further survey modifications are made, what has happened in the most recent years to the estimated retirement plan participation levels from CPS and what that means for the future estimates from the survey will make it difficult to judge any future effects of policies addressing workers’ retirement plan coverage using the CPS…. . Therefore, the use of administrative data—data from the actual administrators of the program or accounts being examined—has become and will continue to be an increasingly important addition to the evaluation of various economic and government program issues facing the United States.”