Test-Retest

It's hard to find stories on statistical and methodological topics. Here's one such rare story.

The New York Times ran a poll (N = 1,554, Sampling error = + or - 3%) on Hillary Clinton in early July, 2007. One question, "Looking back, do you think the United States did the right thing in taking military action against Iraq, or should the United States have stayed out?"

The results surprised the pollsters because more respondents agreed with the question (42%) in July compared to respondents to the same question (35%) in a May poll. All of the other war-related questions had shown no such change.

The pollsters finally concluded they needed to retest the question. The did so after removing a question about Clinton's repudiation of her 2002 vote against the war. In the original poll, that question was near the military action question.

The second poll (N = 889, Sampling error = + or - 3%) produced nearly identical results with 42% agreeing. In this second poll, another question also showed a change from the earlier May data. Fewer respondents (66%) thought that "things are going badly for the United States in its efforts to bring stability and order to Iraq" than in May (76%). After running the second poll, the pollsters were confident they had detected a change in public opinion. However, they did not know what was causing the change.

Here is a link to the New York Times story