Estimating Public Involvement: The Case of Roberts, Miers

Two Gallup surveys, conducted immediately after President Bush announced the nominations of John Roberts and Harriet Miers to the Supreme Court, found the public apparently quite engaged in the confirmation process. Sixty-six percent of Americans said it mattered either "a great deal" or "a moderate amount" to them "whether or not John Roberts is confirmed by the Senate." Sixty-five percent also said that about Miers.

How much does it matter to you whether or not [John Roberts/Harriet Miers] is confirmed by the Senate -- a great deal, a moderate amount, not much, or not at all?

These results show a consistent concern among Americans that is not tied to the specific nominee (because the percentages were so similar), but rather to the office itself.

A CNN/USA Today/Gallup survey in late August found a quite different level of public commitment to the Roberts nomination. Only 47% of Americans said it mattered to them at least a moderate amount, compared with 66% the month before. Another 22% had no opinion on the issue in August, compared with just 3% in July.

What events in the intervening month caused such a decline in public engagement with the Roberts nomination?

The answer: Nothing. The differences are a function of different methods of estimating public engagement in an issue, and demonstrate how elusive that concept can be.

The August questions that measured engagement began with the question that asked whether people favored or opposed Roberts' confirmation. If people did not know or expressed no opinion, they were classified as "no opinion."

People who said they favored confirmation were then asked how much it would "matter" to them if Roberts were not confirmed, while people who opposed confirmation were then asked how much it would "matter" to them if Roberts were confirmed. Many people who initially expressed an opinion then admitted that it would not matter to them if the Senate did the opposite of what they wanted.

As you may know, John Roberts is a federal judge who has been nominated to serve on the Supreme Court. Would you like to see the Senate vote in favor of Roberts serving on the Supreme Court, or not?

(Asked of people who favor confirmation): If the Senate DOES NOT vote in favor of Roberts serving on the Supreme Court, how much would that matter to you -- a great deal, a moderate amount, not much, or not at all?

(Asked of people who oppose confirmation): If the Senate votes in favor of Roberts serving on the Supreme Court, how much would that matter to you -- a great deal, a moderate amount, not much, or not at all?

Twenty-three percent of Americans said they favored Roberts' confirmation, but also said it didn't matter to them if the Senate did not confirm him. Another 8% said they opposed confirmation, but it did not matter to them if Roberts were confirmed. And 22% had no opinion about whether Roberts should be confirmed or not, suggesting that either action would not matter to them.

This method of estimating public commitment suggests that about half of Americans were highly engaged in the process, while the other half were not -- either they didn't have an opinion on whether Roberts should have been confirmed, or it didn't matter to them what action the Senate took. By contrast, when we just ask people directly how much it matters to them "whether or not" the justice is confirmed, about two-thirds of the public indicated that the confirmation mattered.

Why the differences? I can think of two possible explanations.

First, the two methods could be measuring slightly different concepts. The first asks how much it matters whether a candidate is confirmed, which could be interpreted by some, but not necessarily all respondents as asking how much it matters that the position itself be filled. Some could say that whether a candidate is confirmed matters to them, in the sense that the Senate eventually has to confirm someone, but then still admit that it doesn't matter to them what specific person is confirmed.

A second explanation might be that when we ask people directly (as we do with the first method) how much a Supreme Court nomination "matters" to them, the question carries with it a "socially desirable" response. Many people may not want to admit that such an important process does not matter to them. On the other hand, after we go through a more involved process, allowing people the opportunity to express a viewpoint (whether they favor or oppose confirmation), and then ask if it "matters" that their view prevail, many people are more willing to admit that they really don't care that much after all.

Which is the better question?

I prefer the second method, because I think it gets more directly at what we are trying to measure when we ask whether the nomination of Roberts or Miers "matters" to people. We want to know how much they care whether each candidate is confirmed, not whether they think the position itself needs to be filled.

As the results from the second method show, despite all the media attention devoted to the confirmation process of each candidate, about half of all Americans feel the nomination of Roberts (and by inference, Miers as well) doesn't really matter to them.