An example of this is evident in Gallup's annual poll on minority relations. The poll, conducted June 12-18, asked respondents whether they favored or opposed affirmative action for women, and separately asked whether they favored or opposed affirmative action for racial minorities. Knowing that question order can sometimes influence responses, we ran a split-sample experiment: Half the respondents were asked the questions in the order just outlined, while the other half were asked the questions in reverse order.
The results indicate that question order made a significant difference. Respondents who were asked first about affirmative action for women favored it by a margin of 62% to 30% -- a 32-percentage-point spread. But the other half of the sample, asked the same question after a similar question about racial minorities, favored it by a much smaller margin: 56% to 39%, only a 17-point spread.
|
Affirmative Action Programs for Women |
||
|
|
Question asked first |
Question asked after |
|
% |
% |
|
|
Favor |
62 |
56 |
|
Oppose |
30 |
39 |
A reverse pattern is evident on the questions about racial minorities. When that question is asked first, Americans are more opposed (48%) than in favor (45%). But when it is asked after the question about women, respondents support affirmative action for racial minorities -- by 53% to 38%.
|
Affirmative Action Programs for Racial Minorities |
||
|
|
Question asked first |
Question asked after a similar question about women |
|
% |
% |
|
|
Favor |
45 |
53 |
|
Oppose |
48 |
38 |
These differences are not trivial. In the case of affirmative action for racial minorities, either we have a divided public (when the question is asked first), or we have a clear majority in favor -- a 15-point margin -- when the question is asked second.
There is majority support for affirmative action for women regardless of question order, but support is at the 2-to-1 level when that question is asked first, and only 1.5-to-1 when it is asked second.
Why does support for affirmative action for minorities increase after the question about women is asked, and support for affirmative action for women decrease after the question about racial minorities is asked?
One answer is that the context changes dramatically when the second question is posed. Now respondents consider not only the merits of this second question, but also what they said in the first question. And people usually want to be fair.
Thus, if someone has just said they support affirmative action for women, they now feel more obligated to say they support the same benefits for racial minorities (even if they don't). Similarly, if people have just said they oppose affirmative action for racial minorities, they would be less inclined to turn around and now support affirmative action for women.
Not everyone feels they have to be fair or consistent, of course, but enough respondents feel that way to cause a change in the overall opinion measurement.
|
|
Question asked |
Question asked |
|
Percent of Respondents Who SUPPORT Affirmative Action Programs for: |
% |
% |
|
Women |
62 |
56 |
|
Minorities |
45 |
53 |
|
(Difference) |
17 |
3 |
The accompanying chart helps to illustrate these points. It shows that Americans are clearly more likely to support affirmative action for women than for racial minorities. When the questions are asked about each group first -- and no mention has been made of the other group -- 62% of respondents favor the program for women, but only 45% support it for racial minorities -- a 17-point difference.
But when the questions about each group are asked second -- so that now respondents are comparing their responses to the second question with their responses to the first one -- 56% of respondents favor the program for women, and 53% support it for racial minorities -- just a three-point difference. In this "comparative context," the public is more consistent in its support of affirmative action.
What do we draw from these results? When discussion about affirmative action focuses only on racial minorities, then public support for this program is about evenly divided -- even leaning against rather than in favor. This is the real-world context of the past couple of weeks, as the Supreme Court announced its decision about the University of Michigan's admission process designed to increase diversity on campus.
Similarly, if the focus is only on affirmative action for women, public support is quite substantial.
However, if the context of the debate is about affirmative action for both women and minorities, some people try to be consistent -- supporting the program for minorities because they support it for women, or opposing the program for women because they oppose it for minorities. This dynamic in the "comparative" context causes the different responses we get from rotating the question order.
Final Note: A comparison of these effects by race and gender shows that blacks and women are consistent in their level of support for affirmative action for women and for racial minorities. Only the responses of white men are affected by question order.