From the abstract of Bucolo and Cohn 2010 (gated, ungated):

'Playing the race card' reduced White juror racial bias as White jurors' ratings of guilt for Black defendants were significantly lower when the defence attorney's statements included racially salient statements. White juror ratings of guilt for White defendants and Black defendants were not significantly different when race was not made salient.

The second sentence reports that white mock juror ratings of guilt were not significantly different for black defendants and white defendants when race was not made salient, but the first sentence claims that "playing the race card" reduced white juror racial bias. But if the data can't support the inference that there is bias without the race card ("not significantly different"), then how can the data support the inference that "playing the race card" reduced bias?

For the answer, let's look at the Results section (p. 298). Guilt ratings were reported on a scale from -5 (definitely not guilty) to +5 (definitely guilty):

A post hoc t test (t(75) = .24, p = .81) revealed that ratings of guilt for a Black defendant (M = 1.10, SD = 2.63) were not significantly different than ratings of guilt for a White defendant (M = .95, SD = 2.92) when race was not made salient. When race was made salient, a post hoc t test (t(72) = 3.57, p =.001) revealed that ratings of guilt were significantly lower for a Black defendant (M = -1.32, SD = 2.91) than a White defendant (M = 1.31, SD = 2.96).

More simply, when race was not made salient, white mock jurors rated the black defendant roughly 5% of a standard deviation more guilty than the white defendant, which is a difference that would often fall within the noise created by sampling error (p=0.81). However, when race was made salient by playing the race card, white mock jurors rated the black defendant roughly 90% of a standard deviation less guilty than the white defendant, which is a difference that would often not fall within the noise created by sampling error (p=0.001).

---

Here is how Bucolo and Cohn 2010 was described in a 2013 statement from the Peace Psychology division of the American Psychological Association:

Ignoring race often harms people of color, primarily because biases and stereotypes go unexamined. A study by Donald Bucolo and Ellen Cohn at the University of New Hampshire found that the introduction of race by the defense attorney of a hypothetical Black client reduced the effects of racial bias compared to when race was not mentioned (Bucolo & Cohn, 2010). One error in the state's approach in the George Zimmerman murder trial may have been the decision to ignore issues of race and racism.

But a change from 5% of a standard deviation bias against black defendants to 90% of a standard deviation bias against white defendants is not a reduction in the effects of racial bias.

---

Note that the point of this post is not to present Bucolo and Cohn 2010 as representative of racial bias in the criminal justice system. There are many reasons to be skeptical of the generalizability of experimental research on undergraduate students acting as mock jurors at a university with few black students. Rather, the point of the post is to identify another example of selective concern in social science.

Tagged with: , , ,

Jeffrey A. Segal and Albert D. Cover developed the Segal-Cover scores that are widely used to proxy the political ideology of Supreme Court nominees. Segal-Cover scores are described here (gated) and here (ungated). The scores are based on the coding of newspaper editorials, with each paragraph in the editorial coded as liberal, conservative, moderate, or not applicable (p. 559).

Segal and Cover helpfully provided examples of passages that would cause a paragraph to be coded as liberal, conservative, or moderate. Here is Segal and Cover's first example of a passage that would cause a paragraph to be coded liberal:

Scarcely more defensible were the numerous questions about Judge Harlan's affiliation with the Atlantic Union. The country would have a sorry judiciary indeed, if appointees were to be barred for belonging to progressive and respectable organizations.

Here is Segal and Cover's first example of a passage that would cause a paragraph to be coded conservative:

Judge Carswell himself admits to some amazement now at what he said in that 1948 speech. He should, for his were the words of pure and simple racism.

I can't think of a better example of conservatism than that.

Tagged with: , ,

Vox has a post about racial bias and police shootings. The story by Vox writer Jenée Desmond-Harris included quotes from Joshua Correll, who investigated racial bias in police shootings with a shooter game, in his co-authored 2007 study, "Across the Thin Blue Line: Police Officers and Racial Bias in the Decision to Shoot" (gated, ungated).

Desmond-Harris emphasized the Correll et al. 2007 finding about decision time:

When Correll performed his experiment specifically on law enforcement officers, he found that expert training significantly reduced their fatal mistakes overall, but no matter what training they had, most participants were quicker to shoot at a black target.

For readers who only skim the Vox story, this next sentence appears in larger blue font:

No matter what training they had, most participants were quicker to shoot at a black target.

That finding, about the speed of the response, is fairly characterized as racial bias. But maybe you're wondering whether the law enforcement officers in the study were more likely to incorrectly shoot the black targets than the white targets. That's sort of important, right? Well, Desmond-Harris does not tell you that. But you can open the link to the Correll et al. 2007 study and turn to page 1020, where you will find this passage:

For officers (and, temporarily, for trained undergraduates), however, the stereotypic interference ended with reaction times. The bias evident in their latencies did not translate to the decisions they ultimately made.

I wonder why the Vox writer did not mention that research finding.

---

I doubt that the aggregate level of racial bias in the decision of police officers to shoot is exactly zero, and it is certainly possible that other research has found or will find such a nonzero bias. Let me know if you are aware of any such studies.

Tagged with: , , ,

describes an experiment:

With more than 1,500 observations, the study uncovered substantial, statistically significant race discrimination. Bus drivers were twice as willing to let white testers ride free as black testers (72 percent versus 36 percent of the time). Bus drivers showed some relative favoritism toward testers who shared their own race, but even black drivers still favored white testers over black testers (allowing free rides 83 percent versus 68 percent of the time).

The title of Ayres' op-ed was: "When Whites Get a Free Pass: Research Shows White Privilege Is Real."

The op-ed linked to this study, by Redzo Mujcic and Paul Frijters, which summarized some of the study's results in the figure below:

Mujcic Frijters

The experiment involved members of four races, but the op-ed ignored results for Asians and Indians. I can't think of a good reason to ignore results for Asians and Indians, but it does make it easier for Ayres to claim that:

A field experiment about who gets free bus rides in Brisbane, a city on the eastern coast of Australia, shows that even today, whites get special privileges, particularly when other people aren't around to notice.

It would be nice if the blue, red, green, and orange bars in the figure were all the same height. But it would also be nice if the New York Times would at least acknowledge that there were four bars.

--

H/T Claire Lehmann

Related: Here's what the New York Times did not mention about teacher grading bias

Tagged with: , , , ,

Christopher D. DeSante published an article in the American Journal of Political Science titled, "Working Twice as Hard to Get Half as Far: Race, Work Ethic, and America’s Deserving Poor" (57: 342-356, April 2013). The title refers to survey evidence that DeSante reported indicating that, compared to hypothetical white applicants for state assistance, hypothetical black applicants for state assistance received less reward for hard work and more punishment for laziness.

The study had a clever research design: respondents were shown two applications for state assistance, and each applicant was said to need $900, but there was variation in the names of the applicants (Emily, Laurie, Keisha, Latoya, or no name provided) and in the Worker Quality Assessment of the applicant (poor, excellent, or no assessment section provided); respondents were then asked to divide $1500 between the applicants or to use some or all of the $1500 to offset the state budget deficit.

Table 1 below indicates the characteristics of the conditions and the mean allocations made to each alternative. In condition 5, for example, 64 respondents were asked to divide $1500 between hardworking Laurie, lazy Emily, and offsetting the state budget deficit: hardworking Laurie received a mean allocation of $682, lazy Emily received a mean allocation of $566, and the mean allocation to offset the state budget deficit was $250.

DeSanteReproductionTable1blog

---

I'm going to quote DeSante (2013: 343) and intersperse comments about the claims. For the purpose of this analysis, let's presume that respondents interpreted Emily and Laurie as white applicants and Keisha and Latoya as black applicants. Reported p-values for my analysis below are two-tailed p-values. Here's the first part of our DeSante (2013: 343) quote.

Through a nationally representative survey experiment in which respondents were asked to make recommendations regarding who should receive government assistance, I find that American “principles” of individualism, hard work, and equal treatment serve to uniquely benefit whites in two distinct ways. First, the results show that compared to African Americans, whites are not automatically perceived as more deserving of government assistance.

Condition 7 paired Laurie with Keisha, neither of whom had a Worker Quality Assessment. Laurie received a mean allocation of $556, and Keisha received a mean allocation of $600. Keisha received $44 more than Laurie, a $44 difference that is statistically significant at p<0.01. So DeSante is technically correct that "whites are not automatically perceived as more deserving of government assistance," but this claim overlooks evidence from condition 7 that a white applicant was given LESS government assistance than an equivalent black applicant.

Instead of reporting these straightforward results from condition 7, how did DeSante compare allocations to black and white applicants? Below is an image from Table 2 of DeSante (2013), which reported results from eleven t-tests. Tests 3 and 4 provided the evidence for DeSante's claim that, "compared to African Americans, whites are not automatically perceived as more deserving of government assistance."

DeSante2013Table2

Here's what DeSante did in test 3: DeSante took the $556 allocated to Laurie in condition 7 when Laurie was paired with Keisha and compared that to the $546 allocated to Latoya in condition 10 when Latoya was paired with Keisha; that $9 advantage (bear with the rounding error) for Laurie over Latoya (when both applicants were paired with Keisha and neither had a Worker Quality Assessment) did not reach conventional levels of statistical significance.

Here's what DeSante did in test 4: DeSante took the $587 allocated to Emily in condition 4 when Emily was paired with Laurie and compared that to the $600 allocated to Keisha in condition 7 when Keisha was paired with Laurie; that $12 advantage for Keisha over Emily (when both applicants were paired with Laurie and neither had a Worker Quality Assessment) did not reach conventional levels of statistical significance.

So which of these three tests is the best test? My test had more observations, compared within instead of across conditions, and had a lower standard error. But DeSante's tests are not wrong or meaningless: the problem is that tests 3 and 4 provide incomplete information for the purposes of testing for racial bias against applicants with no reported Worker Quality Assessment.

---

Here's the next part of that quote from DeSante (2013: 343):

Instead, the way hard work and "laziness" are treated is conditioned by race: whites gain more for the same level of effort, and blacks are punished more severely for the same level of "laziness."

Here's what DeSante did to produce this inference. Emily received a mean allocation of $587 in condition 4 when paired with Laurie and neither applicant had a Worker Quality Assessment; but hard-working Emily received $711 in condition 6 when paired with lazy Laurie. This $123 difference can be interpreted as a reward for Emily's hard work, at least in relation to Laurie's laziness.

Now we do the same thing for Keisha paired with Laurie: Keisha received a mean allocation of $600 in condition 7 when paired with Laurie and neither applicant had a Worker Quality Assessment; but hard-working Keisha received $607 in condition 9 when paired with lazy Laurie. This $7 difference can be interpreted as a reward for Keisha's hard work, at least in relation to Laurie's laziness.

Test 7 indicates that the $123 reward to Emily for her hard work was larger than the $7 reward to Keisha for her hard work (p=0.03).

But notice that DeSante could have conducted another set of comparisons:

Laurie received a mean allocation of $556 in condition 7 when paired with Keisha and neither applicant had a Worker Quality Assessment; but hard-working Laurie received $620 in condition 8 when paired with lazy Keisha. This $64 difference can be interpreted as a reward for Laurie's hard work, at least in relation to Keisha's laziness.

Now we do the same thing for Latoya paired with Keisha: Latoya received a mean allocation of $546 in condition 10 when paired with Keisha and neither applicant had a Worker Quality Assessment; but hard-working Latoya received $627 in condition 11 when paired with lazy Keisha. This $81 difference can be interpreted as a reward for Latoya's hard work, at least in relation to Keisha's laziness.

The $16 difference between Laurie's $64 reward for hard work and Latoya's $81 reward for hard work (rounding error, again) is not statistically significant at conventional levels (p=0.76). The combined effect of the DeSante test and my alternate test is not statistically significant at conventional levels (effect of $49, p=0.20), so -- in this dataset -- there is a lack of evidence at a statistically significant level for the claim that "whites gain more for the same level of effort."

I conducted a similar set of alternate tests for the inference that "blacks are punished more severely for the same level of "laziness"; the effect size was smaller in my test compared to DeSante's test, but evidence for the the combined effect was believable: a $74 effect, with p=0.06.

---

Here's the next part of that quote from DeSante (2013: 343):

Second, and consistent with those who take the "principled ideology" approach to the new racism measures, the racial resentment scale is shown to predict a desire for smaller government and less government spending. However, in direct opposition to this ideology-based argument, this effect is conditional upon the race of the persons placing demands on the government: the effect of racial resentment on a desire for a smaller government greatly wanes when the beneficiaries of that government spending are white as opposed to black. This represents strong evidence that racial resentment is more racial animus than ideology.

DeSante based this inference on results reported in Table 3, reproduced below:

DeSante2013Table3

Notice the note at the bottom: "White respondents only." DeSante reported results in Table 3 based on responses only from respondents coded as white, but reported results in Table 2 based on responses from respondents coded as white, black, Asian, Native American, mixed race, or Other. Maybe there's a good theoretical reason for changing the sample. DeSante's data and code are posted here if you are interested in what happens to p-values when Table 2 results are restricted to whites and Table 3 results include all respondents.

But let's focus on the bold RRxWW line in Table 3. RR is racial resentment, and WW is a dichotomous variable for the conditions in which both applicants were white. Model 3 includes categories for WW (two white applicants paired together), BB (two black applicants paired together), and WB (one white applicant paired with one black applicant); this is very important, because these included terms must be interpreted in relation to the omitted category that I will call NN (two unnamed applicants paired together). Therefore, the -337.92 coefficient on the RRxWW variable in model 3 indicates that -- all other model variables held constant -- white respondents allocated $337.92 less to offset the state budget deficit when both applicants were white compared to when both applicants were unnamed.

The -196.43 coefficient for the RRxBB variable in model 3 indicates that -- all other model variables held constant -- white respondents allocated $196.43 less to offset the state budget deficit when both applicants were black compared to when both applicants were unnamed. This -$196.43 coefficient did not reach statistical significance, but the coefficient is important because the bias in favor of the two white applicants relative to the two black applicants is only -$337.92 minus -$196.43; so whites allocated $141.49 less to offset the state budget deficit when both applicants were white compared to when both applicants were black, but the p-value for this difference was 0.41.

---

Here's a few takeaways from the above analysis:

1. The limited choice of statistical tests reported in DeSante (2013) produced inferences that overestimated the extent of bias against black applicants and missed evidence of bias against white applicants.

2. Takeaway 1 depends on the names reflecting only race of the applicant. But the names might have reflected something other than race; for instance, in condition 10, Keisha received a mean allocation $21 higher than the mean allocation to Latoya (p=0.03): such a difference is not expected if Keisha and Latoya were "all else equal."

3. Takeaway 1 would likely not have been uncovered had the AJPS not required the posting of data and replication files from its published articles.

4. Pre-registration would eliminate suspicion about research design decisions, such as decisions to restrict only some analyses to whites and to report some comparisons but not others.

---

In case you are interested in reproducing the results that I discussed, the data are here, code is here, and the working paper is here. Comments are welcome.

---

UPDATE (Nov 2, 2014)

I recently received a rejection for the manuscript describing the results reported above; the second reviewer suggested portraying the raw data table as a graph: I couldn't figure out an efficient way to do that, but the suggestion did get me to realize a good way to present the main point of the manuscript more clearly with visuals.

The figure below illustrates the pattern of comparison for DeSante 2013 tests 1 and 2: solid lines represent comparisons reported in DeSante 2013 and dashed lines represent unreported equivalent or relevant comparisons; numbers in square brackets respectively indicate the applicant and the condition, so that [1/2] indicates applicant 1 in condition 2.

 

Tests 1 and 2

---

The figure below indicates the pattern of reported and unreported comparisons for black applicants and white applicants with no Worker Quality Assessment: the article reported two small non-statistically significant differences when comparing applicants across conditions, but the article did not report the larger statistically significant difference favoring the black applicant when a black applicant and a white applicant were compared within conditions.

Tests 3 and 4---

The figure below indicates the pattern of reported and unreported comparisons for the main takeaway of the article. The left side of the figure indicates that one of the black applicants received a lesser reward for an excellent Worker Quality Assessment and received a larger penalty for a poor Worker Quality Assessment, compared to the reward and penalty for the corresponding white applicant; however, neither the lesser reward for an excellent Worker Quality Assessment nor the larger penalty for a poor Worker Quality Assessment was present at a statistically significant level in the comparisons on the right, which were not reported in the article (p=0.76 and 0.31, respectively).

Tests Rest---

Data for the reproduction are here. Reproduction code is here.

---

UPDATE (Mar 8, 2015)

The above analysis has been published here by Research & Politics.

Tagged with: , , , ,

I have posted a working manuscript on symbolic racism here, with its appendix here. Comments are welcome and appreciated. I'll outline the manuscript below and give some background to the research.

---

On 27 October 2012, a Facebook friend posted a link to an Associated Press report "AP poll: Majority harbor prejudice against blacks." I posted this comment about the report:

sr1

During the Facebook discussion, I noted that it not obvious that the implicit measurements indicate racism, given the data on implicit preferences among blacks:

sr2

Bob Somersby at the Daily Howler noticed that the AP report provided data disaggregated by political party but failed to provide data disaggregated by race:

Although Ross and Agiesta were eager to tell you how many Democrats, Republicans and independents were shown to hold "anti-black feelings," they never tell you how many black respondents “hold anti-black feelings” as well!

Why didn't our intrepid reporters give us that information? We can't answer that question. But even a mildly skeptical observer could imagine one possible answer:

If substantial percentages of black respondents were allegedly shown to "hold anti-black feelings," that would make almost anyone wonder how valid the AP's measures may be. It would undermine confidence in the professors—in those men of vast erudition, the orange-shoed fellows who still seem to think that Obama trailed in the national polling all through the summer of 2008.

David Moore at iMediaEthics posted data disaggregated by race that he retrieved from the lead author of the study: based on the same method used in the original report, 30 percent of white Americans implicitly held anti-white sentiments, and 43 percent of black Americans implicitly held anti-black sentiments. Moore discussed how this previously-unreported information alters interpretation of the study's findings:

It appears that racism, as measured by this process, is much more complicated than the news story would suggest. We cannot talk about the 56% of Americans with "anti-black" attitudes as being "racist," if we do not also admit that close to half of all blacks are also "racist" – against their own race.

If we accept the measures of anti-black attitudes as a valid indicator of racism, then we also have to accept the anti-white measures as racism.

Moore did not tell us the results for black respondents on the explicit measures of racism, so that's the impetus behind Study 2 of the working manuscript.

---

The explicit racism measure discussed in the AP report is symbolic racism, also known as racial resentment. Instead of explaining what symbolic racism is, I'll show how symbolic racism is typically measured; items below are from the American National Election Studies, but there were more items in the study discussed in the AP report.

Symbolic racism is measured in the ANES based on whether a survey respondent agrees strongly, agrees somewhat, neither agrees nor disagrees, disagrees somewhat, or disagrees strongly with these four items:

1. Irish, Italians, Jewish and many other minorities overcame prejudice and worked their way up. Blacks should do the same without any special favors.

2. Generations of slavery and discrimination have created conditions that make it difficult for blacks to work their way out of the lower class.

3. Over the past few years, blacks have gotten less than they deserve.

4. It's really a matter of some people not trying hard enough; if blacks would only try harder they could be just as well off as whites.

I hope that you can see why these are not really measures of explicit racism. Let's say that non-racist person A opposes special favors for all groups: that person would select the symbolic racist option for item 1, indicating a belief that blacks should work their way up without special favors. Person A is coded the same as a person B who opposes special favors for blacks because of person B's racism. So that's problem #1 with symbolic racism measures: the measures conflate racial attitudes and non-racial beliefs.

But notice that there is another problem. Let's say that person C underestimates the influence of slavery and discrimination on outcomes for contemporary blacks; person C will select a symbolic racism option for item 2, but is that racism? is that racial animosity? is that a reflection that a non-black person -- and even some black persons -- might not appreciate the legacy of slavery and discrimination? or is that something else? That's problem #2 with symbolic racism measures: it's not obvious how to interpret these measures.

---

Researchers typically address problem 1 with control variables; the hope is that placing partisanship, self-reported ideology, and a few conservative values items into a regression sufficiently dilutes the non-racial component of symbolic racism so that the effect of symbolic racism can be interpreted as its racial component only.

In the first part of the working manuscript, I test this hope by predicting non-racial dependent variables, such as opposition to gay marriage. The idea of this test is that -- if statistical control really does sufficiently dilute the non-racial component of symbolic racism -- then symbolic racism should not correlate with opposition to gay marriage, because racism should not be expected to correlate with opposition to gay marriage; but -- if there is a correlation between symbolic racism and gay marriage -- then statistical control did not sufficiently dilute the non-racial component of symbolic racism.

The results indicate that a small set of controls often does not sufficiently dilute the non-racial component of symbolic racism, so results from symbolic racism research with a small set of controls should be treated skeptically. But a more extensive set of controls often does sufficiently dilute the non-racial component of symbolic racism, so we can place more -- but not complete -- confidence in results from symbolic racism research with an extensive set of controls.

---

The way that I addressed problem #2 -- about how to interpret symbolic racism measures -- is to assess the effect of symbolic racism among black respondents. Results indicate that among blacks -- and even among a set of black respondents with quite positive views of their own racial group -- symbolic racism sometimes positively correlates with opposition to policies to help blacks.

Study 2 suggests that it is not legitimate for researchers to interpret symbolic racism among whites differently than symbolic racism among blacks, without some other information that can permit us to state that symbolic racism means something different for blacks and whites. Study 3 assesses whether there is evidence that symbolic racism means something different for blacks and whites.

Tagged with: , ,