A new Harvard study on police shootings and race is causing all kinds of controversy

In his speech addressing the shootings of Alton Sterling and Philando Castile, President Barack Obama cited a litany of statistics showing how minorities are disproportionately targeted by police. Among them: Black people are 30% more likely than whites to be pulled over, and after being pulled over are 3-times more likely to be searched. In 2015, black people were shot by police at twice the rate of white people, and arrested at twice the rate,

So when Harvard economist Roland Fryer published a study Monday that purported to show that police aren’t racist when they’re involved in shootings, many observers were skeptical.

Fryer and his team set out to answer a narrow question: whether there are racial biases in the split-second decision as to whether an officer uses lethal (gun) or non-lethal (taser) force during an encounter with a citizen.

“The empirical thought experiment here is that a police officer arrives at a scene and decides whether or not to use lethal force,” he writes, and continues, “This does not, however, rule out the possibility that there are important racial differences in whether or not these police-civilian interactions occur at all.”

To answer the question, he and his team use two datasets—one, a detailed set of officer-involved shootings from the Houston Police Department, and the other a slightly less detailed summary of officer weapon discharges from Austin, Dallas, Houston, six large Florida counties, and Los Angeles County.

Altogether, Fryer and his team looked at 1,332 shootings between 2000 and 2015, a little more than a third of which were gleaned from the Houston data. They found that, contrary to what many would expect, minorities were slightly less likely to be shot when a shooting occurred: black people are 23.8% percent less likely to be shot by police, relative to white people. Hispanics are 8.5% percent less likely to be shot.

“Even when officers report civilians have been compliant and no arrest was made, blacks are 21.3% more likely to endure some form of force. Yet, on the most extreme use of force – officer-involved shootings – we are unable to detect any racial differences in either the raw data or when accounting for controls.”

Fryer placed some important caveats on his analysis. It is, of course, possible that the departments willingly supplied the data because they were “enlightened” or not concerned about what the analysis would reveal. Three jurisdictions Fryer and his team reached out to declined to participate in the study. “This is likely not a representative set of cities,” Fryer admits.

He also says that bias may be introduced because it is the officers themselves who provide details of the shootings. As Fryer puts it, “Accounting for contextual variables recorded by police officers who may have an incentive to distort the truth is problematic.” The dataset for Houston was also far more extensive than for other jurisdictions, as it included police-civilian interactions in which lethal force may have been justified.

The biggest caveat is that, with the exception of Houston, the researchers could only look at incidents that involved times where an officer did shoot their gun or taser, and not the times they could have but didn’t. “To the extent that racial bias is prevalent on the extensive margin—whether or not someone is ever in an officer-involved shooting—these data would not capture it,” Fryer writes.

Some experts and commentators say that Fryer’s conclusion could be flawed. Writing on his blog, Barnard economist Rajiv Sethi says he’s suspicious of the fact that in the arrest data only 5% of suspects were armed, and yet 56% of suspects were reported to have “attacked or drew weapon.”

“This would suggest that over half of suspects attacked without a weapon,” which Sethi finds hard to believe. He also argues that only the Houston data seem to validate Fryer’s conclusion. “The analysis of other jurisdictions considered in the paper is restricted to encounters in which shootings actually occurred, and cannot therefore be used to answer the same kinds of questions that the Houston data allows,” he writes.

At a minimum, Sethi says, the conclusion that race does not play a factor in police shootings should only be applied to Houston, where the most detailed data came from.

But even in Houston, civil rights activists rejected Fryer’s conclusion. “In all fairness, I don’t think (the study is) a fair picture of what’s going on,” Johnny Mata, presiding officer of the Greater Houston Coalition for Justice, a civil rights group told the Houston Chronicle. “I don’t care how you cut it, the short end of the stick always ends up in the communities of color.”

“We’ve seen too many examples of bystander videos contradicting police accounts,” added Jeffery Robinson, director of the ACLU-Texas Center for Justice.

University of Minnesota sociologist Michelle Phelps writes that the question Fryer sets out to answer—whether race is a factor in the split-second decision to shoot a suspect—is too narrow.

“When we start the analysis by looking at encounters with police, we have already washed away some of the relevant racial bias,” bias that Fryer himself finds in another part of his study. The unique data on police-citizen encounters Fryer relies on from Houston allows him in effect to ‘control’ for the propensity to come into contact with the police in the first place, Phelps says.

Finally, Salim Furth, an economist at the Heritage Foundation, went on a lengthy tweetstorm about the paper that included this observation:

Fryer’s assistant at Harvard said he is currently on paternity leave. But in the paper, Fryer seems to acknowledge these shortcomings. “Black civilians always have a higher likelihood of force being used on them compared to white civilians,” he says. And like any good economist, he says more research is needed.

But he can ultimately not reject his main conclusion. He believes this result can be explained by the extremely high “cost” (in terms of maximizing utility) to an officer for firing his or her weapon.

“In police departments across the country, any officer-involved shooting – no matter how ‘justified’ – results in the temporary confiscation of the officer’s weapon until an investigation of the incident is complete,” he writes. “This is a potentially high cost relative to other non-lethal uses of force. Moreover, in informal interviews with dozens of police officers in Boston, Cambridge, Camden, and Houston – almost all police officers described pulling the trigger of their weapon as a ‘life altering event.'”

Rob covers business, economics and the environment for Fusion. He previously worked at Business Insider. He grew up in Chicago.