openpolitics.com

Here’s why I’m skeptical of Roland Fryer’s new, much-hyped study on police shootings

When Fryer (an economist by training) tells the Times that he got interested in police shootings because of “his anger after the deaths of Michael Brown and Freddie Gray,” and (in Fryer’s words) “decided I was going to collect a bunch of data and try to understand what really is going on,” that should be another humongous red flag.

AD

It implies that Fryer assumed he was doing something pioneering, rather than asking first what work was already being done and what he could add to the existing conversation. This is something that often happens when people in “quantitative” social sciences, like economics, develop an interest in topics covered in other social sciences — in this case, criminology: They assume that no rigorous empirical work is being done.

 

 

Ask yourself: How broad is this data? How broad are the claims being made about it?

The Times report does explain the data that Fryer and company used in the new study. But it turns out it’s not nearly as broad a sample as the conclusion “these results undercut the idea that the police wield lethal force with racial bias” would suggest:

(Fryer) and a group of student researchers spent about 3,000 hours assembling detailed data from police reports in Houston; Austin, Tex.; Dallas; Los Angeles; Orlando, Fla.; Jacksonville, Fla.; and four other counties in Florida.

That’s 10 police departments in three states, with a majority of them based in major cities. That allowed Fryer and his team to compile a database with a lot of different shootings — about 1,330 over 16 years, from 2000 to 2015 — but not a lot of different police departments or institutional cultures.

When it comes to policing, this is especially important, because so many issues of crime and policing are local. Different cities have different approaches to police-community relations; different tensions; different standards for use of force. (In fact, the cities Fryer and his team worked with are all members of a White House initiative on policing data launched in 2015 — and the kind of department that thinks data collection and transparency are important is likely to have different priorities in other regards than one that isn’t.)

 

In comparison, the FBI’s Uniform Crime Report database includes records from thousands of police departments around the country. The number of shootings in the Fryer data set, spanning 16 years, is about equivalent to what the Uniform Crime Report compiles in two or three.

More importantly, the UCR report includes not just major cities but small towns and rural areas; not just diverse cities but less diverse ones; not just departments that think carefully about data collection but departments for which it’s just a needed chore to qualify for government funds.

 

Ask yourself: Is the question the study answers the same one the public is asking?

The most revealing passage in the Times article is probably the one explaining what Fryer and his team didn’t include in their study:

It focused on what happens when police encounters occur, not how often they happen. Racial differences in how often police-civilian interactions occur reflect greater structural problems in society.

In other words, Fryer and company found that there weren’t big racial disparities in how often black and white suspects who’d already been stopped by police were killed. But they deliberately avoided the question of whether black citizens are more likely to be stopped to begin with (they are) and whether they’re more likely to be stopped without cause (yup).

Avoiding those issues makes sense for the question Fryer was trying to answer. He wanted to know what happens between the moment a police officer stops someone and the moment he pulls the trigger — and how those sequences of events vary by race.

But when people talk about racial disparities in police use of force, they’re usually not asking, Is a black American stopped by police treated the same as a white American in the same circumstances? They’re making a broader critique of the “greater structural problems” in society in general and the criminal justice system in particular. They’re saying that black Americans are more likely to get stopped by police, which makes them more likely to get killed.

Eric Garner was killed in 2014 when police tried to arrest him for selling loose cigarettes. Philando Castile had been pulled over 52 times on misdemeanors (including for driving without a muffler and not wearing a seatbelt) before he was shot and killed last week. Michael Brown was stopped by Darren Wilson for walking in the middle of the street.

Maybe it’s possible (maybe) that those encounters would have been just as likely to escalate to the point of lethal force if each of those men had been white — but it kind of misses the point to say that, because if they’d been white, the encounters probably never would have happened.

Controlling for variables is an extremely important thing in social science. It allows you to figure out which factors actually matter and which ones don’t. In this case, Fryer and his team have given us suggestive evidence that among major-city police forces, police in tense situations are not unusually likely to shoot black suspects. They’ve made a valuable addition to the literature. But it’s just that: an addition, not a discovery, and not the last word.