[122] Arresting Flexibility: A QJE field experiment on police behavior with about 40 outcome variables
A forthcoming paper in the Quarterly Journal of Economics (QJE), "A Cognitive View of Policing" (htm), reports results from a field experiment showing that teaching police officers to "consider different ways of interpreting situations they encounter" led to "reductions in use of force, [and] discretionary arrests" (abstract).
In this post I explain why, having spent a few of hours comparing the pre-registration with the published results, I think the findings are not really statistically significant. As a preview, two findings described as focal in the paper (but not in the pre-registration), are p = .048 and p = .051.
First, true praise I believe this is exactly the kind of study we should strive to have more of in social science. The intervention is directly informed by well-established psychological findings and was applied in a context of obvious societal relevance. I admire the authors' choice of research question, study design, and logistically complex and expensive implementation. I would love it if half of social science studies were like this one, and our best journals published the work even if the results were inconclusive (as I suspect these results actually are).
In terms of (1), the research community is unlikely to check the validity of this particular paper because (i) it is prohibitively costly for other researchers to team up with a police department just to run a replication and (ii)_ the authors indicated the data were proprietary and did not share any of it.
