Two years ago, Mother
Jones published a “Guide
to Mass Shootings in America,” which claimed to show that such
slayings happen more often now than in the past. The feature, whose
data have been updated periodically since it first went up, has
been both
widely cited and
harshly disputed. This week the outlet has published a
new analysis claiming that mass shootings occur far more
frequently than they did decades ago and that the rate has tripled
since 2011. Whether you buy that depends on whether you buy into
the previous piece, because the new article has essentially the
exact same flaws.
The best critique I’ve seen of the original Mother
Jones article was written by Michael Siegel. You should read
the whole thing, but
this is the key passage:
It is a truism of science that the more narrowly you
define your sample and the more you shrink the number of data
points, the less reliable your conclusions will be. If you were to
analyze all gun shootings and violence over the last thirty years,
you’d have hundreds of thousands of data points to base your
conclusions on. You could, as I like to say, achieve Victory
Through Sheer Data Volume. But when you start parsing the data down
further and further, you become more prone to random variation and
even bias.Even if we take Mother Jones’ data at face value, we can see we’re
dealing with less than 120 victims every year and frequently less
than 20. That’s an awfully small number to be drawing
conclusions from. To illustrate why, take the Virginia Tech
killings. 56 people were killed or wounded. That is more than all
but five entire years in their database. Something like that is
simply going to swamp the statistics.But we shouldn’t even take Mother Jones’ data at face value because
it is highly suspect. First, it seems to be based on media
coverage, which is not exactly an objective source and almost
certainly leaves shootings out….Everywhere, they make arbitrary
cuts to exclude murders that may not fit their conclusions. They
limit the sample to lone shooters, but make exceptions for
Columbine and Westside. They exclude gang activity and other crimes
but include the Fort Hood Shootings, which were an act of
terrorism….They arbitrarily throw in a few spree killings.This is simply not a representative sample. It’s cherry-picked to
fit a definition, but leaves huge gaping biases all over the place.
Mother Jones doesn’t even acknowledge this.All this would be fine if you wanted to create an illustrative or
representative sample. This is even fine if you want to draw some
broad and overwhelming conclusions such as that most spree killers
get their guns legally. But the low numbers and the biases blow up
in your face when you try to do a more rigorous analysis….They’ve
narrowed the sample so far down that they are essentially looking
at noise.
The new analysis looks at the intervals between each incident
rather than the annual numbers of crimes and victims. But aside
from the fact that the list has been updated through 2014, this is
the same data as the original article, with all the problems that
Siegel and others pointed out before.
The Mother Jones team does make one reasonable point,
in the course of arguing against those of us who
aren’t convinced mass shootings have been getting more
common:
So why do we keep hearing in the media that mass
shootings have not increased?This view stems from the work of Northeastern University
criminologist James Alan Fox, who has long maintained that mass
shootings are a stable phenomenon. (“The growing menace lies more
in our fears than in the facts,” he has said.) But Fox’s oft-cited
claim is based on a misguided approach to studying the problem: The
data he uses includes all homicides in which four or more
people were murdered with a gun. His analysis, which counts the
number of events per year, lumps together mass shootings in public
places with a far more numerous set of mass murders that are
contextually distinct—a majority of which stem from domestic
violence and occur in private homes.
It is true that Fox’s data cover a lot of incidents that don’t
fit the standard conception of a mass shooting, and that it would
be useful to have a more narrowly defined count. The flipside
of this is that Fox’s figures are based on a relatively solid
source—police reports, which are regularly collected and
tabulated—and thus avoid the problems with relying on press
accounts for your list of incidents.
The best alternative measurement that I’m aware of comes from
Grant Duwe, a criminologist at the Minnesota Department of
Corrections. His definition of mass public shootings does
not make the various one-time exceptions and other jerry-riggings
that Siegel criticizes in the Mother Jones list; he simply
keeps track of mass shootings that took place in public and were
not a byproduct of some other crime, such as a robbery. And rather
than beginning with a search of news accounts, with all the gaps
and distortions that entails, he starts with the FBI’s Supplementary Homicide
Reports to find out when and where mass killings happened, then
looks for news reports to fill in the details. According to Duwe,
the annual number of mass public shootings declined from 1999 to
2011, spiked in 2012, then regressed to the mean.
Finally, a note on why this matters. Violent crime rates have
been moving downward for decades now, and mass shootings—by any
definition—are a very rare phenomenon. I’ve heard arguments from
one direction that there’s no point in putting such a small risk
under a microscope when the most pressing threats to people’s lives
lie elsewhere. I’ve heard arguments from another direction that
even one crime this horrible is too many, and that the effect of
noting how infrequently it happens is just to discourage people
from trying to prevent it.
To the first set of arguments, I say that when the press and
politicians present a problem like this as a rising crisis, it’s
worthwhile to see whether it is indeed rising. To the second set of
arguments, I say that absolutely nothing I’ve said here means we
shouldn’t try to prevent future mass murders. Plane crashes are
extremely rare, but that doesn’t mean airlines don’t look for
ways to make them even less likely. If a measure genuinely makes
people safer without creating an intolerable trade-off, I’m for
it.
Such measures are most likely to be incremental changes adopted
at particular places (such as schools) and then imitated elsewhere,
not big anti-crime bills rushed into law by national politicians
eager to be seen Doing Something. But there may well be ways
federal or state officials can make that experimentation and
imitation easier. Good ideas are good—and bad ideas are bad—whether
or not mass shootings are getting more common.
Past Reason coverage of the issue:
• “Are
Mass Shootings Becoming More Common in the United States?”
(December 2012)
• “Making
Sense of Mass-Shooting Statistics” (January 2013)
• “Life During
Wartime” (May 2013)
• “Why
Can’t Anyone Agree How Many Mass Shootings There Have Been In
2013?” (September 2013)
• “Why
Mass Shootings Haven’t Ushered In a New Age of Gun Control”
(February 2014)
• “Are
School Homicides ‘Becoming the Norm’?” (June 2014)
• “The FBI Says
‘Active Shooter Incidents’ Are On the Rise. What Does That
Mean?” (September 2014)
from Hit & Run http://ift.tt/11uWJOe
via IFTTT