|Takeaway Point: Investigation of the studies underlying claims such as “people who keep guns in homes are almost 3 times more likely to be murdered” (Brady Campaign) and “females living with a gun in the home were 2.7 times more likely to be murdered than females with no gun at home” (New York Times) reveal these assertions to be highly problematic. These simple statistics are not to be taken at face value, for reasons I discuss in this blog entry.In fact, according to the same study cited by the Brady Campaign (Kellermann), people in the case sample were 62 times more likely to be killed in circumstances other than in their own home with a gun they kept there.We are better off distinguishing between homicides involving firearms and suicides involving firearms, since the dynamics of these two acts (notably the effect of gun ownership on the outcomes) are quite different. With respect to the former (homicides), we should focus on the dynamics of gun violence among high-risk individuals, especially those involved in criminal activity and those with a history of non-lethal violence (both of which include but are not limited to domestic violence).More research needs to be done in these areas, especially by individuals less ideologically invested in opposition to guns. Also, those who are ideologically pro-gun might be less reluctant about federal funding of this research if the researchers themselves were more modest about what their findings actually say rather than using oversimplifications to press for political agendas with respect to guns.For example, Wiebe – whose study discussed below was cited by the New York Times – writes both of the following in the same article:
Every day I open my Twitter feed and find statistics on the risks of gun ownership. (E.g., the infographic below from Care2.com.) Given a 140 character limit, most of the time these statistics are not supported by further documentation. This is often the case with politicians, advocacy groups, and the media as well – as befits the culture of fear.
But sometimes people will mention the specific studies on which they are basing their conclusions. It turns out that these summary statistics (pejoratively we could call them “factoids”) are often based on legitimate scholarship using what is known as “case-control” research designs.
In my continuing effort to get a handle on the research on guns, I have already had to wade through some very complex econometric modeling on the relationship between guns and crime, some of which I could not understand despite my decent statistical training in sociology. These studies are typically “ecological” in that they compare rates of gun ownership in a particular geographic area (county or state, typically) to rates of crime in that same area. They cannot directly any particular guns to any particular crimes.
A more direct assessment of the relationship between guns and negative outcomes (crime or injury) is to look at the individual level and ask does an individual possessing a gun have a greater or lesser likelihood of a negative outcome than an individual who does not? Public health researchers (typically medical doctors or epidemiologists) frequently employ case-control designs to address questions like this. As these methods are uncommon in sociology, I have had to take some time to get up to speed on them.
EXAMPLES OF INVOKING STATISTICS ABOUT FIREARMS FROM CASE-CONTROL STUDIES
Here are some of the conclusions people have drawn based on case-control studies.
(A) The New York Times, in the editorial on “Dangerous Gun Myths” I considered in an earlier post, cites two such studies. One is work by Campbell and 17 co-authors: “In domestic violence situations, the risk of homicide for women increased eightfold when the abuser had access to firearms, according to a study published in The American Journal of Public Health in 2003. Further, there was ‘no clear evidence’ that victims’ access to a gun reduced their risk of being killed.”
- Jacquelyn C. Campbell, Daniel Webster, Jane Koziol-McLain, Carolyn Block, Doris Campbell, Mary Ann Curry, Faye Gary, Nancy Glass, Judith McFarlane, Carolyn Sachs, Phyllis Sharps, Yvonne Ulrich, Susan A. Wilt, Jennifer Manganello, Xiao Xu, Janet Schollenberger, Victoria Frye, Kathryn Laughon. 2003. “Risk Factors for Femicide in Abusive Relationships: Results from a Multisite Case Control Study.” American Journal of Public Health 93/7 (July):1089-1097.
The Times continues: “Another 2003 study, by Douglas Wiebe of the University of Pennsylvania, found that females living with a gun in the home were 2.7 times more likely to be murdered than females with no gun at home.”
- Douglas J. Wiebe. 2003. “Homicide and Suicide Risks Associated with Firearms in the Home: A National Case-Control Study.” Annals of Emergency Medicine 41:771-82.
(B) One of the best known studies, which still attracts attention today, even after all these years, is Arthur Kellermann’s work on gun ownership and homicide in the home. (I looked at some of Kellermann’s work in my last post, but noted the difficulty of making causal inferences based on this other study.) Kellermann’s case-control study is promoted on the website of the Brady Campaign to Prevent Gun Violence. The Brady Campaign’s summary is frequently cited: “It was found that people who keep guns in homes are almost 3 times more likely to be murdered.”
- Arthur L. Kellermann, Frederick P. Rivara, Norman B. Rushforth, Joyce G. Banton, Donald T. Reay, Jerry T. Francisco, Ana B. Locci, Janice Prodzinski, Bela B. Hackman, and Grant Somes, “Gun Ownership as a Risk Factor for Homicide in the Home,” New England Journal of Medicine, 1993; 329:1084-1091.
Some dismiss studies like Kellermann’s out of hand by arguing that of course you are going to see more gun deaths in households where there are guns. That is like saying you are going to find more auto deaths in households that own cars, and more drowning in homes that have swimming pools. But this criticism does not consider the logic of causal inference in the specific methodology employed in Kellermann’s and these other studies: the case-control method.
THE CASE-CONTROL METHOD
Medical, epidemiological, and public health researchers attempt to identify the causal factors that explain outcomes of interest. For example, we want to know whether certain drugs (e.g., Interferon and Ribaviron) have beneficial outcomes for those who take them (e.g., the remission of Hepatitis C). To test such a causal relationship, researchers will run an experiment in which individuals are randomly assigned “treatment” and “control” groups, so that the only systematic difference between the two groups is the causal factor of interest. If differences in outcomes are observed after the experiment, the researchers can conclude that the treatment caused the outcome.
The true experiment is the gold standard for making causal inferences because it is best able to isolate the causal factor of interest. Unfortunately, it is not always possible to use a true experimental design. When randomized assignment to treatment and control groups is not possible, researchers use various quasi-experimental designs, including case-control designs. (Public health scholars call these studies “observational” to distinguish them from “experimental” studies, but because the logic of causal inference is based on the experimental design, I prefer to call them “quasi-experimental.”)
For example, if you want to know whether smoking causes carcinoma of the lung, you cannot randomly assign some people to smoke and others not to smoke. But you can use a case-control design to compare those who have carcinoma of the lung to those who do not and to see whether smoking is more prevalent (a “risk factor”) among those with carcinoma compared to those without. In fact, this is exactly what Sir Richard Doll did in his pioneering 1950 article on “Smoking and Carcinoma of the Lung” published in the British Medical Journal. Doll compared 1,732 individuals suffering from carcinoma of the lung, stomach or large bowel (the “case sample”) to 743 general medical and surgical patients (“control sample”). He found that those with carcinoma of the lung were more likely to be smokers, and more likely to be heavy smokers, than those without.
The same logic applies to guns as a health risk: You cannot take a group of research subjects and randomly give half of them guns and make sure half do not have guns then see what the incidence of homicide is for the two groups over time. (Also, as I will discuss below, you cannot easily do a prospective cohort study, either.) So, scholars approaching the issue of gun violence from a public health perspective turn to case-control designs.
A key to doing a successful case-control design is to find an appropriate control sample. The control sample is often called a “matched sample” because the control sample should be matched to the case sample on as many other characteristics as possible, except for the risk factor or exposure of interest. That is, the case and control samples should be from the same population so that only the risk factor of interest varies between the two groups.
Of course, in the case of gun violence, there is not simply one risk factor. In order, therefore, to isolate the effect of each particular risk factor, multivariate statistics are used to statistically control for differences between the case and control samples. At least for those factors that are included in the statistical models, the researcher can say that the factor of interest increases or decreases the likelihood of the outcome, ceteris paribus (all other things being equal). Of course, it is never possible to measure every other possible cause of an outcome, so all other things are never truly equal, but the matched sample along with the use of statistical controls are meant to approximate the treatment and control group design of the randomized controlled experiment.
CASE-CONTROL STUDIES OF GUN VIOLENCE
All case-control studies proceed in roughly the same steps. Here I identify the steps taken in each of the four studies mentioned above here. Note that in some cases I have reduced the methodological complexity for ease of presentation. E.g., Kellerman, et al. originally identified 420 cases, but that was reduced to 388 for various reasons. See the original articles for full details on all of these points.
(1) Identifying the Study Population. In many cases, the population of interest is all residents of a country, state, county, or city. In the four studies of interest here, the study populations are as follows:
|Kellerman, et al.||Residents of Memphis & Shelby Co. (TN), Seattle & King Co. (WA), Cleveland & Cuyahoga Co. (OH)|
|Wiebe||Americans 18 years of age or older|
|Campbell, et al.||Women aged 18 to 50 in abusive relationships in 11 cities: Los Angeles; Portland, OR; Seattle, WA; Chicago, IL; Wichita, KS; Houston, TX; Tampa/St. Petersburg, FL; New York City; Baltimore, MD; Kansas City, MO; Kansas City, KS|
(2) Selection of the Case Sample. Case control studies work backwards from outcome to risk exposure, so all of the four studies above begin by identifying a case sample.
|Authors||Case Sample||# of Cases|
|Kellerman, et al.||Firearm homicide victims in private homes between 8/87 and 8/92||388|
|Wiebe||Homicide victims over 17 years old who had a gun in their home in the 1993 National Mortality Followback Survey (NMFS)||1,720|
|Campbell, et al.||Women aged 18 to 50 with history of previous abuse killed by current or former intimate partners between 1994 and 2000||220|
(3) Selection of the Control Sample. The control cases should come from the same population as the case sample, so in their selection they should be matched on certain demographic criteria that are separate from the risk factors of interest.
|Authors||Control Sample||# of Cases|
|Kellerman, et al.||Control subjects were matched to case subjects by sex, race, age group, and neighborhood of residence.||388|
|Wiebe||Control subjects were drawn from the 1994 National Health Interview Survey (NHIS) and matched by sex, race, and age group.||8,084|
|Campbell, et al.||Stratified random sample of women aged 18-50 in same 11 cities who reported having been physically assaulted or threated with a weapon by a current or former intimate partner during past 2 years.||343|
(4) Measurement of Risk Exposure. Information must be gathered about exposure to risks by individuals in both the case and control samples. This information can be collected directly from the individual (especially for the control sample) or from various proxies (families of the deceased, police records, hospital records).
|Authors||Method of Measuring of Risk Exposure|
|Kellerman, et al.||Control subjects and proxies for case subjects were interviewed and asked an identical set of questions to identify risk factors such as gun ownership, drug and alcohol use, previous violence in the home and so on. Proxies for case subjects were identified from police records, newspaper accounts, obituaries, and funeral homes.|
|Wiebe||For NMFS, next of kind were surveyed about deceased, and for NHIS respondents answered questions about themselves. Both groups were asked about “firearms kept in or around [the decedent’s/your] home? Include those kept in a garage, outdoor storage area, truck or car.”|
|Campbell, et al.||Proxies and control subjects were interviewed about demographic and relationship characteristics (type, frequency, severity of violence, etc.), alcohol and drug use, weapon availability.|
(5) Estimate of Likelihood of Outcome Associated with Risk Exposure. Now the researcher is in a position to estimate how much more likely individuals exposed to a particular risk are to experience the outcome of interest than those who were not exposed to the risk. These estimates are typically expressed in the form of crude/raw odds ratios (from univariate analyses) or adjusted odds ratios (from multivariate analyses). Simply put, “a person exposed to X, is Y times more likely to experience Z,” where X is the risk factor, Z is the outcome of interest, and Y is the odds ratio.
|Authors||Likelihood of Outcome Associated with Risk Exposure|
|Kellerman, et al.||Adjusted odds ratios for homicide from multivariate logistic regression (Table 4, p. 1089):
|Wiebe||Adjusted odds ratios for homicide from multivariate logistic regression (Table 3, p. 776 for gun at home; pp. 775-6 for other statistics, though regression tables not provided):
|Campbell, et al.||Adjusted odds ratios for intimate partner femicide among women in physically abusive relationships from multivariate logistic regression (Table 3, Model 6, pp. 1095-96):
Note: NS = not significant. Odds ratios < 1.0 indicate reduced risk of outcome
(6) Notable Conclusions. Having established the likelihood that a particular risk factor is associated with an outcome of interest, researchers are in a position to draw conclusions.
|Kellerman, et al.||“In the light of these observations and our present findings, people should be strongly discouraged from keeping guns in their homes” (p. 1090)|
|Wiebe||“Nevertheless, it would be unwarranted to infer that such a limited body of research conclusively links gun availability to gun-related mortality” (p. 777).“The public health would benefit greatly if persons were less exposed to home-kept firearms” (p. 780)|
|Campbell, et al.||“When women are identified as abused in medical settings, it is important to assess perpetrators’ access to guns and to warn women of the risk guns present. This is especially true in the case of women who have been threatened with a gun or another weapon and in conditions of estrangement. Under federal law, individuals who have been convicted of domestic violence or who are subject to a restraining order are barred from owning firearms. Judges issuing orders of protection in cases of intimate partner violence should consider the heightened risk of lethal violence associated with abusers’ access to firearms” (p. 1094).|
THE DEVIL IS IN THE DETAILS
There is a beauty in the simplicity of a statistic like “people who keep guns in homes are almost 3 times more likely to be murdered” (Brady Campaign) and “females living with a gun in the home were 2.7 times more likely to be murdered than females with no gun at home” (New York Times). Digging into the details of these studies, however, reveals these assertions to be highly problematic. These simple statistics are not to be taken at face value, and certainly not used to formulate public policies.
Problems with Kellermann, et al., “Gun Ownership as a Risk Factor for Homicide in the Home”
Just as people criticized Doll’s early studies linking smoking and lung cancer based on his case-control design, so too do people criticize case-control designs of guns and homicide. One needs to be careful because what we are talking about here are not causes but risk factors. We know guns do not cause homicide. The presence of a gun increases the risk that someone will be a victim of a homicide. But the mechanism by which that risk is increased is not at all clear from Kellermann’s study. One needs to look very closely at the details of his study to determine this.
I previously criticized Kellerman’s research because he included many cases of individuals who were shot in someone else’s home, and also did not specify whether the victim’s own gun was used against them. Similar problems plague this study. In the first place, of the original 420 homicides committed in the homes of victims, only 209 (49.8%) of them were by firearm. 26.4% were by cutting instrument, 11.7% by blunt instrument, 6.4% by strangulation or suffocation, and 5.7% by other means. Of those 209 cases, Kellermann does not report how many were shot with their own firearm as opposed to a firearm carried into the home. Sociologist Gary Kleck, however, has used Kellermann’s data and some additional assumptions to try to determine what percentage of homicide victims were killed in their own home using a gun “kept in the home where the shooting occurred.” He concludes that as few as 9.7% and as many as 14.2% of gun homicides were committed in the victims’ home with a gun kept there (Kleck, “Can Owning a Gun Really Triple the Owner’s Chances of Being Murdered? The Anatomy of an Implausible Causal Mechanism,” Homicide Studies 5 , pp. 69-70). So, 209 gun homicides x 0.142 (proportion own gun, own home) = 30 cases. This leads to two conclusions:
- Of the total number of homicides committed in the homes of victims, only 7.1 percent (30 of 420) were committed using a gun kept in that home. 92.9 percent were committed using a gun brought into the home or another mechanism of death.
- Of the total number of homicides committed in the study area, only 1.6 percent (30 of 1,860) were gun homicides committed in the victim’s home using a gun kept there. 98.4 percent we either outside the home, were not gun homicides, or did not use the victim’s gun. People in the case sample are 62 times more likely to be killed under these other circumstance than to be killed in their own home with a gun kept there.
In his effort to prove that guns are dangerous, Kellermann clearly overdraws his conclusions. As with his other study that I examined previously, he might have been better off focusing on the relative infrequency of justifiable homicides to argue that there is not a huge protective benefit from owning a firearm, rather than characterizing it as a risk factor for homicide. But that claim is much weaker and doesn’t make for good anti-gun advocacy group talking points.
Problems with Wiebe’s “Homicide and Suicide Risks Associated with Firearms in the Home: A National Case-Control Study.”
No research is perfect and I approached Wiebe’s study from a sympathetic perspective, trying to understand what he was attempting to do within the constraints of his data. I recognize, for example, that he was very clever to use the National Mortality Followback Survey (NMFS) for his case sample and the National Health Interview Survey (NHIS) for his control sample. Unlike Kellerman’s in-depth study of cities/counties, Wiebe’s study is based on a nationally-representative sample of homicide victims and a nationally-representative control sample. His findings, therefore, are more generalizable to the American population than Kellerman’s – if his findings are meaningful.
Unfortunately, there is a tradeoff here because surveys of nationally-representative samples of individuals tend to “tell us a little bit about a lot of people” (as my undergraduate research methods teacher Kim Voss would say). What get sacrificed are details, and as with Kellerman’s work, those details are crucial. Most notably, because Wiebe is conducting a secondary analysis of data he did not originally collect, he is dependent upon those who collected the data for his variables. This introduces problems with both his dependent variable (homicide) and his independent variables (risk factors such as keeping a firearm in the home).
With respect to the dependent variable, the homicides included in this study were not all by firearm, not all in the individual’s own home, and of course (therefore) not all by the firearm that was kept in the individual’s home.
- 74.4 percent of the victims in this study died from gunshot trauma. Although this is more than the 49.8 percent observed by Kellerman, it is notable that a quarter of the cases in Wiebe’s sample were not homicide victims. Other than losing cases and therefore having less statistical power, it is not clear why Wiebe did not select only those cases in which the cause of death was gunshot trauma.
- Furthermore, only 32.9 percent of the homicide victims in this study had their location classified as “home or private area around the home.” BUT, as Wiebe himself recognizes, “Where the incident actually took place is unclear for victims who were at someone’s home when injured (the category is ‘home or private area around the home,’ which does not distinguish the victim’s home from someone else’s)” (p. 773).
- As these other two points already make clear, and as Wiebe acknowledges, his “data do not indicate whether a fatal gunshot wound was inflicted with a gun that had been kept in the victim’s home” (p. 773).
The homicides that Wiebe is explaining in this study, therefore, are all homicides regardless of the means or location. This is significant (and problematic) because presumably Wiebe wants to understand keeping a gun in one’s home as a direct, immediate risk factor for being a homicide victim. But how, exactly, does keeping a firearm in one’s home make that person more at risk of dying by strangulation, cutting instrument, etc.? And why does keeping a gun in one’s own home make that person more at risk of being killed (by whatever means) in someone else’s home? And how does keeping a gun in one’s own home make that person more at risk of being killed by a gun brought into the home?
If we want to understand keeping a gun in one’s home as a risk factor for homicide, we would want our sample cases to be individuals who were killed at home by guns kept in their home. Otherwise, it is simply not meaningful as a risk factor, at least as would be commonly understood by people trying to assess the risk of some decision (e.g., keeping a gun in their home). In fact, if keeping a gun in one’s own home is associated with an increased likelihood of being killed in general, then keeping the gun may not be a cause of death but an effect of someone’s (apparently rational) fear of being killed.
A person could have an elevated fear of being killed for a number of reasons associated with their social situations, behaviors, or lifestyle: a history of violence or abuse, illicit drug and alcohol use, gang membership, drug dealing. These are certainly risk factors for homicide independent of gun ownership, but they can also be risk factors for gun ownership itself. Keeping a gun in the home, then, might just be a proxy for other risk factors, not a risk factor in itself. Failing to consider these other risk factors creates the possibility (even the probability) of erroneously attributing their effect to guns.
I suspect that this is exactly what happened in Wiebe’s study. Due to his dependence on existing national datasets, Wiebe is unable to control for several very important risk factors for homicide because they were not included in the data he is using. In multivariate models, Wiebe controls for: living arrangement (alone or not alone), marital status, education, annual family income, military veteran status, geographic region, and population of the locality where the subject lived (p. 774). Weibe himself acknowledges that he has no measures of risk factors that are well-known to be associated with homicide: “histories of violence, illicit drug and alcohol use, time spent (exposed) at home, and lifestyle factors like gang membership and drug dealing” (p. 780).
Not controlling for these other risk factors, not distinguishing between the instrument of death, and not knowing the location of death all makes it very difficult to accept the veracity of the New York Times’ claim (based on Wiebe) “that females living with a gun in the home were 2.7 times more likely to be murdered than females with no gun at home.” Much less Wiebe’s overdrawn conclusion that “public health would benefit greatly if persons were less exposed to home-kept firearms” (p. 780).
Problems with Campbell, et al.,“Risk Factors for Femicide in Abusive Relationships: Results from a Multisite Case Control Study.”
Perhaps because it is focused on a more clearly defined population of interest – women aged 18 to 50 in abusive relationships – and had more specifically defined risk factors (e.g., characteristics of the relationships between abusers and victims), the results of this study appear less problematic than those of Kellermann, et al. and Wiebe. The authors are also much more modest in their conclusions.
It is interesting that the New York Times says, “In domestic violence situations, the risk of homicide for women increased eightfold when the abuser had access to firearms.” This is true of the 3rd of Campbell, et al.’s 7 models, but in the more complete models 4 and 5, the risk of homicide is reduced to fivefold. Still a significant risk, but more in line with the graphic shown above than the Times’ editorial.
A simplistic reading of this study can also obscure some important details. For example, only 38.2 percent of the victims in this study were killed by firearms. Women were nearly twice as likely (1.88x) to be killed by something other than a firearm. This despite the fact that 65 percent of the perps had access to firearms (defined as personal ownership or living in a household with a firearm). This is a good reminder of the difference between a “risk factor” and a “cause.” And, again, the authors’ conclusions are appropriate to what they can actually document.
The New York Times also pointed out (quoting Campbell, et al.) that “there was ‘no clear evidence’ that victims’ access to a gun reduced their risk of being killed.” This is descriptively true, but not a keen analytical point since only 5 percent of victims of femicide (n=10 individuals) and control subjects (n=17) met the researcher’s criteria: “woman’s sole access to a firearm on the basis of her living apart from her partner and reporting having a gun in the home.”
I have more questions about this study at this point than criticisms:
- For the 27 percent of murderers in this study who have access to guns but do not use them, why? What does it tell us about the role of guns in abusive relationships that so many kill by other means? Also, this percentage is very close to the 23% of abusers of the women in the control sample. How do the 23% of abusers who have guns but don’t kill compare to the 27% of murderers who have guns by kill by other means?
- An acknowledged limitation of this study is that it encompassed only 11 cities, and excluded non-urban areas. It would be interesting to know whether abuse and femicide is more prevalent in urban areas. If so, would including non-urban areas – where gun ownership is more prevalent – even further decrease the risk of access to guns for women in abusive relationships?
Case-control designs are employed both when randomized controlled trials are not possible (e.g., assigning some people to have guns and others not), and when the outcome of interest is rare (e.g., homicide). If we simply took a random sample of the American population in any given year, the probability that any given individual would be a victim of homicide would be .0032%. (Which is why homicide rates are expressed as per 100,000 population and not as simple percentages.) With a homicide rate of 3.2 per 100,000 in 2011, we would need to sample millions of people in a standard prospective cohort study to get even a few dozen homicide victims in our sample.
It is also worth noting, for those of us who think of the risk of homicide not simply as an academic question but as a personal concern, that the homicide rate of 3.2 per 100,000 includes all of the people who are at special risk of being a victim of homicide. This includes, as Campbell, et al. shows, women who are in abusive relationships. It also includes criminals and drug users and heavy drinkers. The people who do not share these characteristics – people like myself – have an even smaller risk of being a victim of homicide than 3.2 per 100,000.
As Canadian journalist Daniel Gardner writes, in his book The Science of Fear, “if you are not a drug dealer or the friend of a drug dealer, and you don’t hang out in places patronized by drug dealers and their friends, your chance of being murdered with a handgun shrinks almost to invisibility” (p. 12). Or, as gun trainer John Farnam and certainly many others have said, “Don’t go stupid places or do stupid things with stupid people.” This lowers your risk of homicide substantially, whether you have a gun or not.
Of course, the very low likelihood of being a victim of homicide not only argues against excessive concerns about the availability of guns to anyone other than criminals, it can also be an argument against the need for private citizens to keep guns in their home for self-defense or to arm themselves in public by carrying concealed firearms. Why some people do this despite the very low odds that they will need to shoot another person in self-defense will be the subject of a later entry on the issue of risk and risk assessment.