A- A+
Alt. Display

# Evaluating the Effect of Homicide Prevention Strategies in São Paulo, Brazil: A Synthetic Control Approach

#### Danilo Freire

##### King's College London, GB

Danilo Freire is a PhD candidate in the Department of Political Economy at King’s College London. He received a master’s degree in international relations from the Graduate Institute of International and Development Studies, Geneva, and a master’s in political science from the University of São Paulo. His main research interests are urban crime in Latin America, prison gangs, political violence, Bayesian data analysis, and survey experiments.

## Abstract

Although Brazil remains severely affected by civil violence, the state of São Paulo has made significant inroads into fighting criminality. In the last decade, São Paulo has witnessed a 70 percent decline in homicide rates, a result that policy-makers attribute to a series of crime-reducing measures implemented by the state government. While recent academic studies seem to confirm this downward trend, no estimation of the total impact of state policies on homicide rates currently exists. The present article fills this gap by employing the synthetic control method to compare these measures against an artificial São Paulo. The results indicate a large drop in homicide rates in actual São Paulo when contrasted with the synthetic counterfactual, with about 20,000 lives saved during the period. The theoretical usefulness of the synthetic control method for public policy analysis, the role of the Primeiro Comando da Capital as a moderating variable, and the practical implications of the security measures taken by the São Paulo State government are also discussed.

Resumo

How to Cite: Freire, D. (2018). Evaluating the Effect of Homicide Prevention Strategies in São Paulo, Brazil: A Synthetic Control Approach. Latin American Research Review, 53(2), 231–249. DOI: http://doi.org/10.25222/larr.334
Published on 13 Jun 2018
Accepted on 17 Nov 2016            Submitted on 09 Feb 2016

## Introduction

Brazil has long been ravaged by an undeclared civil war. According to the Citizen Council on Public Security and Criminal Justice, a Mexican think tank, nineteen of the fifty most violent cities in the world are located in Brazil (Consejo Ciudadano para la Seguridad Pública y Justicia Penal 2014).1 The 2014 Violence Map survey shows that 56,337 people were murdered in Brazil in 2012 alone, the highest incidence rates of intentional homicides on the planet (Waiselfisz 2014; United Nations Office on Drugs and Crime 2013). Paradoxically, the sharp rise in lethal violence has occurred during Brazil’s longest period of political openness (Ahnen 2003; Pinheiro 2000, 2001). Murder rates have almost doubled over three decades of democracy, jumping from 15 homicides per 100,000 people in 1985 to roughly 29 per 100,000 in 2012 (Waiselfisz 2014).2

São Paulo has traditionally occupied a key position in Brazil’s violence statistics. It is the country’s richest and most densely populated state, and in the 1990s its homicide rate was roughly 50 percent higher than the national average (Barata and Ribeiro 2000, 120). Some areas of the namesake capital city had even worse numbers. Between 1996 and 1999, the ramshackle districts of Jardim São Luiz and Jardim Ângela had respectively 103 and 116 violent deaths per 100,000 residents (Cardia, Adorno, and Poleto 2003, 8), figures that placed them among the deadliest neighborhoods on the globe (World Health Organization 2015).

Nevertheless, the state of São Paulo has experienced a drastic reduction in homicides during recent years (Camargo 2007). The decline is so remarkable that some authors have called it “the great homicide drop” (Goertzel and Kahn 2009). The city of São Paulo, which is currently home to about 11 million inhabitants, provides a telling example. Over a span of only seven years (2000–2007), the number of annual violent deaths in the capital fell from 5,979 to 1,311, a 78 percent decrease.3 Significantly, São Paulo became the safest capital in Brazil (Waiselfisz 2011).

São Paulo’s success should be attributed to local factors. From 1999 onward, the state government created or expanded a number of policies that have arguably contributed to the decrease in criminality. In a move coherent with the basic tenets of the economics of crime (e.g., Becker 1968; Cornish and Clarke 2014), the administration increased the certainty and the intensity of punishment to discourage potential offenders. Among other measures, the government implemented strict gun control policies (Goertzel and Kahn 2009), raised incarceration rates (Salla 2007), and imposed harsher sentences on those convicted of a crime (Carvalho and Freire 2005).

But whereas several authors acknowledge the effectiveness of these policies, few quantitative studies have gone beyond statistical correlations to justify their arguments. In the case of São Paulo, a major difficulty is separating the state’s particular time trend to that of Brazil. Ideally, one should compare São Paulo to a control case that shares the same characteristics of the existing state, except that it has not been subjected to the specific set of policies implemented by the São Paulo government. This thought exercise, which emulates the logic of a controlled experiment (Angrist and Pischke 2008; Imbens and Rubin 2015; Holland 1986; Morgan and Winship 2014), would allow practitioners to untangle the effects of homicide reduction programs from other potential confounders.

In this article, I employ the synthetic control method (henceforth SCM) to approximate this experimental ideal and measure the total causal effect of post-1999 public policies on São Paulo homicide rates. The method consists of creating an artificial counterfactual to estimate the impact of a given intervention on a unit of interest. SCM has gained widespread acceptance in many fields, having been successfully applied in political science (Abadie, Diamond, and Hainmueller 2014; Montalvo 2011), economics (Billmeier and Nannicini 2013; Coffman and Noy 2012; Jinjarak, Noy, and Zheng 2013), education studies (Hinrichs 2012), and public health science (Heim and Lurie 2014). However, SCM has rarely, if ever, been used to evaluate homicide prevention strategies in São Paulo, despite being a useful tool for this particular type of question. SCM was specifically designed for situations where there is only one treated unit of interest, no readily available counterfactual, and no certainty as to whether the treated and the control units follow parallel trends after the intervention (Abadie and Gardeazabal 2003; Abadie, Diamond, and Hainmueller 2010, 2014). Moreover, SCM also has some of the desirable properties of other popular causal inference tools such as differences-in-differences (Angrist and Pischke 2008; Bertrand, Duflo, and Mullainathan 2004) and matching estimators (Dehejia and Wahba 2002; Ho et al. 2007; Rubin 1973; Stuart 2010).

I find that from 1999 to 2009, about 20,000 lives were saved in São Paulo. When compared to a synthetic counterfactual, São Paulo’s actual homicide rates were less than 50 percent of what would be expected in the absence of policy implementation (15 versus 32 homicides per 100,000 people). Additional tests confirm the robustness of the results and indicate a 96.3 percent chance of a causal effect in the intervention period.

## Theoretical Background

### Deterrence, information, and the drop in homicides

A myriad of explanations have been proposed for the fall in homicide rates in São Paulo. Some authors have stressed the importance of long-term factors on local levels of violence. Mello and Schneider (2010) claim that the shrinking proportion of males in the fifteen to twenty-five age bracket has led to fewer violent deaths at both state and city levels. Hughes (2004) argues that São Paulo’s spatial segregation patterns have had a lasting impact on murder rates. Barata and Ribeiro (2000), in turn, posits that macroeconomic conditions, mainly inequality indicators, are positively correlated with violent crime in São Paulo.

Structural variables have likely been important in reducing violence, but the role of public policies should not be underestimated. The Brazilian Social Democracy Party (Partido da Social Democracia Brasileira, PSDB), which has ruled São Paulo since 1995, has repeatedly asserted its commitment to reducing urban crime throughout the state (Bueno 2014). In 1998, former governor Mário Covas, then running for reelection, set the ambitious goal of “slashing criminality rates in half” during his second term in office (Santos 2008). This commitment was then followed by his vice-governor and successor, Geraldo Alckmin, who has expanded those measures and taken a notoriously tough stance on crime (Feltran 2012a).

Methods of crime prevention have received considerable attention from the authorities. Firstly, the São Paulo government significantly increased incarceration rates in the past decade (Salla 2007). The state currently holds around 200,000 convicts in prison (35 percent of Brazil’s inmate population) and adds another 15,000 inmates to the official statistics every year (Brasil de Fato 2013). Furthermore, prisoners have also become subject to harsher legal punishments. The São Paulo administration has also been making large use of the Regime Disciplinar Diferenciado (Special Disciplinary Regime), which provides for up to 360 days of solitary confinement for disobeying the law (Carvalho and Freire 2005).

Secondly, the state government has successfully enforced a ban on gun possession in São Paulo. Studies show that this policy has been effective in reducing homicides resulting from both drug-related crimes and domestic disputes (Goertzel and Kahn 2009; Kahn and Zanetic 2005). Furthermore, the impact of the Brazil’s 2003 National Disarmament Act was particularly pronounced in São Paulo. Cerqueira and Mello (2013) argue that between 2005 and 2007 the enforcement of the anti-firearm legislation was responsible for saving between 2,000 to 2,750 lives in cities with more than half a million inhabitants in the state of São Paulo.

This set of policies is largely in line with the rational choice theory of crime (e.g., Becker 1968; Ehrlich 1975; Levitt 1996, 1997; Paternoster 2010). The rational choice school posits that criminals are motivated by utilitarian cost-benefit analysis. Individuals calculate what the possible trade-offs are between the benefit of the committing a crime and the risk of being punished for it. Criminal offenders, therefore, only differ from noncriminals in their choices (Nagin 2007). To reduce criminality, policy-makers have to ensure that the costs of committing a crime outweigh the eventual utility an individual derives from it.

Deterrence measures have been complemented by investments in police intelligence. In 1999, the state administration created a new system for crime prevention, Infocrim (Risso 2014, 3) The system gathers geo-coded information on homicides and maps the most important “hot spots” of criminal activity in the state. The government has also developed a new photo database, Fotocrim, to speed up the process of facial recognition of criminals (Mello and Schneider 2010, 3).

More information improves the effectiveness of police strategies via two mechanisms. First, police forces can be quickly moved to where they are most needed. This reinforces the role of deterrence as it increases the likelihood of punishment for criminals. Second, the system also makes clear what regions are making progress in reducing crime. This allows police chiefs to monitor local personnel and take measures to improve performance if required.4

Recent evidence shows that the intelligence system has effectively lowered the crime statistics in São Paulo. Using a spatial differences-in-differences estimator, Cabral (2016) argues that Infocrim has had a large negative impact on homicide rates in the municipalities where it was implemented. The author also notes that the effect remains important even after accounting for possible displacement effects. As expected, some criminals did take their activities elsewhere after the creation of Infocrim, but this movement has not offset the benefits of the system.

How well have these policies performed over time? The results suggest a favorable outlook. Compared to other Brazilian states, São Paulo is an outlier when it comes to homicide rates. Despite the fact that crimes against property have declined little over the last decades,5 the number of violent deaths per 100,000 inhabitants shows a steep downward trend. Figure 1 presents the evolution of homicide rates in São Paulo in comparison with the Brazilian average.

Figure 1

Homicide rates per 100,000 population: São Paulo and Brazil (excluding the state of São Paulo). Data from Instituto de Pesquisa Econômica Aplicada, http://www.ipeadata.gov.br/.

The trends are even more striking if we consider that deterrence policies are still controversial in the literature. Barbarino and Mastrobuoni (2014), Buonanno and Raphael (2013), Levitt (1996, 2004) and Owens (2009) claim that incapacitation measures effectively reduce crime, but Eck and Maguire (2006) and Beattie and Mole (2007) suggest that increases in police forces and incarceration rates in the United States and in Canada did not lead to expected outcomes.

There is good evidence that incapacitation measures have worked well in São Paulo during the last decade. Gun-related homicides declined about 74 percent from 2001 to 2008 (Peres et al. 2011) while São Paulo experienced an increase of 770 percent in the arrests of repeated murderers (Manso 2012, 36). Although there are studies that indicate possible “hardening effects” of imprisonment, that is, longer sentences may increase an individual’s tendency to commit further crimes (e.g., Chen and Shapiro 2007; Glaeser, Sacerdote, and Scheinkman 1996; Western, Kling, and Weiman 2001), the São Paulo case appears to suggest otherwise. Moreover, violent deaths have decreased in all population strata but especially among males (–74.5%), fifteen-to twenty-four-year-old men (–78.0 percent) and those who live in extreme poverty (–79.3 percent), groups that are generally associated with criminality (Peres et al. 2011).

Nevertheless, it is difficult to know which of the policies have contributed more to this large homicide reduction. Not only do we lack disaggregated data to test preliminary hypotheses, but there may be large interaction effects among different public security measures. Therefore, at the moment it is not possible to disentangle micro-level causes from macro effects. But the aggregated impact of the anticrime policies can be correctly identified if there is no other variable in the causal path leading from the policies mentioned above to our dependent variable (state homicide rates). I argue below that this type of estimation is feasible for the São Paulo case. To back this claim, I suggest that a competing explanation for the homicide drop in São Paulo—the rise of the Primeiro Comando da Capital (PCC)—interferes only with the direct effect of the policies on crime but not with their effect. In this sense, the synthetic control method provides a plausible identification strategy for my question of interest.

### Alternative explanation: The emergence of the PCC

A recent hypothesis attributes the decrease in violent deaths in São Paulo to the Primeiro Comando da Capital (PCC) (Biondi 2010; Dias 2009a, 2011; Feltran 2010, 2012a; Willis 2015). The PCC is a prison gang that emerged in the early 1990s as a response to the demands of a growing prison population. The PCC provides personal security and financial assistance to its members and affiliates. The gang’s internal statute clearly declares that “those who are in liberty [must contribute] to the brothers inside prisons [PCC members] through lawyers, money, help to family members and prison outbreak operations” (Folha de São Paulo, 2001).

A group of scholars argues that the PCC significantly contributed to the reduction in violence mainly through the São Paulo prison system. At least since the mid-2000s, these authors argue that the PCC has been able to emerge as an undisputed mediator and solve conflicts between inmates. Dias (2009b, 83) writes that “when unable to constitute a universal source of regulation, the official law leaves gaps which are filled by informal instances—such as the Primeiro Comando da Capital (PCC), in the prisons of São Paulo.” The gang has implemented informal courts that resemble state institutions, and those meetings have progressively replaced other forms of popular justice such as lynchings or the hiring of target killers (Feltran 2012b, 3). Moreover, the Comando has developed a series of assertive ways to terrorize inmates. Since the PCC’s threats are credible, the group is able to impose discipline within the São Paulo prison system (Biondi 2010; Dias 2009a).

Paradoxically, the PCC might have also helped to reduce crime in São Paulo by collaborating with street-level police. The Brazilian state does not hold a perfect monopoly of force in many areas of the country (Feltran 2012a; Hughes 2004; Pinheiro 2000), thus access to local knowledge may prove vital for the success of a given operation. In this regard, the PCC and the state may collude if the situation is beneficial to both, and in this way there is an informal—but potentially unstable—“killing consensus” in the state (Willis 2014, 2015).

There has been a vigorous debate over whether the PCC has had a significant impact on violence rates. A few authors see the PCC as the sufficient condition behind the homicide rate decline across São Paulo State (Biondi 2010; Dias 2009a, 2011), while others take a more nuanced view of the role of the prison gang (e.g., Willis 2015). But both groups of scholars affirm that, based on their firsthand experience, the PCC is the key explanatory variable behind the drop in murders in São Paulo.

Recent econometric works, however, do not seem to confirm that argument. Marcelo Nery has found no convincing results in favor of the PCC hypothesis using geo-referenced data for São Paulo (Guimarães 2016). Biderman et al. (2014) use anonymous calls to a crime hotline as a proxy for PCC presence in São Paulo city favelas. The authors find some support for the idea that the gang reduces lethal violence in areas under its control, but PCC presence corresponds to only a minor drop in violent crime. Although the PCC impact is not negligible, the gang is not a sufficient condition for the homicide decline.

Another counterargument to the PCC thesis is that homicides also decreased in areas and groups over which the PCC does not exert control. First, descriptive statistics show that the decline in violent deaths started before the PCC’s expansion period.6 Second, the drop in crime was evenly distributed throughout the state: urban and rural areas, small and large cities alike experienced fewer murders.7 Finally, as noted above, Peres et al. (2011) point out that violent death rates decreased in all age groups and social classes in the city São Paulo. Hence, cohorts that do not correspond to typical PCC members (such as the elderly or middle-aged females) are also less affected by violence. It seems that the influence of the PCC on physical violence has been overstated. It is unlikely that the PCC, which is underfunded for its size, could have achieved such deep penetration into society and lowered the violence levels across all population groups in the whole state.8

Yet the group’s importance cannot be fully dismissed either. Data on PCC-controlled areas are likely to contain measurement errors that may bias the coefficients, thus caution is required before making strong causal claims on this discussion. Despite mounting observational evidence that the PCC may not provide a complete explanation to São Paulo’s lower crime rates, the argument could only be comprehensively tested in a counterfactual case in which the PCC is present and the state policies are not.9 Currently available data do not allow us to evaluate such scenario.

### Causal paths, moderators, and total effects

A methodological issue remains. If we are to estimate the causal effect of the public measures on the crime rates, how should we proceed? I have noted above that the specific impact of micro-level policies cannot be evaluated due to lack of data. Nonetheless, it is theoretically possible to estimate the total effect of policies on crime.

The difference between direct and total effects can be understood as follows. The direct effect captures the sensitivity of a dependent variable Y to changes in X when this relationship is not mediated by any other variables in the model. Holding all factors constant, the direct effect is a causal chain of length one (Sobel 1987, 160) and could be described simply as XY. In turn, the total effect can be defined as P (Yx = μ), that is, “the probability that response variable Y would take on the value μ when X is set to x by external intervention” (Pearl 2001, 1572). The total effect is the sum of direct and indirect (or mediated) effects.

In our case, gun control, incarceration, and police intelligence have likely had a direct effect on homicides. Combined, these variables constitute a direct aggregate policy effect. The omission of a variable measuring the impact of the PCC could bias such an effect but not interfere with the total policy effect. This point is worthy of further consideration. The total policy effect would be unbiased under the assumption that the PCC is in fact a moderator between the public policies and the homicide rates, even if the gang’s impact over the violence levels is not particularly large.

Although this argument has rarely been posited in such terms, this position is largely supported by the qualitative literature on the PCC. Fieldwork research generally traces the group’s origins and growth to the rising incarceration rates in São Paulo and the need for protection among prisoners (Dias 2011; Manso and Godoy 2014). Like other prison groups, the PCC would only mobilize resources to provide welfare and act as an arbitrator under the condition that the certainty of punishment by the state is high (Skarbek 2011; Freire 2014). Had the state not increased the costs associated with crime, the prison gang would not have expanded their reach or even have been created in the first place. Hence the impact of the PCC on street-level violent deaths—if it exists—can be safely assumed to be a moderator effect.

Whereas it would be interesting for researchers to separate these types of effects and isolate the PCC from the other causal outcomes, such estimation is not possible at the state level. However, as these measures were implemented throughout São Paulo State at roughly the same time, their combined effect is computable even though their individual direct effects are not. To do so, it is only necessary to contrast the treated unit (São Paulo) with a counterfactual without the time-assigned treatment (1999 onward) and evaluate the aggregated effect of the public policies.

This analysis can be estimated in a consistent manner with the synthetic control method. In the following sections I describe how the method creates a valid counterfactual case under a certain set of assumptions. The assumptions are: (1) the PCC is an outcome, not a cause, of the crime-targeting policies; (2) the model does not include unnecessary control variables; (3) interpolation bias is not very severe because the cases in the “donor pool” are relatively similar to the treated unit.

## Methods

The synthetic control approach provides an adequate solution for two enduring problems in the social sciences: the arbitrary selection of comparative cases and the poor estimation of causal effects when few pretreatment observations are available (Abadie and Gardeazabal 2003; Abadie, Diamond, and Hainmueller 2010). With respect to the first issue, scholars often resort to ambiguous criteria in their choice of control units. This practice ends up casting doubts over the validity of their selected counterfactual (Abadie, Diamond, and Hainmueller 2011). The synthetic method provides a reliable comparative case by adopting a purely data-driven process in order to select a counterfactual. Also, the researcher can still specify what control cases enter the donor pool. In this sense, qualitative expert knowledge can be incorporated in the estimation via the selection of cases.

Regarding the second issue, the accurate estimation of coefficients from a small number of cases, SCM employs a consistent statistical solution to problems of incorrect data extrapolation and model dependence. SCM can be understood as a combination of matching with differences-in-differences. SCM uses matching as a flexible preprocessing tool to reduce imbalance between treated and control units (Ho et al. 2007; Rubin 1973, 2006). But unlike matching, SCM deals with only one treated unit over time. Therefore, the method can also be interpreted as a semi-parametric extension to differences-in-differences estimators in which both treated and control units are not required to follow parallel trends in the whole period (Abadie 2005). By combining semi-parametric matching with differences-in-differences, SCM provides a rigorous yet versatile method to evaluate time-dependent treatment effects.

SCM starts with the assumption that one case in the sample has received a treatment.10 The treatment is defined as a time-delimited event that affects the unit of interest, such as the implementation of a new policy or the outbreak of a conflict. SCM also requires a series of control cases to estimate the models, that is, units that did not receive the treatment during the same period. These cases are often related to the treatment case in some meaningful way, and natural choices for the donor pool are provinces within the same country or states that share important characteristics. These traits can also be more specifically defined and included as quantitative variables in the estimation models.

SCM then selects a few cases from the donor pool to create a new, artificial control for the treated unit of interest. The main goal of SCM is to construct a counterfactual that resembles the treatment unit more closely than any individual control in the donor pool. Cases are combined in a way similar to a weighted average, in which controls that are more similar to the treated unit receive more weight. The weights make explicit the contribution of each separate case to the synthetic control, which also increases the transparency and reliability of the method (Abadie, Diamond, and Hainmueller 2014). The closer the synthetic control matches the original treated unit before the assignment of the period, the better the quality of the counterfactual.

The method uses an algorithm to minimize the difference between the control cases and the treated unit before the intervention. The authors adopt the mean squared prediction error (MSPE) as a measure of fit (Abadie and Gardeazabal 2003). MSPE is simply the difference between the fitted and the observed trends of the treatment case. A small value means that the two lines are highly correlated and the artificial control is a good approximation of the missing counterfactual in the post-intervention period. In our case, the counterfactual would be São Paulo from 1999 to 2010 without the crime-reducing policies.

SCM has an intuitive interpretation. Although numeric summaries and other statistics can be obtained from the model, a simple time series graph is usually enough to assess the results. The causal effect is the difference between the treated and the synthetic cohort. The larger the post-treatment gap, the stronger the treatment impact.

As with all types of observational studies, SCM can also suffer from omitted variable bias. One can never be sure whether all required confounders have been included in a given model. However, the graphical output of the SCM helps diagnose the presence of large disparities between treatment and control cases. If the trends follow similar paths during the control period, it provides some indication, albeit only informally, that omitted variable biases are not driving the output. This bias can also be mitigated with expert knowledge. Econometric studies show that the inclusion of a large number of covariates and post-treatment variables to correct for omitted variables bias can actually worsen the problem (Achen 1992, 2002; Clarke 2005, 2009; Pearl 2009). This is particularly true for matching methods. Authors have noted that “over-matching” can lead to severe statistical bias (Baser 2006; Brookhart et al. 2006; Marsh, Hutton, and Binks 2002). In this regard, the most plausible solution seems to be attention to the trends and sensible selection of control variables. As I discuss below, the covariates included in this article are some of the most robust quantitative predictors of homicides.

Furthermore, placebo tests can be run to test the robustness of the findings. For instance, researchers can include “in-time placebos,” dates under which the treatment did not occur. Results should change only in the period when the treatment starts and not at any other point in time. Moreover, scholars can also add “in-space placebos” to their models. This test consists of adding different members of the donor pools into the models to see if the estimation varies (Abadie, Diamond, and Hainmueller 2014). Finally, one can also compare the effects of the treatment of interest by creating a distribution of synthetic cohorts, where every unit (treated or not) is matched with a specific synthetic control case. The parameter of interest should still be relevant. The results of all these tests can be seen in the following sections.

## Data

I build panel data for the variables Homicide Rate, State GDP per Capita, State GDP Growth, Years of Schooling, Gini Index, Natural Logarithm of Population, and Population Living in Extreme Poverty. These variables are very common in the specialized literature and represent important social and economic factors I wish to control for.11

The unit of analysis is State-Year. I have data from all of the twenty-six states plus the capital city (Distrito Federal), ranging from 1990 to 2009. The data for years prior to 1990 are scarce and for years after 2009 have not yet been published. All data used in this article come from the same source, the Instituto de Pesquisa Econômica e Aplicada (IPEA), a government-led research group.12

My dependent variable measures the number of homicides per 100,000 inhabitants, which is the most commonly used unit of analysis for lethal violence. This variable was coded by the Brazilian Health Ministry from obituary records, therefore it is less likely than police files to suffer from intentional misrepresentation.

There are six control variables in the models. State GDP per Capita is adjusted in 2010 Brazilian Reals (at the time 1 Brazilian real bought roughly 0.5 US dollars). State GDP Growth is measured in constant 2010 Brazilian reals and varies by percentage points. Years of Schooling describes the average number of years of formal instruction at educational facilities (males and females, twenty-five years old or older). Gini Index is a measure of inequality ranging from 0 to 1, where 0 is the most equal and 1 the most unequal. Natural Logarithm of Population represents yearly projections of the state population. Since Brazil only runs a census every ten years, these projections represent the most accurate data available. I have taken the natural logarithm of this variable to account for size effects. Finally, Population Living in Extreme Poverty describes the percentage of the state population which do not meet the minimum intake of 2,000 calories per day. This is the only variable that I created specifically for this study. It was coded by simply taking the number of individuals classified as extremely poor by the IPEA and dividing this number by the state’s total population.13

## Analysis

### Main model

I construct the synthetic cohort (synthetic São Paulo) by imputing information from all of the Brazilian states plus the Federal District. The synthetic control method outputs a set of weights for states and variables such that the treatment state is approximated optimally by these weighted components. This method not only provides a quantitative way of selecting comparison cases but also gives us a much better baseline to compare with the treatment unit. Synthetic São Paulo is constructed using the six out of the twenty-seven possible cases that received non-zero weights. Table 1 shows that the states that best synthesize São Paulo are Santa Catarina (0.274), Distrito Federal (Brasília) (0.210), Espírito Santo (0.209), Rio de Janeiro (0.169), Roraima (0.137), and Pernambuco, which only accounts for 0.01 of the weights. In this regard the state selection does not appear as a complete surprise. Apart from Roraima, the other members of the federation are richer, more densely populated and better schooled than the country average, thus being indeed similar to São Paulo.

Table 1

Synthetic weights for São Paulo.

State Synthetic Control Weights Predictor Weights

Santa Catarina 0.274 Years of Schooling 0.469
Distrito Federal 0.210 State GDP per Capita 0.275
Espírito Santo 0.209 Homicide Rate 0.241
Rio de Janeiro 0.169 Population Living in Extreme Poverty 0.009
Roraima 0.137 Gini Index 0.005
Pernambuco 0.001 Natural Logarithm of State Population 0.001

Among the independent variables, only three out of six receive substantial weights. Given the data I could obtain, the predictors that receive more weight are Years of Schooling (0.469), State GDP per Capita (0.275), and past Homicide Rate (0.241). The three remaining variables are much less relevant to the model. They are the Population Living in Extreme Poverty (0.009), Gini Index (0.005), and Natural Logarithm of the Population (0.001). Table 2 compares characteristics of São Paulo and its synthetic control prior to policy implementation. We see that synthetic São Paulo has very similar coefficients to those of the treatment unit. Moreover, the synthetic control clearly outperforms the sample means in all of the three relevant predictors. The worst measure is State GDP Growth, whose mean is about 2.6 whereas the figure for São Paulo is roughly 1.3 during that period. However, this outcome does not affect the results since the variables that received zero weight were discarded from the models.

Table 2

Homicide rate predictor means before policy implementation.

Predictor São Paulo Synthetic São Paulo Sample Mean

Years of Schooling 6.089 6.110 4.963
State GDP Per Capita 23.285 23.079 11.830
Homicide Rate 32.672 32.479 21.843
Population Living in Extreme Poverty 0.054 0.082 0.185
Gini Index 0.536 0.561 0.578
Ln Population 17.335 14.838 14.867
State GDP Growth 1.330 2.585 3.528

The results show that the synthetic control method has successfully created a valid counterfactual to our case of interest. Figure 2 depicts the evolution of the dependent variable for the treatment and synthetic control cases. We can see that São Paulo and synthetic São Paulo have very close homicide rates series for the period ranging from 1990 until 1998. From 1999 onward we observe the trajectories departing sharply from each other. The increase in homicide rates shown in the graph is consistent with previous statistical evidence. It indeed confirms that São Paulo had higher than expected levels of lethal violence, which I noted in the first part of this text.

Figure 2

Trends in homicide rates: São Paulo versus synthetic São Paulo.

Despite the high levels of violence in 1999, when the new crime-reducing program was implemented, the number of homicides consistently declined until 2009. The trend is indeed monotonic and there is not a single peak in homicide rates after the policies have been put into practice. I interpret that as strong evidence in favor of the public policies.

With respect to the size of the effect, in 1998 the homicide rate in São Paulo was around 40 deaths per 100,000 inhabitants. In 2009, the last year for which data are available, the rate dropped to 15, whereas synthetic São Paulo observed above 30 deaths per 100,000. That means a gap of –20 deaths for every 100,000 people in São Paulo in 2009, as can be seen in Figure 3. I estimate that the new policies implemented in São Paulo saved roughly 20,300 lives in the period from 1999 to 2009.14 It is important to mention that the homicide rate in São Paulo continues to drop by the year, while the same is not happening in the rest of the country.

Figure 3

Homicide rates gap between São Paulo and synthetic São Paulo.

### Robustness checks

To further analyze the findings, I run five robustness tests. I first create an “in-time placebo” synthetic control. This test consists of creating a false starting date for the intervention period to check if one could observe false treatment effects in the pretreatment years (Abadie, Diamond, and Hainmueller 2014). If that were to be the case, the validity of the main results could be put into question. The result of this placebo test can be seen in Figure 4. When I run the model with 1994 as the year when there was a supposed policy change, the result shows that there is only a minor gap between both lines. In other words, the method does not indicate a definite departure of trends.

Figure 4

Placebo policy implementation in 1994: São Paulo versus synthetic São Paulo.

I also conducted a leave-one-out robustness test. In this test I drop the states composing the synthetic control one at a time. The main goal of this analysis is to evaluate whether a single control state is driving the results. This would suggest that the original synthetic control, which is composed of five states at a time, is probably not a reasonable counterfactual. The results of this analysis can be found in Figure 5. We see that the synthetic control (dashed line) is a reasonable amalgam of cases. Also, because the relative positions of treatment and controls are stable across controls, we observe that no control state is biasing the estimates.

Figure 5

Leave-one-out distribution of the synthetic control for São Paulo.

Figure 6 shows the difference in homicide rates between the treated units and their synthetic controls. Here I estimate a synthetic control case for São Paulo and for each of the other twenty-six Brazilian states. This test assesses whether there is any previously unobserved national or regional trend that explains the original results. We observe that in São Paulo the homicide rate gap increases consistently during the treatment period, whereas the lines for the other states are moving randomly. Several lines fail to show any substantial difference between the state line and that of its synthetic counterfactual case. This indicates the results for São Paulo are unlikely to be a result of broader trends.

Figure 6

Permutation test: Homicide rate gaps in São Paulo and twenty-six control states.

Figure 7 presents the same test displayed in Figure 6, but it uses a stricter threshold for the simulated synthetic controls. The graph features cases in which the mean squared prediction error, a measure of goodness-of-fit, is no higher than twice that of São Paulo. That is, only placebos that have good synthetic matches were selected for the analysis (Abadie, Diamond, and Hainmueller 2010, 503). In this group, the negative gap for the homicide rate São Paulo is by far the most relevant, providing further evidence for the original findings.

Figure 7

Permutation test: Homicide rate gaps in São Paulo and selected control states.

Last, I estimate another synthetic control using another approach. In Figure 8, I employ a Bayesian structural time-series model to verify the stability of the previous results (Brodersen et al. 2015). This inference procedure is similar to that described in the methods section of this article and it also consists of matching pretreatment values of the unit of interest, São Paulo, to other potential control states. However, in this model only the time trends of the dependent variable are matched. In a sense, this is closer to a traditional differences-in-differences approach, but without the restrictive assumption that the treated and the control cases would follow parallel trends over time (Abadie, Diamond, and Hainmueller 2010, 494).

Figure 8

Bayesian structural time series model: São Paulo and synthetic São Paulo.

The model shows that in 2009 we should have expected São Paulo to have a homicide rate equal to 32.3 deaths per 100,000, but we observe only 15.2. Thus, the actual rate in São Paulo corresponds to only 47 percent of the expected counterfactual. The method also generates an estimate for the probability of causal effect. The calculations indicate a 96.3 percent chance of a causal impact in the period. In this sense, it is unlikely that the results are a statistical fluke.

## Conclusion

As I have demonstrated, when compared to a synthetic control case, homicide rates were drastically reduced in São Paulo. Although it is not possible to estimate the treatment effect of each specific policy implemented between the 1990s and 2010 I suggest that their aggregate impact is surely not negligible. If the estimation strategy employed in this article is correct, the state of São Paulo offers an example that it is feasible to fight crime with targeted policies. This as an encouraging result, as it suggests that governments can make progress in reducing crime with the resources they already have at hand and need not rely exclusively upon structural conditions that are largely beyond their control, such as unemployment, per capita income and inequality. Robustness tests provide further evidence for my findings.

I also argue in favor of the synthetic control method as a tool to evaluate government policies. This approach offers an intuitive way to assess causality claims when there is only a single treated unit and it can be easily applied in a great number of situations. Assuming that there is a reasonable number of potential cases in the donor pool, a synthetic control can be meaningfully compared to the actual case. In this way, the technique allows the researcher to use the potential outcomes framework even in unusual conditions.

Future research can extend the present findings in a number of ways. First, it would be interesting to test whether other criminal activities have been affected by the state government policies I mentioned previously. Since property crimes are pervasive in São Paulo, scholars could evaluate the causal link (or lack thereof) between public policies and the incidence of theft or robberies. Unfortunately, several states in Brazil do not publish time-series data for property crime, so I could not use the synthetic control method for that dependent variable. As more data become available, this will create an interesting opportunity for investigation. Secondly, micro-level studies are needed to clarify the mechanisms behind São Paulo’s homicide reduction, and isolate direct from indirect effects of each individual policies. Due to the shortage of data on targeted policies, qualitative research may explain what the motivations, successes and shortcomings of São Paulo’s recent security measures were. Finally, there are still unresolved questions with regard to the PCC hypothesis, and this is a promising avenue for future academic work. New research could provide insights into how public policies work and, hopefully, help public authorities to design more effective policies against crime.

Supplementary Appendix

Replication R Code (dataset available at https://github.com/danilofreire/homicides-sp-synth). DOI: https://doi.org/10.25222/larr.334.s1

## Notes

1The study disregards war zones and cities with unavailable data.

2Cerqueira (2013) argues that the actual rates may be different from the official statistics. He states that many homicides from 1996 to 2010 were (intentionally or not) misclassified as “death by undetermined causes.” After performing data correction procedures, the author estimates that the number of homicides in Brazil during that period should be 18.3 percent higher than the reported figures. Recent criticism about the quality of São Paulo homicide data can also be found at “Deputado Eduardo Bolsonaro questiona ‘método Alckmin’ para taxa de homicídios: ‘Iludiu o povo,’” HuffPost Brazil, January 29, 2016, https://www.huffpostbrasil.com/2016/01/29/bolsonaro-homicidios-alckmin_n_9102858.html. In this article, I avoid these issues by using obituary data instead of police records.

3The homicide statistics cited in this paragraph come from the Centre for the Study of Violence, a research group of the University of São Paulo. Their dataset can be found at http://nevusp.org/downloads/bancodedados/homicidios/distritossp/num-homicidios-distritos-2000–2007.htm (accessed March 2016).

4See Felipe Pontes and Jones Rossi, “Conheça software que ajudou a reduzir o crime em São Paulo,” Galileu, 2009. http://revistagalileu.globo.com/Revista/Common/0,,EMI136808-17770,00-CONHECA+SOFTWARE+QUE+AJUDOU+A+REDUZIR+O+CRIME+EM+SAO+PAULO.html (accessed August 2016).

5Recent data on property crimes in São Paulo can be seen on the website of São Paulo’s Public Safety Secretary at http://www.ssp.sp.gov.br/Estatistica/Default.aspx (last update 2011; accessed July 2016).

6As shown in Figure 1, São Paulo’s homicide rates started to drop in 1999. The PCC fully consolidated their power in the prison system only around 2006 (Dias 2011).

7See “A sedução do PCC,” Federação Nacional dos Policiais Federais, May 3, 2010, http://www.fenapef.org.br/27764/.

8A Parliamentary Commission of Inquiry has stated that the PCC earns about 16 million Brazilian reals per month, which amounts to approximately 60 million US dollars per year. See “PCC fatura R\$ 16 milhões ao mês e tem cofres secretos, aponta CPI,” UOL Notícias, August 8, 2015, https://noticias.uol.com.br/cotidiano/ultimas-noticias/2015/08/08/pcc-fatura-r-16-milhoes-ao-mes-e-tem-cofres-secretos-aponta-cpi.htm. Given the size of the organization and its undisputed position as the leading crime syndicate in São Paulo, the figures are rather small. As a comparison, Mexico’s Sinaloa Cartel profits by about 3 billion dollars per year, a sum comparable to the annual earnings of Netflix or Facebook (see Patrick Radden Keefe, “Cocaine Incorporated,” New York Times, June 15, 2012, http://nyti.ms/1B09qyV).

9I would like to thank an anonymous reviewer for highlighting this point.

10Please refer to the appendix for a formal presentation of the synthetic control method.

11For overviews of cross-national studies of homicide, see LaFree (1999), Nivette (2011), and Trent and Pridemore (2012).

12The data are publicly available at Instituto de Pesquisa Econômica Aplicada, http://www.ipeadata.gov.br/. The original data files have also been added to a GitHub repository for reproducibility purposes.

13Years of Schooling and Gini Index had a small number of missing observations (about 15 percent) and those cases were imputed with linear interpolation. Both original and imputed variables are available online. See the supplementary appendix for further details on how to replicate this study.

14My estimate of lives saved by the policies implemented in São Paulo is done as follows. I consider the years after policy implementation (1999–2009), then I sum the number of homicides in São Paulo in that period. This gives us 124,077 homicides between 1999 and 2009. I do the same procedure for the synthetic São Paulo; I sum the number of homicides in each state that makes the synthetic control in the period, while adjusting the contribution of each of these states by their respective weights in the synthesis. The number of homicides in synthetic São Paulo between 1999 and 2009 is 144,408. Finally, I subtract the number of homicides in the control by the number of homicides in the treatment. The result is 20,331 lives saved.

## Appendix

This appendix presents a formal presentation of the synthetic control estimator. Let j = 1, …, J + 1 be a series of units in periods t = 1, …, T. In our case, the units are the twenty-seven Brazilian states, including the federal district, and the time period spans from 1990 to 2009. Assuming that the first unit, São Paulo, has been exposed to the treatment, we have J control units to be included in the case studies donor pool, i.e., the twenty-six remaining states. We define treatment as the series of post-1999 government anticrime policies implemented in the São Paulo.

Let ${Y}_{\mathit{\text{it}}}^{N}$ be the homicide rate that would be observed for unit i, São Paulo, at time t with no treatment (1990–1998). Conversely, let ${Y}_{\mathit{\text{it}}}^{I}$ be the observable outcome for unit i at time t had it been subjected to the treatment in periods T0 + 1 to T (1999–2009). An important assumption is that the treatment has no effect on unit i before the date of intervention, therefore, the values for São Paulo with and without the policy interventions are the same for the pretreatment period (1990–1998). In formal terms, ${Y}_{\mathit{\text{it}}}^{I}={Y}_{\mathit{\text{it}}}^{N}\text{\hspace{0.17em}}\forall \text{\hspace{0.17em}}t<{T}_{0}$. The observed outcome is defined by ${Y}_{\mathit{\text{it}}}^{I}={Y}_{\mathit{\text{it}}}^{N}+{\alpha }_{\mathit{\text{it}}}{D}_{\mathit{\text{it}}}$, where αit is the effect of crime-reducing policies on homicide rates, and Dit is a binary variable that takes the value of 1 if we refer to post-intervention period (after 1999) and 0 otherwise. The goal of this article is to estimate αit, the effect of the “treatment” (homicide reduction policies), for the state of São Paulo for all tT0, that is, from 1999 to 2009. However, we cannot observe São Paulo without those policies, as there is no way for the state to have and not have the intervention at the same time. This is what Holland (1986) calls the “fundamental problem of causal inference”: only one of the outcomes of interest is measurable at any given time.

But although we cannot accurately know how São Paulo would be without the treatment, we can approximate it by using a weighted average of the remaining Brazilian states such that . In this model, δt is an unobserved time-dependent factor common to all cases, Zi is a (1 × r) vector of observed covariates not affected by the policy, θt is a (r × 1) vector of unknown time-specific parameters, λt is a (1 × F) vector of unknown common factors to all states, μt is is a state-specific unobservable variable and it represents unobserved transitory shocks with mean 0 for all units. Basically, what SCM tries to do is to match Zi and the pretreatment Yit of the treated unit so that μt is matched as a result.

To state again clearly, synthetic São Paulo is the weighted average of the other twenty-six Brazilian states. Therefore, it is a (J × 1) vector of weights W = (w2, …, wJ + 1)′ with wj ≥ 0 for j = 2, …, J + 1 and w2 + … + wJ + 1 = 1. Each of the elements included in W represents a specific weighted average of control states, that is, a potential synthetic control for São Paulo. The idea is to select a case that resembles the treated unit as closely as possible. Let X1 be a (k × 1) vector of pre-1999 predictor variables for São Paulo and let X0 be a (k × J) matrix containing the predictor variables for the potential control states. Let ${\overline{Y}}_{i}^{{K}_{1}},\text{\hspace{0.17em}}\dots ,\text{\hspace{0.17em}}{\overline{Y}}_{i}^{{K}_{M}}$ be M linear functions of pretreatment outcomes (MF). One can choose w* such that:

$\sum _{j=2}^{J+1}{w}_{j}^{*}{Z}_{j}={Z}_{1},\text{\hspace{0.17em}}\sum _{j=2}^{J+1}{w}_{j}^{*}{\overline{Y}}_{j}^{{K}_{1}}={\overline{Y}}_{1}^{{K}_{1}},\text{\hspace{0.17em}}\dots ,\text{\hspace{0.17em}}\sum _{j=2}^{J+1}{w}_{j}^{*}{\overline{Y}}_{j}^{{K}_{M}}={\overline{Y}}_{1}^{{K}_{M}}$

Consequently, as noted by Abadie, Diamond, and Hainmueller (2010), if T0 is sufficiently large when compared to the scale of it, an approximately unbiased estimator for α1t the effect of public security policies in São Paulo, can be described by:

for all , that is, after the intervention period (1999–2009). In practice, W* is chosen non-parametrically as to minimize $‖{X}_{1}-{X}_{0}W‖$, subject to the weight constraints. We consider $‖{X}_{1}-{X}_{0}W‖v=\sqrt{\left({X}_{1}-{X}_{0}W\right)\prime \text{\hspace{0.17em}}V\left({X}_{1}-{X}_{0}W\right)}$ where V is a (k × k) symmetric and semi-definite positive matrix with the relative importance of each assigned homicide rate predictor. From various possible ways to choose V, in this article I follow the recommendation of Abadie and Gardeazabal (2003) and choose V* as the value of V that minimizes the root mean squared prediction error (RMSPE) for homicide rates in the entire pretreatment period (1990–1998).

## Acknowledgements

I would like to thank André Amaro, Guilherme Arbache, Fábio Barros, Rodolpho Bernabel, Nick Cowen, Guilherme Kerr, Rodrigo Martins, Robert Myles McDonnell, David Skarbek, the members of the Political Economy Group at Caeni/USP, two anonymous reviewers and the editor Aníbal Pérez-Liñán for their valuable comments and suggestions. The usual disclaimer applies. All data, code, and information required to replicate this study are available at the following web address: https://github.com/danilofreire/homicides-sp-synth. The author gratefully acknowledges the support of the National Council for Scientific and Technological Development and the Faculty of Social Science and Public Policy at King’s College London.

## Author Information

Danilo Freire is a PhD candidate in the Department of Political Economy at King’s College London. He received a master’s degree in international relations from the Graduate Institute of International and Development Studies, Geneva, and a master’s in political science from the University of São Paulo. His main research interests are urban crime in Latin America, prison gangs, political violence, Bayesian data analysis, and survey experiments.

## References

1. Abadie, Alberto. 2005. “Semiparametric Difference-in-Differences Estimators.” Review of Economic Studies 72(1): 1–19. DOI: https://doi.org/10.1111/0034-6527.00321

2. Abadie, Alberto, Alexis Diamond, and Jens Hainmueller. 2010. “Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program.” Journal of the American Statistical Association 105(490): 493–505. DOI: https://doi.org/10.1198/jasa.2009.ap08746

3. Abadie, Alberto, Alexis Diamond, and Jens Hainmueller. 2011. “Synth: An R Package for Synthetic Control Methods in Comparative Case Studies.” Journal of Statistical Software 42(13): 1–17. DOI: https://doi.org/10.18637/jss.v042.i13

4. Abadie, Alberto, Alexis Diamond, and Jens Hainmueller. 2014. “Comparative Politics and the Synthetic Control Method.” American Journal of Political Science 59(2): 495–510. DOI: https://doi.org/10.1111/ajps.12116

5. Abadie, Alberto, and Javier Gardeazabal. 2003. “The Economic Costs of Conflict: A Case Study of the Basque Country.” American Economic Review 93(1): 113–132. DOI: https://doi.org/10.1257/000282803321455188

6. Achen, Christopher H. 1992. “Social Psychology, Demographic Variables, and Linear Regression: Breaking the Iron Triangle in Voting Research.” Political Behavior 14(3): 195–211. DOI: https://doi.org/10.1007/BF00991978

7. Achen, Christopher H. 2002. “Toward a New Political Methodology: Microfoundations and ART.” Annual Review of Political Science 5(1): 423–450. DOI: https://doi.org/10.1146/annurev.polisci.5.112801.080943

8. Ahnen, Ron. 2003. “Between Tyranny of the Majority and Liberty: The Persistence of Human Rights Violations under Democracy in Brazil.” Bulletin of Latin American Research 22(3): 319–339. DOI: https://doi.org/10.1111/1470-9856.00080

9. Angrist, Joshua D., and Jörn-Steffen Pischke. 2008. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton, NJ: Princeton University Press.

10. Barata, Rita Barradas, and Manoel Carlos Sampaio de Almeida Ribeiro. 2000. “Relação entre homicídios e indicadores econômicos em São Paulo, Brasil, 1996.” Pan American Journal of Public Health 7(2): 118–124. DOI: https://doi.org/10.1590/S1020-49892000000200008

11. Barbarino, A., and G. Mastrobuoni. 2014. “The Incapacitation Effect of Incarceration: Evidence from Several Italian Collective Pardons.” American Economic Journal: Economic Policy 6(1): 1–37. DOI: https://doi.org/10.1257/pol.6.1.1

12. Baser, Onur. 2006. “Too Much Ado about Propensity Score Models? Comparing Methods of Propensity Score Matching.” Value in Health 9(6): 377–385. DOI: https://doi.org/10.1111/j.1524-4733.2006.00130.x

13. Beattie, Sarah, and Amy Mole. 2007. Police Resources in Canada, 2007. Canadian Centre for Justice Statistics. http://www.statcan.gc.ca/pub/85-225-x/85-225-x2007000-eng.htm.

14. Becker, Gary S. 1968. “Crime and Punishment: An Economic Approach.” Journal of Political Economy 76(2): 169–217. DOI: https://doi.org/10.1086/259394

15. Bertrand, Marianne, Esther Duflo, and Sendhil Mullainathan. 2004. “How Much Should We Trust Differences-in-Differences Estimates?” Quarterly Journal of Economics 119(1): 249–275. DOI: https://doi.org/10.1162/003355304772839588

16. Biderman, Ciro, Renato Sergio De Lima, João Manoel Pinho De Mello, and Alexandre Schneider. 2014. “Pax Monopolista and Crime: The Case of the Emergence of the Primeiro Comando da Capital in São Paulo.” Working paper, July 14. Caracas: CAF. http://scioteca.caf.com:8080/xmluiprod/handle/123456789/712.

17. Billmeier, Andreas, and Tommaso Nannicini. 2013. “Assessing Economic Liberalization Episodes: A Synthetic Control Approach.” Review of Economics and Statistics 95(3): 983–1001. DOI: https://doi.org/10.1162/REST_a_00324

18. Biondi, Karina. 2010. Junto e misturado: Uma etnografia do PCC. São Paulo: Editora Terceiro Nome.

19. Brasil de Fato. 2013. “Com maior população carcerária do Brasil, São Paulo registra 15 mil prisões em um ano.” Brasil de Fato, August 27. https://www.brasildefato.com.br/node/25651/.

20. Brodersen, Kay H., Fabian Gallusser, Jim Koehler, Nicolas Remy, and Steven L. Scott. 2015. “Inferring Causal Impact Using Bayesian Structural Time-Series Models.” Annals of Applied Statistics 9(1): 247–274. DOI: https://doi.org/10.1214/14-AOAS788

21. Brookhart, M. Alan, Sebastian Schneeweiss, Kenneth J. Rothman, Robert J. Glynn, Jerry Avorn, and Til Stürmer. 2006. “Variable Selection for Propensity Score Models.” American Journal of Epidemiology 163(12): 1149–1156. DOI: https://doi.org/10.1093/aje/kwj149

22. Bueno, Samira. 2014. “Letalidade na ação policial: Os desafios para a consolidação de uma agenda de políticas públicas no estado de São Paulo.” Administração Pública e Gestão Social 7(1): 9–15. DOI: https://doi.org/10.21118/apgs.v7i1.571

23. Buonanno, Paolo, and Steven Raphael. 2013. “Incarceration and Incapacitation: Evidence from the 2006 Italian Collective Pardon.” American Economic Review 103(6): 2437–2465. DOI: https://doi.org/10.1257/aer.103.6.2437

24. Cabral, Maria Viviana de Freitas. 2016. “Avaliação do impacto do Infocrim sobre as taxas de homicídio nos municípios paulistas: Uma aplicação do método de diferenças em diferenças espaciais.” PhD dissertation, Federal University of Juiz de Fora.

25. Camargo, Antonio Benedito Marangone. 2007. “Mortes por causas violentas no estado de São Paulo.” São Paulo em Perspectiva 21(1): 31–45.

26. Cardia, Nancy, Sérgio Adorno, and Frederico Z. Poleto. 2003. “Homicide Rates and Human Rights Violations in São Paulo, Brazil: 1990 to 2002.” Health and Human Rights 6(2): 14–33. DOI: https://doi.org/10.2307/4065428

27. Carvalho, Salo de, and Christiane Russomano Freire. 2005. “O regime disciplinar diferenciado: Notas críticas à reforma do sistema punitivo brasileiro.” Revista Transdisciplinar de Ciências Penitenciárias 4(1): 7–26.

28. Cerqueira, Daniel. 2013. “Mapa de homicídios ocultos no Brasil.” Brasília: Instituto de Pesquisa Econômica Aplicada (IPEA).

29. Cerqueira, Daniel, and João M. P. de Mello. 2013. Evaluating a National Anti-Firearm Law and Estimating the Causal Effect of Guns on Crime. Texto para Discussão, no. 607. Rio de Janeiro: PUC, Rio de Janeiro, Departamento de Economia.

30. Chen, M. Keith, and Jesse M. Shapiro. 2007. “Do Harsher Prison Conditions Reduce Recidivism? A Discontinuity-Based Approach.” American Law and Economics Review 9(1): 1–29. DOI: https://doi.org/10.1093/aler/ahm006

31. Clarke, Kevin A. 2005. “The Phantom Menace: Omitted Variable Bias in Econometric Research.” Conflict Management and Peace Science 22(4): 341–352. DOI: https://doi.org/10.1080/07388940500339183

32. Clarke, Kevin A. 2009. “Return of the Phantom Menace Omitted Variable Bias in Political Research.” Conflict Management and Peace Science 26(1): 46–66. DOI: https://doi.org/10.1177/0738894208097666

33. Coffman, Makena, and Ilan Noy. 2012. “Hurricane Iniki: Measuring the Long-Term Economic Impact of a Natural Disaster Using Synthetic Control.” Environment and Development Economics 17(2): 187–205. DOI: https://doi.org/10.1017/S1355770X11000350

34. Consejo Ciudadano para la Seguridad Pública y Justicia Penal. 2014. The 50 Most Violent Cities in the World 2014. Mexico City.

35. Cornish, Derek B., and Ronald V. Clarke. 2014. The Reasoning Criminal: Rational Choice Perspectives on Offending. London: Transaction Publishers.

36. Dehejia, Rajeev H., and Sadek Wahba. 2002. “Propensity Score-Matching Methods for Nonexperimental Causal Studies.” Review of Economics and Statistics 84(1): 151–161. DOI: https://doi.org/10.1162/003465302317331982

37. Dias, Camila Caldeira Nunes. 2009a. “Da guerra à gestão: Trajetória do Primeiro Comando da Capital (PCC) nas prisões de São Paulo.” Revista Percurso 10(2): 79–96.

38. Dias, Camila Caldeira Nunes. 2009b. “Ocupando as brechas do direito formal: O PCC como instância alternativa de resolução de conflitos.” Dilemas 2(4): 83–105.

39. Dias, Camila Caldeira Nunes. 2011. “Da pulverização ao monopólio da violência: Expansão e consolidação do Primeiro Comando da Capital (PCC) no sistema carcerário paulista.” PhD dissertation, Universidade de São Paulo.

40. Eck, John, and Edward Maguire. 2006. “Have Changes in Policing Reduced Violent Crime? An Assessment of the Evidence.” In The Crime Drop in America, edited by Alfred Blumstein, and Joel Wallman, 207–265. Cambridge: Cambridge University Press.

41. Ehrlich, Isaac. 1975. “The Deterrent Effect of Capital Punishment: A Question of Life and Death.” American Economic Review 65(3): 397–417.

42. Feltran, Gabriel de Santis. 2010. “Crime e castigo na cidade: Os repertórios da justiça e a questão do homicídio nas periferias de São Paulo.” Caderno CRH 23(58): 59–73. DOI: https://doi.org/10.1590/S0103-49792010000100005

43. Feltran, Gabriel de Santis. 2012a. “Governo que produz crime, crime que produz governo: O dispositivo de gestão do homicídio em São Paulo (1992–2011).” Revista Brasileira de Segurança Pública 6(2): 232–255.

44. Feltran, Gabriel de Santis. 2012b. “Manter a ordem nas periferias de São Paulo: Coexistência de dispositivos normativos na ‘era PCC.’” In Ilegalismos, cidade e política, edited by Christian Azaïs, Gabriel Kessler, and Vera da Silva Telles, 253–279. Belo Horizonte: Fino Traço.

45. Folha de São Paulo. 2001. “Estatuto do PCC prevê rebeliões integradas.” Folha de São Paulo, February 19. http://www1.folha.uol.com.br/folha/cotidiano/ult95u22521.shtml.

46. Freire, Danilo. 2014. “Entering the Underworld: Prison Gang Recruitment in São Paulo’s Primeiro Comando da Capital.” Master’s thesis, Graduate Institute of International and Development Studies, Geneva.

47. Glaeser, Edward, Bruce Sacerdote, and José Scheinkman. 1996. “Crime and Social Interactions.” Quarterly Journal of Economics 111(2): 507–548. DOI: https://doi.org/10.2307/2946686

48. Goertzel, Ted, and Tulio Kahn. 2009. “The Great São Paulo Homicide Drop.” Homicide Studies 13(4): 398–410. DOI: https://doi.org/10.1177/1088767909348711

49. Guimarães, Thiago. 2016. “PCC não derrubou homicídios sozinho em SP, dizem pesquisadores.” BBC Brasil, February 18. http://www.bbc.com/portuguese/noticias/2016/02/160217_pcc_debate_tg.

50. Heim, Bradley T., and Ithai Z. Lurie. 2014. “Does Health Reform Affect Self-Employment? Evidence from Massachusetts.” Small Business Economics 43(4): 917–930. DOI: https://doi.org/10.1007/s11187-014-9572-6

51. Hinrichs, Peter. 2012. “The Effects of Affirmative Action Bans on College Enrollment, Educational Attainment, and the Demographic Composition of Universities.” Review of Economics and Statistics 94(3): 712–722. DOI: https://doi.org/10.1162/REST_a_00170

52. Ho, Daniel E., Kosuke Imai, Gary King, and Elizabeth A. Stuart. 2007. “Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference.” Political Analysis 15(3): 199–236. DOI: https://doi.org/10.1093/pan/mpl013

53. Holland, Paul W. 1986. “Statistics and Causal Inference.” Journal of the American Statistical Association 81(396): 945–960. DOI: https://doi.org/10.1080/01621459.1986.10478354

54. Hughes, Pedro Javier Aguerre. 2004. “Segregação socioespacial e violência na cidade de São Paulo: Referências para a formulação de políticas públicas.” São Paulo em Perspectiva 18(4): 93–102. DOI: https://doi.org/10.1590/S0102-88392004000400011

55. Imbens, Guido W., and Donald B. Rubin. 2015. Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9781139025751

56. Jinjarak, Yothin, Ilan Noy, and Huanhuan Zheng. 2013. “Capital Controls in Brazil: Stemming a Tide with a Signal?” Journal of Banking and Finance 37(8): 2938–2952. DOI: https://doi.org/10.1016/j.jbankfin.2013.04.007

57. Kahn, Tulio, and André Zanetic. 2005. “O papel dos municípios na segurança pública.” Estudos Criminológicos 4: 1–68.

58. LaFree, Gary. 1999. “A Summary and Review of Cross-National Comparative Studies of Homicide.” In Homicide: A Sourcebook of Social Research, edited by M. Dwayne Smith and Margaret A. Zahn, 125–145. London: Sage.

59. Levitt, Steven. 1996. “The Effect of Prison Population Size on Crime Rates: Evidence from Prison Overcrowding Litigation.” Quarterly Journal of Economics 111(2): 319–351. DOI: https://doi.org/10.2307/2946681

60. Levitt, Steven. 1997. “Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime.” American Economic Review 87(3): 270–290.

61. Levitt, Steven. 2004. “Understanding Why Crime Fell in the 1990s: Four Factors That Explain the Decline and Six That Do Not.” Journal of Economic Perspectives 18(1): 163–190. DOI: https://doi.org/10.1257/089533004773563485

62. Manso, Bruno Paes. 2012. “Crescimento e queda dos homicídios em SP entre 1960 e 2010: Uma análise dos mecanismos da escolha homicida e das carreiras no crime.” PhD dissertation, University of São Paulo.

63. Manso, Bruno Paes, and Marcelo Godoy. 2014. “20 anos de PCC – o efeito colateral da política de segurança pública.” Interesse Nacional 24(6): 26–35.

64. Marsh, J. L., J. L. Hutton, and Keith Binks. 2002. “Removal of Radiation Dose Response Effects: An Example of Over-Matching.” BMJ 325(7359): 327–330. DOI: https://doi.org/10.1136/bmj.325.7359.327

65. Mello, Joao Manoel Pinho de, and Alexandre Schneider. 2010. “Mudança demográfica e a dinâmica dos homicídios no estado de São Paulo.” São Paulo em Perspectiva 21(1): 19–30.

66. Montalvo, Jose G. 2011. “Voting after the Bombings: A Natural Experiment on the Effect of Terrorist Attacks on Democratic Elections.” Review of Economics and Statistics 93(4): 1146–1154. DOI: https://doi.org/10.1162/REST_a_00115

67. Morgan, Stephen L., and Christopher Winship. 2014. Counterfactuals and Causal Inference. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9781107587991

68. Nagin, Daniel S. 2007. “Moving Choice to Center Stage in Criminological Research and Theory.” Criminology 45(2): 259–272. DOI: https://doi.org/10.1111/j.1745-9125.2007.00078.x

69. Nivette, Amy E. 2011. “Cross-National Predictors of Crime: A Meta-Analysis.” Homicide Studies 15(2): 103–131. DOI: https://doi.org/10.1177/1088767911406397

70. Owens, Emily G. 2009. “More Time, Less Crime? Estimating the Incapacitative Effect of Sentence Enhancements.” Journal of Law and Economics 52(3): 551–579. DOI: https://doi.org/10.1086/593141

71. Paternoster, Raymond. 2010. “How Much Do We Really Know about Criminal Deterrence?” Journal of Criminal Law and Criminology 100(3): 765–824.

72. Pearl, Judea. 2001. “Direct and Indirect Effects.” In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, edited by Jack Breese and Daphne Koller, 411–420. San Francisco: Morgan Kaufmann Publishers, Inc.

73. Pearl, Judea. 2009. Causality. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511803161

74. Peres, Maria Fernanda Tourinho, Diego Vicentin, Marcelo Batista Nery, Renato Sérgio de Lima, Edinilsa Ramos de Souza, Magdalena Cerda, Nancy Cardia, and Sérgio Adorno. 2011. “Queda dos homicídios em São Paulo, Brasil: Uma análise descritiva.” Revista Panamericana de Salud Publica 29(1): 17–26. DOI: https://doi.org/10.1590/S1020-49892011000100003

75. Pinheiro, Paulo Sérgio. 2000. “Democratic Governance, Violence, and the (Un)Rule of Law.” Daedalus 129(2): 119–143.

76. Pinheiro, Paulo Sérgio. 2001. “The Paradox of Democracy in Brazil.” Brown Journal of World Affairs 8(2): 113–122.

77. Risso, Melina Ingrid. 2014. “Intentional Homicides in São Paulo City: A New Perspective.” Stability: International Journal of Security and Development 3(1): 1–12.

78. Rubin, Donald B. 1973. “Matching to Remove Bias in Observational Studies.” Biometrics 29(1): 159–183. DOI: https://doi.org/10.2307/2529684

79. Rubin, Donald B. 2006. Matched Sampling for Causal Effects. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511810725

80. Salla, Fernando. 2007. “De Montoro a Lembo: As políticas penitenciárias em São Paulo.” Revista Brasileira de Segurança Pública 1(1): 72–90.

81. Santos, Fabio Franklin Storino dos. 2008. “Um partido, três agendas? Política de segurança pública no estado de São Paulo: 1995–2006.” PhD dissertation, Fundação Getúlio Vargas.

82. Skarbek, David. 2011. “Governance and Prison Gangs.” American Political Science Review 105(4): 702–716. DOI: https://doi.org/10.1017/S0003055411000335

83. Sobel, Michael E. 1987. “Direct and Indirect Effects in Linear Structural Equation Models.” Sociological Methods and Research 16(1): 155–176. DOI: https://doi.org/10.1177/0049124187016001006

84. Stuart, Elizabeth A. 2010. “Matching Methods for Causal Inference: A Review and a Look Forward.” Statistical Science 25(1): 1–21. DOI: https://doi.org/10.1214/09-STS313

85. Trent, Carol L. S., and William Alex Pridemore. 2012. “A Review of the Cross-National Empirical Literature on Social Structure and Homicide.” In Handbook of European Homicide Research, edited by Marieke Liem and William Alex Pridemore, 111–135. New York: Springer. DOI: https://doi.org/10.1007/978-1-4614-0466-8_7

86. United Nations Office on Drugs and Crime. 2013. Global Study on Homicide 2013: Trends, Contexts, Data. Vienna: United Nations Office on Drugs and Crime.

87. Waiselfisz, Julio Jacobo. 2011. “Mapa da violência 2011: Os jovens do Brasil.” Brasília: Brazilian Federal Government/FLACSO.

88. Waiselfisz, Julio Jacobo. 2014. “Mapa da violência 2014: Os jovens do Brasil.” Brasília: Brazilian Federal Government/FLACSO.

89. Western, Bruce, Jeffrey R. Kling, and David F. Weiman. 2001. The Labor Market Consequences of Incarceration. Crime and Delinquency 47(3): 410–427. DOI: https://doi.org/10.1177/0011128701047003007

90. Willis, Graham Denyer. 2014. “Antagonistic Authorities and the Civil Police in São Paulo, Brazil.” Latin American Research Review 49(1): 3–22. DOI: https://doi.org/10.1353/lar.2014.0016

91. Willis, Graham Denyer. 2015. The Killing Consensus: Police, Organized Crime, and the Regulation of Life and Death in Urban Brazil. Berkeley: University of California Press. DOI: https://doi.org/10.1525/california/9780520285705.001.0001

92. World Health Organization. 2015. “Social Determinants of Health.” Geneva: United Nations. http://www.who.int/social_determinants/en/.