I am applied mathematician and probabilist living in downtown Durham and working in the Duke Mathematics Department. I returned to my home state of North Carolina in 2003 after 15 years away. I am a graduate of Durham's NCSSM, Yale and Princeton.
Today the state court chose to accept the Remedial Map which the legislature produced in response to the maps we have been using since 2016 being declared an illegal gerrymander in Harper v. Lewis.
Sadly this new map still has districts containing an abnormally large number of Democrats and others with comparably few. The result is a map which elects the same number of Democrats and Republicans over an abnormally large range of partisan election outcomes.
My analysis shows that the Remedial Map was much less sensitive to swings in the partisan vote fractions than the vast majority of the maps in the ensemble. The plots below, where the Remedial Plan is labeled HB 1029, show that under a uniform swing analysis the nonpartisan maps in the ensemble often produce 6 and sometimes 7 Democratic seats in election environments when the Democrats perform well (a statewide vote fraction in the low 50%) for many sets of votes, while the 2019 Remedial Map reliably produces 5 Democratic seats in most instances. Each of the different plots uses a different set of historical votes (Labels: USS16- US Senate 2016, AG16 – Attorney General 2016, USH16 – US House 2016, GOV16 – Governor 2016).
The important feature is the atypically large jump between the 6th and 5th most Democratic district. This is the same “signature of gerrymandering” jump which we saw in Common Cause v. Rucho. It makes the maps much less responsive to the votes cast. They produce the same results over a large number of election scenarios.
The fact that the court felt it did not have time to investigate this fully shows how the judicial pathway of preventing gerrymandering is limited. We need legislative reform. It seems the only way to ensure that the maps used in elections are responsive to shifts in the political sentiment of the electorate.
We have revised our work quantifying gerrymandering in the N.C. Congressional Districts. The original work was presented in Common Cause v Rucho. We have taken the opportunity to use the improved code base which performs more sophisticated convergence test and update our discussion to better reflect our current perspective. We have also corrected a few inaccuracies and small errors. That being said, the results are fully in line with those from the original version.
[Note: there is a known bug between WordPress and Safari; if the videos below do not render on Safari try switching to a different browser]
In our analysis of the 2017 North Carolina General Assembly redistricting plan, one of our central findings was that the NC Legislature’s 2017 Redistricting Plan implement a firewall protecting Republican majorities and supermajorities.
In trial, for the House districts, we showed animated bar-graphs that demonstrated how the range of democratic seat counts shifted with the statewide fraction of Democratic votes, under various shifts to historical elections. For example, under the United States Senate vote in 2016, the enacted plan elects a typical number of democrats when compared to the ensemble when the statewide Democratic vote fraction is below 49%. As the Democratic vote fraction rises to roughly 50.5% to over 52%, nearly all plans in the ensemble break the Republican supermajority, but the enacted plan remains stuck electing fewer than 48 Democrats to the state House. As the Democratic vote fraction continues to rise, the enacted plan consistently elects fewer Democrats than the ensemble; at a Democratic vote fraction of 54.5%, nearly all plans in the ensemble predict a Democratic majority in the House, yet the ensemble retains a Republican majority. The Republicans retain their majority in the enacted plan even when the Democratic vote fraction surpasses 55%; plans in the ensemble yield a strong majority to the democrats at this point.
The story is NOT about proportional representation, rather about how the #ncga 2017 maps (represented by arrow) systematically under-elect Democrats to a shocking degree.
The story is similar when using vote counts from the 2012 presidential race. Notice, in both videos, as the Democratic vote fraction rises to break the Republican supermajority in the ensemble, the enacted plan dramatically remains to the left of the majority/supermajority line.
And as we examine more elections, the story still remains the same. With Commissioner of Insurance votes from 2012, notice how Democrats are systematically under-elected by the #ncga 2017 Redistricting Plan when compared to our ensemble of thousands and thousands of non-partisan maps
Now with 2008 United States Senate votes. Getting worried about our democracy yet? Across many different vote patterns, same exceptionally atypical under-electing of Democrats persists. The effect is very robust across different offices, across different years.
Same story, now using votes from 2012 Governor election. Each election has a different spatial vote pattern, yet story persists. 2017 #ncga map under-elects Dems when they would typically gain more power. Notice map keeps a Republican majority even when some maps give Democrats a supermajority.
Finally, using 2016 Lt. Governor votes. Again same story. The ensemble accounts for natural packing and the voting geography of North Carolina. But natural packing is not enough to explain the enacted plan’s extreme Republican bias. By comparing the enacted plan to the ensemble of non partisan maps, we separate the effects of natural packing from partisan gerrymandering.
Although the above maps will be remedied by the decision of Lewis v. Common Cause, many states are still highly vulnerable to the effects and consequences of partisan gerrymandering with no hope for remedy from the federal courts. Map makers have inserted themselves in the electoral process and suppressed the Will of the People. If you are worried about the state of our democracy, you should be.
North Carolina’s constitution requires that state legislative districts should not split counties. However, counties must be split to comply with the “one person, one vote” mandate of the U.S. Supreme Court. Given that counties must be split, the North Carolina legislature and courts have provided guidelines that seek to reduce counties split across districts while also complying with the “one person, one vote” criteria. Under these guidelines, the counties are separated into clusters. For a great explainer about the County Clustering problem see this blog entry on Districks.
A group of high school students from the NC School of Science and Mathematics worked with us over the last academic year and summer to develop computer algorithms to optimally cluster counties according to the guidelines set by the court in 2015. We recently released an article that presents the algorithm along with publicly accessible code which anyone can use.
Additionally, in our article, we use this to investigate the optimality and uniqueness of the enacted clusters under the 2017 redistricting process. We verify that the enacted clusters are optimal, but find other optimal choices. We emphasize that the tool we provide lists all possible optimal county clusterings. We also explore the stability of clustering under changing statewide populations and project what the county clusters may look like in the next redistricting cycle beginning in 2020/2021.
The posting to the ArXiv can be found here.
The article is now published and can be found here.
The code is referenced in the ArXiv post, and may also be accessed here.
[Edits: Added like to Districks explainer (9/4/2019)]
One recurring theme we see is that the results in enacted maps are “baked in.” That is to say that over a wide range of elections changes in the votes do not lead to changes in the partisan make up of the representatives elected. This effect was very pronounced in the N.C. Congressional maps used in 2012 and 2016 . The enacted maps (NC 2012 & NC 2016) produce the same result over a large range of elections. These elections have varying statewide vote fractions; the ensemble shows that the number of elected Democrats change as the elections change, whereas the enacted plans nearly always give the same result.
Here we have shown a number of historical elections and arranged them on the plot from most Democratic to most Republican. We see that as the Republican vote fraction increases the blue histograms show that the typical number of seats elected by our ensemble of 24,000 or so maps shifts towards the Republican direction. The number of Republican seats elected by the Beyond Gerrymandering Judge’s map responds to the shifting public opinion by shifting the number of Republicans elected. The Enacted plans from 2012 and 2016 don’t change over a wide range. In the enacted plan, the loss of support for a party does not result in the loss of seats. In the ensemble and the judges plan, the loss of support for a party causes that party to lose seats.
This effect can be understood in the box-plot graphs of the ordered marginal plots.
The large jump in the plot, which leads to a range of election results for which the partisan outcomes don’t change, was has been referred to as the “Signature of Gerrymandering.”
One of the strengths of sampling the space of redistricting plans is that it makes no assumptions about the relationship between the statewide votes cast and the seats won. The collection of plans do not consider proportionality between seats and votes or symmetry in electoral outcomes.
These methods reveal the expected election results under our election system; they assume only the redistricting criteria (usually non-partisan). The sampled maps provide a null hypothesis. Any enacted map can be compared to this null hypothesis to understand the extent of any possible gerrymandering.
The collection of maps, and the expected election outcome they reveal, automatically includes the effect of cities, the distribution of voters and the shape of the region in a simple and principled way. We have looked at North Carolina, Wisconsin and more informally Maryland and Pennsylvania. We have never seen proportionality between the votes and typical seats given by an ensemble.
For example, consider the election of the US House of representatives for North Carolina and the election of the General Assembly of delegates for Wisconsin. In the first, we compare the 2012 enacted plan against the ensemble using the 2012 Congressional votes; in the second, we compare the 2011 enacted Wisconsin plan against the ensemble using the 2014 General Assembly votes.
NC House 2012
Votes and Seats for N.C. House in 2012 using 2012 enacted map.
Votes and Seats for Wisconsin General Assembly using votes and enacted plan from 2014.
Clearly neither seat outcome of the enacted plan would be call proportional representation. However, the large ensemble of non-partisan maps reveals that the results in Wisconsin are, in fact, typical for this set of votes whereas the North Carolina outcome is not.
Although the WI enacted plan is representative for this particular set of votes, when the Democratic vote fraction drops below 50%, the enacted plan acts as an extreme outlier producing highly atypical results. (This effect is well understood: See Firewall and The Signature of Gerrymandering.) This effect is shown below.
Notice that the trend shown by the ensemble histograms in blue do not follow the vote-seat proportionality line marked on the plot. However the Enacted map is still shown to be an outlier as the vote fraction drops below 50%. It shows a “baked in” result which is do not move as the people’s will expressed in their votes changes.
This “baked in” election outcome is even more pronounced in N.C. where the enacted maps (NC 2012 & NC 2016) produce the same result over a large range of elections. These elections have varying statewide vote fractions; the ensemble shows that the number of elected Democrats change as the elections change, whereas the enacted plans nearly always give the same result.
Again the proportionally line on the plot does not track the shift in the blue histograms produced by the ensembles as the election used varies. The ensemble reveals the natural baseline without assuming proportionality.
This post continues the theme of localized analysis begun in the post Towards a Localized Analysis. The analysis presented in our articles and blog post such as The Signature of Gerrymandering , Firewalls, and Hearing the Will of the People are largely based on a global, statewide analyses. In many ways, Gerrymandering is a global phenomenon as one can not change one district with out effecting its neighbors which often causes a cascade of changes across a state. Packing a particular group in one district is done to dilute the groups effect in other districts.
The most basic measure of a districting plan’s character is the partisan make up of its elected representatives. When comparing a plan’s partisan make up with a set of comparison maps, we obtain a view that is fundamentally global in measuring gerrymandering. Marginal box-plots can be used to identify particular districts as unusual. This is a step towards a more localized analysis. As described in The Signature of Gerrymandering post the marginal box-plots of the ranked vote curve were used to identify specific districts, in the legislatures 2016 maps, which were arguably cracked and packed in court case Common Cause v. Rucho concerning the 2016 NC Congressional maps.
In the Marginal box-plots, a particular set of votes is used to order the districts from most to least Republican in the ensemble of districting plans; to examine a particular district in a given plan, we first consider the district’s rank in the order (from most to least Republican) and then compare that district’s partisan make up with the partisan make up of the districts in other plans with the same rank in the order. Such analysis examines deviations in the the overall statewide structure of votes along with with where this deviation may occur, but it does not geographically constrain the districts with which to compare the district of interest.
We wish, however, to perform a more geographically localized analysis. Such analysis still contains the fundamental framework of examining outliers within the context of an ensemble of nonpartisan maps. (Outlier analysis using an ensemble of maps is generally discussed here. The basic idea of the localized analysis was previously presenter here.)
The Typical Local Political Environment of a Precinct
Localized analysis begins by choosing a particular precinct of interest and collecting the districts containing this precinct from an ensemble of state wide maps. The resulting collection of districts, all which contain the precinct of interest, can then be used to characterize the typical district-level political environment that voters in this precinct could expect to experience.
By contextualizing a precinct within the collection of districts containing it, we may relate the partisan preferences of the precinct’s voters to the preferences of their typical district. For example, a precinct’s voters may prefer one party, but typically find themselves in a district that votes for the opposing party; in this sense we may determine whether or not it is natural for the typical voter within a given precinct to be able to elect a candidate associated with the party they have voted for.
We contextualize the typical district containing a precinct by constructing a histogram over the ensemble of districts (this process was described in detail the previous post Towards a Local Analysis.) This distribution can then be used to deicide if the district containing the precinct in a given plan is atypical. If a precinct does find itself in an atypical district, then its residents may have cause to object. In this way, the ensemble of districts can be used in a normative way to identify outliers.
Extreme Outliers in the NC Congressional Races
Similarly to our previous blog post (Towards a Local Analysis) we again apply the above analysis the North Carolina Congressional Map from the 2016 and 2012 elections which we will abbreviate respectively NC2106 and NC2012. We will also consider the map produce by the retired judges from Beyond Gerrymandering Project lead by Tom Ross. This map will be denoted “Judges”. Previously we used a 5% outlier condition which many may classify as rare, but is more arguably extreme. We adapt the current analysis to examine more extreme outliers (those that occur less than 1% of the time, rather than 5%).
We begin by analyzing each precinct to determine if the district counting the precinct in the map of interest has an unusual partisan make up from that precincts perspective. Precincts which find themselves in a district whose partizan make up is in the in tails corresponding less than 1% of probability are labeled as extreme (at p=99%)
In the table below, we show the number of precincts which are extreme outliers from 2,692 predicts in different NC Congressional maps for the using the 2012 and 2016 US Congressional elections, denoted by USH12 and USH16 respectively. For comparison purposes, we then tabulate the number of maps in our ensemble which have that many or more precincts which are extreme outliers.
# in ensemble
w/ more outliers
# in ensemble
w/ more outliers
The histograms below give the full histograms of the number of maps with different numbers of outlier precincts using the two sets of election data mentioned above.
The spatial location of each of the outlier districts is show on the following maps. The first three us the US12 election data
The second three use the USH16 election data.
Precinct by Precinct Log Likelihood
Instead of just using the using the precinct localized distribution constructed above to flag districts individual as outliers, one can calculate the likelihood of partizan vote fraction being as far or farther in the tail then that each precinct in a given map. Averaging this across the state gives a measure of how typical the precinct by precinct are across the whole map. To contextualize this average spatial log likelihood calculate the value for each map in our ensemble. The following two histograms summarize the results.
Again to show the spatial structure better, we plot the log likelihood across the state, precinct by precinct. So that the direction of the swing is visible, we label democratic with positive values and republican with negative.
The above maps use USH12 election results while the maps below use UH16 election results.
In Gill v. Whitford, twelve voter-plaintiffs challenged the Wisconsin legislature’s 2011 redistricting as a violation of Fourteenth Amendment equal protection rights and a burden on First Amendment associational rights. After a bench trial, the district court concluded that the 2011 map had the intent and unjustified effect of “plac[ing] a severe impediment on the effectiveness of the votes of individual citizens on the basis of their political affiliation,” and entered judgment in favor of plaintiffs on both constitutional claims. The trial court based its findings of discriminatory effects on a compelling and extensive record that would have withstood appellate review under the deferential standard applicable to such causal and statistical inferences.
In a recent unanimous decision, however, the Supreme Court did not reach the merits of the claims, and instead vacated the district court’s judgment on the grounds that the plaintiffs had failed to demonstrate standing to bring their case. Describing the necessary standing as limited to “voters who allege facts showing disadvantage to themselves as individuals,” the Court determined that only four of the twelve voter-plaintiffs had complained of injuries specifically stemming from the packing or cracking of their own districts, and that the case at trial had improperly focused on statewide harm. The four plaintiffs pleading individual harms will have another opportunity to advance their claims on remand.
Without directly addressing the justiciability or merits of the claims, the Court noted that the plaintiffs had based their showing of statewide harms on measurements of partisan asymmetry, in accordance with the “social science tenet that maps should treat parties symmetrically.” The basic idea that courts can evaluate the severity of partisan gerrymandering in terms of “the extent to which a majority party would fare better than the minority party, should their respective shares of the vote reverse,” had been previously considered in LULAC v. Perry but found by Justice Kennedy to “shed no light on ‘how much partisan dominance is too much.’ ” The Gill plaintiffs had responded to this concern by offering the efficiency gap as a quantitative measure of partisan asymmetry, defined by political scientist Eric McGhee and legal scholar Nicholas Stephanopoulos as “the difference between the parties’ respective wasted votes in an election, divided by the total number of votes cast.” As the Court pointed out, however, a single statewide number could not describe the effects of the 2011 redistricting on individual voters in different parts of the state.
The efficiency gap has other shortcomings and peculiarities, as other commentators have pointed out. It can treat highly competitive 50-50 districts as problematic and lopsided 75-25 districts as fairly drawn, and is prone to imprecision in states having congressional delegations too small to closely reflect vote shares. It also enshrines a previously unrecognized principle that a party’s legislative majority should be twice the proportionate size of its statewide vote majority (at least when turnout is roughly uniform across districts). Continue reading “Analyzing Partisan Gerrymandering Through Geopolitical Structure After Gill”
Up until now, this blog has investigated whether gerrymandering has occurred. In this post we begin to investigate where gerrymandering has occurred. The question of ‘where’ is interesting for both scientific and legal reasons. Scientifically, one may want to determine which precincts were atypically manipulated to achieve a political goal. Legally, an argument for individual harm is needed to pursue suits based on 14th amendment claims: Whitford vs Gill was decided on standing and dismissed because the plaintiffs did not establish that they had been individually harmed by the redistricting process.
In this post we explore the merits of a particular measure of localized gerrymandering. The idea is a simple one and continues to rely on the ensemble analysis that we have already employed. Any given precinct will always lie within some district; given historical vote counts this district will have a democratic vote fraction. For a given precinct we can construct a histogram of all democratic vote fractions over all maps within the ensemble for a fixed historical election. Then, given some reference map, we can ask how atypical the precinct’s observed margin is, given this distribution. If a precinct lies within a district that has a much higher democratic vote fraction than expected, we will color it blue; if it lies in a district that has a much higher republican vote fraction than expected, we will color it red. We color based on the log-likelihood of of the cumulative distribution functions (e.g. more red means there is a low probability that the likelihood of finding a more Republican district is small). We first give an example distribution of the vote fractions of a precinct’s districts along with how we color them
Using the votes cast in the 2016 congressional election, we color the district maps for NC2012, NC2016 and the Judges plan, respectively:
The dictionary definition of gerrymandering, “to manipulate or change the boundaries of (an electoral constituency) so as to favor one party or class,” presupposes some conception of the expected outcome and structure of an election. Implicitly a complainant challenges a map as gerrymandered because it fails to align with this expectation in some critical way.
Measures like the efficiency gap or partisan symmetry offer conceptual, idealized alternatives against which a given map’s compliance is measured. The efficiency gap values a rather arbitrary relationship between the number of seats and the overall vote fractions. (See Bernstein and Duchin or Tapp). Partisan symmetry posits the superficially appealing idea that both parties should be treated symmetrically when each have the same global vote fraction, ignoring if symmetry is truly reflected in a geopolitical landscape. Neither takes into account the heterogeneity of population densities, the shapes of states or differences in where various political constituencies live. Requirements imposed by the Voting Rights Act or statutes requiring that counties or municipalities be left whole where possible, are not directly incorporated into the partisan symmetry measure.
There is another path, which builds on well- established methods of modern computational statistics: constructing a representative ensemble which can be used normatively to establish a baseline against which deviations can be measured.
It does not dictate political outcomes based on some idea of fairness where it is difficult to incorporate demographic ground truths, the effects of geography, or the preservation of municipal entities and communities of interest. Rather one encodes the design principles of a desirable map in an ensemble of maps and then lets the typical electoral behavior reveal itself.
The design criteria are typically nonpartisan criteria such as spatial compactness, contiguity, equal population division and the minimization of fragmented counties and municipalities. It can also include items such as compliance with the Voting Rights Act (VRA). With the possible exception of VRA compliance, these criteria do not overtly create any new partisan bias, leaving (rightly) only those inherent in our political system and the distribution of voters about a particular geographically distinct state.
It is important to separate (i) the use of such an ensemble, (ii) its generation and (iii) the choice of the design criteria. Each is intellectually independent and, as such, can be critiqued separately. Here we mainly wish to concentrate on the use of an ensemble. Though we will comment briefly on the other elements, we will largely leave the discussion of the other elements to future works or posts.
Using Normative Ensembles
Once one has fixed a collection of representative maps, they can be used to probe the structure of a given election. Among other things, the ensemble reveals typical outcomes of an election and the typical over all structure of the election. The ensemble encodes the geopolitical structure of the state as it includes the population distribution as well as the spatial alignment and connectivity of the state. If desired, it can also encode the need to keep cities or counties whole or to satisfy the Votings Rights Act. Hence any conclusions drawn on the typical structure and outcome of a collection of votes is inherently informed by the geopolitical makeup of the region and the other desired criterion encoded in the construction of the ensemble. Continue reading “Ensembles and Outliers”