Congressional Elections are More Partisan, National than Ever Before

There are many hundreds of quotes by our nation’s founders noting the importance of representative government. For each one, there is also a quote about the dangers of improper government — my favorite is John Adams’ proclamation that “…division of the republic into two great parties, each arranged under its leader….is to be dreaded as the greatest political evil under our Constitution.” John Adams — along with James Madison (early on), George Washington, and others — recognized the danger of organized groups competing for control of the National government.

This is not news to any student of American politics, but geez, would John Adams et. al. be mad.

The powerful influences of parties on national politics are all around us. Did you vote in a primary election in 2016? Thank the parties. When someone says “Democrat” or “Republican” do your ears perk up as if they’re talking about you? That too is an artifact of Adams’ “(not so-)great parties.” I aim to show you that these partisan influences have gained much ground over time; where it once held sway just over candidate selection at the national level, partisanship is now being used to dictate 90% of nearly every congressional election outcome.

Our story begins in 1952 — not because that’s when Congress passed the Communist Control Act but because that’s where our data begins. Thanks to Gary Jacobson, professor of political science at the University of California, San Diego; Geoffrey Skelley, associate editor of Sabato’s Crystal Ball; and Carlos Algara, political science Ph.D. candidate at the University of California, Davis for providing the election results used in this post.

My plan is to use these data to show that, over time, both chambers of Congress have become more stable and more reflective of national vote intention. This “nationalization” (Abramowitz and Webster 2015) of the House and Senate has broad implications, notably the decline of the “incumbency advantage” — a small boost in votes that Representatives and Senators receive for holding office before.

In sum, our politics are more partisan than they have been ever before, reducing the importance of the local and moving our civics towards the national. The popularity of the president has become more influential than ever.

Here’s the long version:


Nationalization of the US House


In 1952 the United States was in the midst of a proto-nationalist surge in party politics. Opposition leaders were accused of being Communists and many Americans were brought to testify before Congress via unseemly procedures (to put it lightly). It was a time of ideological patriotism, of “us” versus “them.” Parties banded together to further a “democratic” (capitalistic) vision for the United States, but at substantial costs.

That unity may have been responsible for relatively high levels of partisanship and electoral stability. If we were to predict the 1952 House election results with results from 1950, we would have about 81% of the answer across each House race. This indicates a very low year-to-year shift in the public’s electoral preferences. Given that the reelection rate of congressional incumbents has been above 90% in 21 of the last 27 House elections this is not surprising.

More significant to the “nationalization” narrative, if we were to predict each individual election to the House of Representatives with a district’s vote in the previous two presidential elections[1] we could have explained 74% of the variation in outcome. The presidential “benchmark” (though calculated differently here) is theorized by Abramowitz and Webster to more closely measure the true partisanship of a district; since more people vote in presidential elections than congressional elections we should have a more-fleshed-out picture of the electorate’s average (or “aggregate”) partisanship.


From 1952 to 1990, however, we see a decline in the importance of partisanship to House election outcomes. The variance in lower chamber results explained by presidential elections falls to a low of 19% in 1966, rising only to 30-40 percentage points before a huge spike in 1990.

And that’s when our present state of nationalization began to form. From 1990 to 2016 the predictability of House elections using our presidential benchmark increased more than twofold. In 2016 (a very partisan election, to be sure), House elections played out in such a nationalized political environment that partisanship was more correlated with outcomes than individual 2014 election results. In other words, in districts where the electorate moved towards one political party, voters expressed that movement in ballots for either Hillary Clinton/President Trump and for their local congressperson.

The increase in nationalization prompts some serious questions about a long-held belief that incumbent Representatives enjoy an advantage over their opponent. We can determine that advantage by means of linear regression, where we predict a district’s two-party Democratic margin with our presidential benchmark and a variable for incumbency. The results show a decrease in the effects of holding office that starts in 1990 and dramatically accelerates at the turn of the century. The graphic below illustrates the evolution — or rather, the decrease — of the incumbency advantage over time.


The boost in electoral performance enjoyed by incumbents could be due to a variety of causes, but one thing is clear: a major result of congressional nationalization is the decline of the electoral benefits of holding office.

I have to wonder — does this trend present itself in the upper chamber of the United States Congress? That would offer some validity to our findings thus far. Indeed, an analysis of Senate election results since 1952 brings us to the same conclusion.


Nationalization of the US Senate


In keeping with our framework, Adams also had some wisdom to offer about their particularly unique aristocratic legislative body. Regarding the United States Senate, He had a few choice words: articulating with a political vigor our second president said “the rich, the well-born and the able” should have a voice separate from the House; Only then could “the people” be represented absent the diluting influence of an upper social class. Yet Adams also said that “There never was a democracy yet that did not commit suicide,” and James Madison wrote that “Democracies have, in general, been as short in their lives as they have been violent in their deaths” (Federalist 10). It’s clear that Adams wanted a government that reflected a “perfect portrait” of the American public[2] — and the Senate certainly removes them from that — but he also clearly favored a Republic removed from violent popular influence, “if [we] can keep it” (said Adams).

So Adams may not be happy for me to devote significant attention to the US Senate… yet here we are.

Traditional political thought on the Senate says that it is resistant to the often reactionary attitudinal shifts of the American public — what I like to join others in calling the “democratic whims” of the people. Partly because a term in the Senate is three times as long as a that of a Representative — and also because they represent a very large constituency — they are largely resistant to the violent changes in seat share that the House experiences every two years.

However valid this traditional theory is (and I’m skeptical of “folk” democratic theory in modern US politics) we can investigate its relationship to presidential elections over time.


It’s no surprise that we observe the same nationalization of the Senate that we did in the House. The above graphic displays the accuracy of a model using a state’s presidential vote to predict Senate election outcomes compared with a model using vote shares from the previous election.

Not only do we see a huge increase in nationalization, but we also see immense gains in the cycle-to-cycle electoral stability of our upper chamber. Where the House has almost always had a high rate of stability, that is just a recent development in the Senate. Where we do observe congruence with the House trend is in the time frame: nationalization and stability look to have begun their trend upward in 1990 and accelerated rapidly in past decade.

But the Senate does seem to adhere to at least some of the traditional institutional theory we espoused. Compared with the House, it has a weaker relationship to presidential vote margin and a much weaker relationship to the Democrat’s vote margin in the most recent cycle. It is worth noting that while the former are certainly comparable, the latter is likely a product of the longer terms of office.

Predictive Power Of Model (R2)
Chamber Presidential Vote Previous Vote
Predicting US House and Senate Results
House .93 .91
Senate .83 .60

We can square the above findings with the rationale on incumbency advantages we detailed for the House. If fewer people are voting in Senate elections — or if other mechanisms are at play to move the Senate electorate away from its aggregate preference — results will likely have a weaker relationship to presidential vote share (which we believe represent the closest to “true” partisan characteristics of a state).

Given the increasing nationalization of the US Senate, it makes sense that the chamber’s incumbency advantage is experiencing a corresponding decrease.


Notably, the advantage of holding prior office in the Senate is greater than in the House. Although both were separated by just 1 percentage point in 2016, a trend (local regression, or LOWESS) drawn between the points reveals that incumbent Senators get almost twice as large a boost than their Representative counterparts.

Unfortunately for the more exclusive branch of the legislative, they are experiencing a much quicker decrease in the value of office-holding. Over the past 17 years, the value of the Senate’s advantage has decreased by nearly 13 percentage points, where it has only decreased by 5-6% in the House. There are multiple explanations for this discrepancy (recall my remarks on the institution’s separation from the “people”) but the facts remain the same: Just as we saw with the House, the United States Senate is undergoing a period of rapid “nationalization” with a corresponding decline in the incumbency advantage.


The White House Effect


This nationalization is not something to be taken lightly. In the simplest of terms, a rise in partisan voting at the local and state levels means that political parties and party identification are dictating more and more of the public’s voting behavior, and thus the people’s attitudes and political decisions as well. Whether or not you believe parties are a corrupting influence on the ability of the American people to shape public policy, this likely decreases their ability to do so.

That being said, the data do give us a glimmer of hope in the realm of electoral accountability. Using a method similar to the one used by FiveThirtyEight’s Harry Enten we can look at the relationship between shifts in aggregate House/Senate vote share and the approval ratings of the president. Hopefully, we can learn something about the importance of presidential approval ratings.

The first step in this analysis is to predict how any given House/Senate candidate should perform in their election. We can use the estimates from the incumbency advantage regressions used above to do so. That is, we can predict a candidate’s performance using their presidential benchmark plus a point boost corresponding to their status as an incumbent.

After we have a prediction for that candidate, we can subtract their actual performance from the “expectation” and get a residual over/underperformance for each House district and Senate contest. Averaging every candidate’s “residual” for every year gives us an idea of how all of a party’s candidates do in each year, relative to what we would expect they do.

In the below graph I chart these estimated residuals for the incumbent party in each midterm election since 1952. This gives us a very good look at how the White House’s party does at the local and state level: the picture is not good.


On average, a candidate belonging to the incumbent party (Republicans in 2018) does about 8% worse in their margin against the opposition. So if we posit that a hypothetical Republican in 2018 is “expected” to win by 6%, they would (on average) lose by 2%. However, there is a confidence interval around that underperformance, so the office-holding candidate could do better or worse than that eight-point decline indicates. In 2014 (the most recent midterm election) that 80% confidence interval was about 10 percentage points in either direction: The same hypothetical Senator above could win by a margin as large as eight points or lose by up to twelve percentage points.

This is an important finding. It tells us that incumbent parties are punished at the down-ballot level and that they are consistently being punished more than they have been in the past — that’s (presumably) why Harry titles the main graphic in his piece “Wave elections are becoming more common.” This would follow naturally from the work above that suggests politics is much more national and partisan than it was thirty years ago — even ten years ago!

But I am still left wondering, is there some data we can use to predict aggregate underperformance for the party in the White House? This would explain some of the year-to-year variation in a how much the incumbent party misses its mark. The president’s job approval ratings would be a good place to start. Indeed, when we graph underperformance for the White House’s party alongside the president’s approval rating[3] a clear relationship appears.


There is a fairly strong relationship between the popularity of the president and the performance of their party. Using all of the midterm elections since 1952, approval ratings explain roughly 23 percent of the variation. But if we break down those relationships over time (as we should, because we know there are differences over time — recall the previous 2,200 words) we can see that the performance of the incumbent party in midterm elections is getting more tied to the presidency, reaching 96% explanatory power in the midterm elections between 1996 and 2008 and 78% since.


Final Considerations


The nationalization of congressional elections is a rapid trend in American politics that does not show signs of slowing. We live in an environment where party identification, which clearly governs a large amount of our political decision making, is a more powerful electoral indicator than it has ever been before. A district/state’s partisanship explains more variation in election results than even last cycle’s vote share does. As Geoffrey Cohen found in his landmark study “Party Over Policy: The Dominating Impact of Group Influence on Political Beliefs”, when “… information about the position of [respondents] party was available, … participants assumed the positions as their own regardless of the content of the policy.” Partisanship, as the saying goes, is “a hell of a drug.”

I posit that these partisan influences are going to have more power than ever before in the coming midterm electoral cycle. It’s likely that we will see a similar value for incumbency advantage to that which we observed in 2010 when incumbents gained only 1% over non-incumbents in the US House. More importantly, if our findings are held constant, we can guess up to 90% of next cycle’s congressional elections with a high degree of accuracy. And if there is a swing, or “democratic wave,” we can estimate what it would look like: using the data from our “White House Effect” section, a 35% approval rating for President Trump (something he is well on his way towards achieving) would indicate a 14% average shift towards the generic Democratic candidate.

Finally, some considerations. We have looked at a lot of data in this post, and in doing so we have made a lot of choices about how it should be analyzed. All researchers make these decisions, and in doing so often arrive at different inferences. My analysis leads us to observe a fair bit of variation in the value of incumbency advantages than Abramowitz and Webster, Gelman and King (1991), or Jacobson (2015) find. This discrepancy is almost certainly due to the use of democratic vote margin, not Democratic vote share, in our model. But the conclusions we draw are still the same[4].

The local has become the national, Congress has become tied to the presidency, and partisanship influences us all — perhaps more than it ever has before. These influences are not to be taken lightly. Many questions remain, but one sticks out like a sore thumb: will the trend continue? Surely there is an upper limit to the influence of partisanship — but then again, these data suggest there’s no reason to believe that’s the case.

Per usual, only time will tell.

Side note, do you like gifs? Here are four gifs illustrating the evolution of House/Senate predictability by presidential benchmark and lagged vote share over time. The go well with posts on Twitter!




[1]This is a weighted average of the Democrat’s relative margin in the district over the past two presidential elections, giving 75% of the weight to the most recent and 25% to the election 8 years prior.

[2]Or, at least those who were allowed to vote.

[3]As measured by Gallup

[4]As always, this is not the only look at this subject. The importance of national partisanship to congressional elections, reelection rates for incumbents, or the predictiveness of presidential job approval for midterm swings have been studied quite extensively. In that vein of thought, I encourage you to read all of the linked literature in this pieces.

… . .



Thanks for reading today everyone. Tune in to my twitter for updates and make sure you sign up for my newsletter to get notifications of recent posts.