By-elections: how they happen and why they shake governments

Between general elections, the only place real voters cast real ballots for Westminster is the by-election, which is why a contest in one constituency can dominate national politics for a month and shift the behaviour of parties holding hundreds of seats.

The triggers are a short list. An MP dies, or is disqualified, by bankruptcy restrictions, by conviction carrying a sentence above the threshold, or by an expulsion vote. An MP cannot simply resign, a prohibition dating from 1624, so the constitution supplies a theatrical workaround: appointment to a nominal paid Crown office, the Chiltern Hundreds or the Manor of Northstead, which vacates the seat automatically. And since 2015 there is recall: an MP convicted of an offence, suspended from the House for ten sitting days or more, or convicted over expenses, faces a petition, and if ten percent of constituents sign within six weeks, the seat falls vacant. Recall has ended several parliamentary careers already, and its existence now shapes how the Commons disciplines its members, since a suspension of a particular length is, in effect, a sentence to face the voters.

Timing is a tactic in itself. The writ for a by-election is customarily moved by the party that held the seat, usually within three months, and governments choose dates the way defendants choose venues, burying difficult contests on crowded days or delaying them past conference season. Campaigns are short, saturated and disproportionate: national organisers, leaders' visits and activist coachloads descend on a single seat, and the parties treat it as a laboratory for messages and machines ahead of the general election.

Reading the results without being fooled

By-election arithmetic exaggerates by construction. Turnout drops steeply, the governing party's supporters are the likeliest to stay home, and a protest vote is free when no government is being chosen, so oppositions overturn majorities that look impregnable and third parties achieve swings unthinkable nationally. The record books are full of spectacular by-election victors whose seats reverted at the following general election. Psephologists therefore read them with discounts: the swing matters less than the pattern across several contests, and the most predictive detail is often where the losing side's vote went rather than the winner's margin.

Yet the exaggeration is precisely the mechanism of their power. MPs in similar seats extrapolate the swing to their own majority, and leadership crises, policy retreats and pre-election repositioning follow from contests involving a few tens of thousands of voters. A by-election rarely predicts the next parliament accurately. It reliably changes the behaviour of the current one, which is a stranger and arguably larger form of influence.