1 Introduction

Membership ballots (the terms “membership ballot” and “primary” will be used interchangeably in this article) are receiving increasing popularity in democracies across the globe: Studies estimate that between 30% and 50% of parties in Western democracies use primaries to select their leaders (Cross and Blais 2012; Cross et al. 2016; Kenig 2009; Pilet and Cross 2014). A fast-growing body of literature has examined the causes (and consequences) of this development. Several reasons could explain this trend, such as, the assumption that primaries will positively influence a party’s electoral performance (so-called primary bonus hypothesis, e.g., Adams and Merill 2008; Ramiro 2016; but see Pedersen and Schumacher 2015), attract new members (e.g., Scarrow 1999; but see Wauters and Kern 2020), or peacefully settle internal conflicts because of their higher legitimacy (Kemahlioglu et al. 2009).

The aim of this article is to examine the causal mechanisms that explain why parties use primaries. Existing studies (e.g., Astudillo and Detterbeck 2020; Chiru et al. 2015; Kemahlioglu et al. 2009) provide valuable insights into the links between external shocks or intraparty factors and the democratisation of selection rules. However, they often do not show precisely how a driver of intraparty change is connected to the change of selection method. Additionally, there is an astonishing deficit concerning the application of methods based on causal complexity (e.g., process tracing or qualitative comparative analysis) for research into intraparty decision-making and reforms (for a notable exception, see Michels and Borucki 2021). While quantitative studies must translate complex phenomena such as internal conflict or subjective concepts such as electoral defeat into measurable variables—which often requires nonideal operationalisations—in a qualitative study, we can circumvent the difficulties regarding operationalisation by directly listening to the intraparty actors. Also, quantitative studies cannot account for equifinality (i.e., different paths lead to the same outcome) or the complex interactions of multiple processes. Therefore, by using a new, in-depth, within-case approach, this study will be able to provide new insights into intraparty decision-making, for example by teasing out interactions between mechanisms which have not been included in existing theories. I will apply theory-testing process tracing (TTPT) (Beach and Pedersen 2019), which allows for an in-depth and systematic study of the causal mechanisms. Although ex ante theorised mechanisms are tested, the study design remains open for discovering alternative factors and dynamics in the cases under study.

I will use Germany as my example because the shift to more inclusive leadership selection methods is nonpermanent in Germany, and there is back and forth between more exclusive and inclusive selection methods. This means we can study each decision for actually conducting a primary. I will use four case studies from the level of regional party branches (Landesverbände) to test my ex ante theorised causal mechanisms because the shift to more inclusive selection methods has occurred more frequently at the regional than at the national level. The focus of this paper thus is on the intraparty decision-making processes for holding leadership primaries for party chairs and top candidates (subsumed under the term party leader).Footnote 1 I find that different mechanisms have to interact to result in a primary, thus it is often a combination of an electoral shock, internal conflict, and instrumental motives that explains the decision to give the members a say in leadership selection. While in-depth process tracing has the limitation of focusing on only four cases within a single country, these findings may have broader applicability to instances in which primaries are not permanent but are used occasionally. However, generalisation is limited to established parties lacking a strong tradition of grassroots democracy.

2 Literature Review—Why Do Parties “Democratise” Themselves?

For several years, the literature has documented a trend towards the democratisation of party leadership selection. Studies estimate that in Western democracies, 30% to 50% of all parties now grant their rank-and-file members a direct say in the party leadership selection process (Cross and Blais 2012; Cross et al. 2016; Kenig 2009; Pilet and Cross 2014). Three strands of research can be identified in the literature on the democratisation of leadership selection: (1) Some of the work describes the different designs of the selection rules for appointing party leaders (e.g., Kenig 2009). (2) Another subarea addresses the question of why parties “democratise”; i.e., researchers investigate why primary elections are held or introduced into the party statutes (e.g., Cross and Blais 2012; Wauters 2014). These studies are embedded in the larger debate of party change research, which deals with the causes of party change and party reforms (e.g., Barnea and Rahat 2007; Gauja 2017; Harmel and Janda 1994). (3) In addition, the consequences of changed selection rules are examined (e.g., Astudillo and Lago 2020; Kenig 2008; Küppers 2021a; Wauters and Kern 2020). In the following, only the second strand of the research on party primaries will be discussed.

Making use of membership ballots is a form of organisational change, and studies on party organisational reform agree that changes are caused by a combination of external and internal factors (Barnea and Rahat 2007; Cross et al. 2016; Gauja 2017; Harmel and Janda 1994; Panebianco 1988). Although internal factors (such as power shifts) can cause organisational change on their own, external stimuli must be perceived as problematic, and change must be introduced to the party from actors within (Gauja 2017; Harmel and Janda 1994).

There are two types of external factors that can be differentiated: short-term factors located at the level of the party system and long-term factors located at the level of the political system (Barnea and Rahat 2007; Gauja 2017). Cross et al. (2016) have shown that the latter do not trigger reforms but rather explain the direction of reforms (i.e., processes of social change favour reforms towards more participation). Factors at the level of the party system (e.g., electoral defeats) influence the timing of the democratisation. At the level of the party system, electoral defeats or opposition party status are usually identified as external triggers for intraparty reforms (Barnea and Rahat 2007; Chiru et al. 2015; Cross and Blais 2012; Gauja 2017).

One of the main external drivers for party change identified by Panebianco is electoral defeat (Panebianco 1988). Later studies add that there are a variety of external stimuli and that the power of an external stimulus is related to a party’s primary goal (office-, vote-, policy-, or democracy-seeking) (Harmel and Janda 1994). Internal power struggles, as well as instrumental motives of intraparty actors (e.g., bypassing the mid level), are described as internal drivers for reforms (Chiru et al. 2015; Katz and Mair 1995). A third group of studies has highlighted the central importance of internal conflict as a driver for party primaries. Due to the greater legitimacy of the grassroots democratic process, it is believed that primaries can help to overcome internal divisions because the result of the membership vote is more widely accepted (Astudillo and Detterbeck 2020; Kemahlioglu et al. 2009).

However, there are two shortcomings of existing studies. First, quantitative studies have to translate complex phenomena such as internal conflict or subjective concepts such as electoral defeat into measurable variables—which often requires nonideal operationalisations (see Astudillo and Detterbeck 2020 and Kemahlioglu et al. 2009 for examples). One example is the party leadership’s strategic use of membership ballots to circumvent a decision by the party’s midlevel elites. Astudillo and Detterbeck (2020) use the high proportion of party leaders who are defeated in a membership ballot to select a new party leader or a top candidate to falsify the tactical-use claim. The authors, however, disregard the fact that a primary election can also be used by an internal party camp competing with the incumbent party leadership to circumvent a decision by the delegates, as Ignazi (2020) points out. A suitable way to quantitatively measure the hypothesis claiming that primary elections are used to bypass the midlevel elites has yet to be found. Another example is election results. In perceiving an election result as a defeat, the party will not necessarily have lost many votes. Even electoral gains could be viewed as a defeat if the party loses its status as a governing party because a different coalition is formed. In a qualitative study, we can circumvent these difficulties regarding operationalisation by directly listening to the intraparty actors.

The second shortcoming of existing studies is their failure to account for the complexity of intraparty decision-making processes because they cannot account for equifinality (i.e., different paths lead to the same outcome). Qualitative comparative analysis (QCA; see, e.g., Rihoux and Ragin 2009; Schneider and Wagemann 2012), which combines approaches from quantitative and qualitative research, may offer a potential solution. Although QCA is able to take causal complexity into account (Schneider and Wagemann 2012), and the interaction of various factors can be identified, no statements can be made about how these factors interact. That is why a new methodological approach is chosen here in order to answer the question of why primary elections are held: The focus of this paper is on causal complexity and intraparty processes (i.e., causal mechanisms). By choosing process tracing as my methodological approach, I will be able to account for equifinality as well as detect interaction effects between different mechanisms.

3 The Case of Germany

In Germany, the selection of party chairs and top candidates through party conference delegates is still considered the “default mechanism,” and there are only a few cases in which a membership ballot was conducted.Footnote 2 Usually, leadership races in Germany at the national and regional level are noncompetitive—here, party primaries are not required. Moreover, in the case of the selection of party chairs, delegate conferences are even required by law. It is, nevertheless, possible to hold a nonbinding party primary. Its result is later “confirmed” by a party convention. If there are two or more candidates, such a membership ballot becomes a viable option.Footnote 3 It is important to highlight that the change of the selection method towards membership ballots is nonpermanent in Germany.Footnote 4 Often, there is a back and forth between the different selection methods, even for multicandidate races. This means that a party can decide to ballot its members on a case-by-case basis. That is why the German case provides an interesting research opportunity: We can study each intraparty decision for actually conducting a primary.

I limit my analysis to the two most-voted-for parties at the national level: the Social Democratic Party (SPD) and the Christian Democratic Party (CDU). Furthermore, I will focus on the regional level because the shift to more inclusive selection methods has occurred more frequently at the regional than at the national level. In total, one-third of all multicandidate leadership races in the regional branches of the CDU and SPD since 1990 have been decided by party members rather than delegates. There were no instances of leadership primaries in the regional branches of the smaller parties (Greens, Left, Liberals, or Alternative for Germany) until the end of the research period (in 2017). All primaries were so-called closed primaries which means that only formal party members were eligible to vote, in contrast to open primaries, which allow for participation by all voters, or semiopen primaries, in which the selectorate is composed of members and supporters.Footnote 5

4 Methods, Case Selection, and Data

4.1 Process Tracing

With process tracing, we “shift the analytical focus from causes and outcomes to the hypothesized causal mechanism in between” (Beach and Pedersen 2019, p. 1). In TTPT, a causal mechanism is first theorised and then tested empirically step by step using within-case evidence. The aim is to demonstrate that the theorised mechanisms operated in the studied case (George and Bennett 2005). Note, however, that although ex ante theorised mechanisms are tested, the study design remains open for discovering alternative factors and dynamics in the cases under study.

Mechanisms transmit causal forces, whereby each mechanism consists of a “series of interlocking parts that transmit causal forces from C[ause] to O[utcome]” (Beach and Pedersen 2016, p. 35). Each step of the mechanism is understood as consisting of entities engaging in activities (Beach and Pedersen 2016). The activities are what transmits causal force from one part to the next. It is important to note that mechanisms are unobservable. However, they leave behind “fingerprints” (so-called observable implications) that can be traced by using various data collection techniques (interviews, document analysis, etc.). Beach and Pedersen propose a broad definition of what constitutes suitable evidence in process-tracing studies:

“Evidence can be any type of material that might be left by the workings of our theorized causal mechanisms that enables us to say something about whether the relationship was present or not in a case” (Beach and Pedersen 2016, p. 166).

To summarise, TTPT follows three steps: (1) formulation of the expected causal mechanisms, (2) operationalisation of the observable implications, and (3) establishment of the connection between the empirical data obtained in the case study and the previously formulated observable implications (Beach and Pedersen 2019).

In the last two steps, the empirical value of the observable implications or the evidence must be determined. This value depends on the degree of uniqueness and the degree of certainty. Uniqueness and certainty are assessed both at the theoretical level (during the formulation of the expected causal mechanism) and at the level of empirical evaluation (the actual within-case evidence). In the context of uniqueness, evaluation concerns which alternative explanations are possible. To assess certainty, possible reasons for the theoretically possible or actual lack of a “fingerprint” are determined (Beach and Pedersen 2019). In addition, the accuracy of the empirical material is assessed (e.g., whether there are doubts about the authenticity of documents or the truthfulness of statements) (Beach and Pedersen 2016).

4.2 Case Selection

To select appropriate cases to test the ex ante theorised causal mechanisms, a QCA was conducted (Rihoux and Ragin 2009; Schneider and Wagemann 2012). Cases included in the QCA ranged from 1994 to 2017, whereby 1994 was the year of the first ever occurrence of a party primary to select a party leader at the regional level. The truth table can be found in the supplementary material (for more information, see Küppers 2021b). Based on the QCA results, two “typical” cases (SPD Lower Saxony 2011 and CDU Baden-Wuerttemberg 2014) and two “deviant cases” (CDU North Rhine-Westphalia and SPD Bremen 2016) were chosen. Deviant case coverage shows the outcome (i.e., a membership ballot took place) but cannot tell us anything about why this outcome occurred. These cases remain unexplained by the QCA solution formula.Footnote 6 Therefore, the primary is likely explained by previously unaccounted factors (for details, see Oana et al. 2021, p. 182–184). Typical cases in QCA are explained by the QCA solution formula and the outcome is present (i.e., a membership ballot took place). As can be seen in Table S. 4 in the supplementary material, the cases of SPD Lower Saxony 2011 and CDU Baden-Wuerttemberg 2014 are explained by the QCA solution formula (the upper six rows of the table include the cases that are explained by the QCA), whereas cases CDU North Rhine-Westphalia 2010 and SPD Bremen 2016 are not covered by the QCA solution formula (the QCA solution formula can be found in Fig. S. 1 in the supplementary material).

4.3 Data and Materials

To reconstruct the processes leading to primaries, this study draws on a variety of data sources. A document analysis of around 100 party documents and more than 300 newspaper articles was conducted. The newspapers analysed were Frankfurter Allgemeine Zeitung and Süddeutsche Zeitung for the federal level as well as one to three regional newspapers each. The main source of information, however, was 40 qualitative interviews with the relevant intraparty actors involved in the decision-making processes (8–11 interviews were conducted per case). Interviewees were selected based on “positional criteria” and “reputational criteria” (Tansey 2009). According to positional criteria, members of the core party executive (chair, vice chair, secretary-general, leader of the parliamentary party group), as well as leading party staffers (so-called Landesgeschäftsführer), and candidates were interviewed. In regional party branches with strong districts, district leaders (Bezirksvorsitzende) were also interviewed. Following reputational criteria, additional interviewees were selected who were “deemed influential by their own peers” (Tansey 2009, p. 493). This selection included, for example, members of ad hoc reform committees or supporters of certain candidates. In one case, I had the opportunity to interview both candidates involved in the leadership selection process. In two cases, I interviewed only the winning candidate. In the remaining case, I conducted an interview with the losing candidate. To ensure balanced representation, I conducted interviews with supporters of the other candidate that I could not interview in each case. All materials were analysed using MAXQDA software (VERBI Software, Berlin, Germany). More detail on the analysed material is provided in the supplementary online materials.

Because of the temporal distance from the events (up to ten years), there were sometimes memory gaps or factually incorrect reconstruction of events. Moreover, when questioning different members of the party elite, we might have encountered the so-called Rashomon problem, in which “different participants in the process have different views as to what took place” (George and Bennett 2005, p. 103). On several occasions there is reason to assume that ex post rationalisation occurred, meaning that individuals may have retrospectively portrayed their behaviour in the decision-making process as more strategic than it actually was. Furthermore, given professional dependencies (party staffers) or further career ambitions, some interviewees may have had an interest in withholding information. Other actors might have acted this way in order to protect their own legacy after leaving office (Rathbun 2008).

5 Theoretical Expectations: Three Ideal–Typical Primary Mechanisms

Building on existing scholarship, it is possible to derive three causal mechanisms that were tested in this study. For TTPT, it is necessary to formulate the causal mechanism and their observable implications (also known as the expected “fingerprints”) in advance. The full mechanisms with all theorised observable manifestations can be found in the online supplementary material.

5.1 Mechanism 1: Primaries to Circumvent Delegates

According to Katz and Mair (1995), primaries can be used instrumentally to strengthen the party leadership by depriving midlevel elites of some of their power. This means that the party leadership will decide to hold an intraparty election only if it benefits from this rule change. This mechanism is triggered by an internal power struggle. For the first step of the mechanism, it would be expected that one of the candidates would assume they would not have a majority at a party congress; at the same time, a majority at the party base is expected (see mechanism 1 in Table 1). We do not have to measure actual resistance among the delegates for the mechanism to be triggered. It is sufficient that the candidate thinks that the delegates will not support their decision.

Table 1 Expected causal mechanisms leading to membership ballot
Full size table

In either case, only one side—the one that feels a primary would be to its advantage—will actively lobby for a membership ballot. Possible fingerprints left behind by this second step of the mechanism are, e.g., that demands to hold the primary in the media or at party meetings can largely be attributed to one side only. Also, we might hear explicit “confessions” of a tactical use in the interviews.

The candidate and their supporters who believe they will have a smaller likelihood of success in the primary will prefer a delegate convention in which a majority can be more easily secured. They might not openly object to the primary (because it is hard to justify that one is against more democracy) but will not lobby for it, either. Possible fingerprints are (secretive) efforts by the disadvantaged side to prevent the primary (we could, for instance, see long internal debates or a nonunanimous decision to hold the primary).

5.2 Mechanism 2: Peaceful Conflict Resolution

Party primaries could also be used as a means to settle an internal conflict. When the party leadership is divided and, therefore, cannot agree on a single candidate, it is hoped that, through the stronger democratic legitimacy associated with membership ballots, it can be ensured that the losing factions will not refrain from supporting the winner of the primary (Kemahlioglu et al. 2009).

If the membership ballot is used to solve an internal conflict, we should expect the following mechanism: Intraparty conflict may manifest in frequent changes at the top of the party. Moreover, parts of the party may fail to support the party leader, which would be reflected in poor election results (less than 80%) for the party leader in internal leadership elections (Astudillo and Detterbeck 2020). Other pieces of evidence would be media reports about the conflict.

The party leadership perceives an internal conflict about the selection of the top position as damaging to the party (step 1). The primary is conducted in the belief that through the perceived greater legitimacy of its results, it will help to settle the conflict (Kemahlioglu et al. 2009)—this forms step 2 of the theorised mechanism. The greater legitimacy of the primary’s results will then, for example, be stressed in the interviews.

In this case, it is likely that the party elite is especially concerned about guaranteeing a fair procedure (step 3), as any doubts about the legitimacy of the result are damaging for peaceful conflict settlement. Possible observable implications consist of the party board cautiously evaluating the fairness of different procedural measures, as well as lengthy debates about the technical aspects of the selection procedure. Another indicator could be the party seeking legal advice.

5.3 Mechanism 3: Renewal After Electoral Defeat

Panebianco wrote that “electoral defeat and deterioration are pressures leading to organizational change” (Panebianco 1988, p. 243). By conducting (or introducing) party primaries after an electoral defeat, the party wants to regain the electorate’s support by presenting a new, more democratic image.Footnote 7 Empirically, several studies have found a link between electoral defeat and/or the party being in opposition and a change of the selection rules (Chiru et al. 2015; Cross and Blais 2012; Pilet and Cross 2014).Footnote 8

According to this mechanism, the membership ballot results from an electoral defeat. Electoral results are, however, subjective. To give some examples: In perceiving an election outcome as a defeat, the party will not necessarily have lost many votes. Even electoral gains could be viewed as a defeat if the party did not achieve its goal, e.g., winning the office of prime minister or, if a coalition was formed, not being part of it. For office-seeking parties, an election will be perceived as a defeat when the party loses the office of prime minister. That is why Harmel and Janda advise “to listen to the party itself [rather] than to attempt to indirectly assess when such shocks have occurred” (1994, p. 269). Possible within-case evidence would be statements by party officials on election night (or during the following days) framing the outcome of the election as a defeat. Evidence would also be party officials framing the election result as a defeat in the elite interviews.

Consequently, the party elite will trigger an internal dialogue process to discuss the election result (step 1). These debates can occur in meetings of the party board or at more open forums to which party members are also invited. Including members in these debates can be a step to enhance the legitimacy of a reform proposal (Gauja 2017). During these broader discussions, demands for a more democratic party might be voiced by the party’s base (local branches, collateral/youth organisations, etc.). Cross and Blais (2012) highlight the importance of demands from below for a change of leadership selection rules in the British case.

After listening to its midlevel elites or its rank and file, the party leadership recognises a more democratic party as necessary for a renewal of the party image. Observable implications would be if the party elite declares that “something” has to change (step 2).

Thus, a reform process is triggered (step 3). Observable implications of a reform process are ad hoc commissions introduced with the goal of drafting proposals for organisational or policy reform. Other actors from within the party elite could also draft their own reform proposals.Footnote 9

The theorised mechanisms are not contradictory. Multiple mechanisms might work at the same time within the same party branch. Alternatively, there might be an interaction effect between different mechanisms, which makes the primary possible in the first place.

6 Empirical Results

6.1 Case Study: CDU North Rhine-Westphalia 2010

The case of the CDU North Rhine-Westphalia emerged as a deviant case during the QCA. Hence, it is expected that the in-depth case analysis will uncover the workings of a causal mechanism that was not captured by the QCA solution formula (see supplementary materials for more information on the QCA). Following their 2005 election victory, the Christian Democrats formed a coalition with the Liberals, ending 39 years of Social Democratic rule in the western German region of North Rhine-Westphalia. Five years later, however, the CDU-led government of Prime Minister Jürgen Rüttgers suffered a painful defeat, with a loss of more than ten percentage points and the ousting from government party status (Feist and Hoffmann 2010).Footnote 10 After the defeated prime minister resigned from his position as party chair, there were three potential candidates for his succession: the regional Integration Minister Armin Laschet, the then Federal Minister for the Environment Norbert Röttgen, and Andreas Krautscheid, the regional party branch’s secretary-general. Krautscheid later declared that he would not run. Thus, the primary in October 2010 was a race between two candidates: Laschet and Röttgen.Footnote 11 The latter won the membership ballot with 55% of the vote. As shown in Fig. 1, the membership ballot was caused by instrumental motives to circumvent a decision by the convention delegates. The electoral defeat made the party elite more susceptible to the idea of giving its rank and file a say, and, therefore, is an important contextual condition. The within-case evidence, however, shows no direct causal link between the poor electoral showing and the leadership primary.

Fig. 1
figure 1

Christian Democratic Party North Rhine-Westphalia: schematic summary of causal process leading to membership ballot

Full size image

In retrospect, the election outcome was seen as a severe defeat by all relevant actors (I-21, I‑23) but three things are important to note: First, in 2010, it took the party several weeks to realise the full scale of its defeat. Lacking a majority for a two-party coalition (which was the standard coalition format at that time), government formation was complicated and lasted for several weeks. During that time, the Christian Democrats could still hope to achieve their office-seeking goal. Thus, there was no need for internal renewal (I-21). Second, the then still incumbent party chair viewed the election outcome not as the party leadership’s fault but as if the party was a victim of external forces—namely, Chancellor Merkel’s decision to bail out Greece (interview with Rüttgers). Consequently, he saw no need for change. A third explanation for the absence of a renewal process is that the SPD and Greens formed a minority government. Such governments, however, are an exception in Germany, and the SPD-led minority government was considered unstable (Wittke 2010; I‑35). That is why the sword of Damocles of a snap election was always present, and the regional party did not want to risk a year-long reform process.

To summarise: The decision to hold a primary was, therefore, not embedded in a renewal process (with broad internal debates, ad hoc reform commissions, etc.). Some members of the party executive, nevertheless, concluded from the defeat that the party needed to democratise its leadership selection process:

“But even without this previous experience [the leadership primary in 1994, the author], I believe, in any case, that a delegate convention was not enough. But now, after such a defeat, the rank-and-file should help with decision-making.” (I-32; similarly: I‑21; I‑41)

Thus, the election result had made the party elite more susceptible to the idea of giving its rank and file a say. That is why Nobert Röttgen and his supporters could easily succeed with their instrumentally motivated demands for a membership ballot.

After the defeated prime minister announced his demission from the office of party chair as well, there was a power vacuum in which three (later only two) candidates competed for his succession: Andreas Krautscheid, Armin Laschet, and Norbert Röttgen. The vast majority of district chairs (Bezirksvorsitzende), as well as heads of the local party branches, supported either Laschet or the secretary-general Krautscheid (e.g., Bröcker 2010a). Thus, Röttgen would have most likely lost in a decision by the party’s delegates. That also explains why the outgoing party leadership (Rüttgers/Krautscheid) initially planned a different selection procedure: so-called regional conferences followed by a delegates’ vote at a party convention (Bröcker 2010b; CDU Nordrhein-Westfalen 2010). This procedure would have benefitted Krautscheid (or Laschet).

It was the supporters of Röttgen who—before and during the decisive executive board meeting—successfully campaigned for a membership ballot to be carried out (I-38; I‑48). Röttgen’s supporters launched two calls for a primary from “below.” In both cases, the demand for a membership ballot was framed as a means to democratise the party and create a new spirit of optimism from within the party (see MIT NRW 2010; Aufruf “Kreisvorsitzende für Mitgliederbefragung” 2010). However, one of the signatories of the local branches’ call for a primary confessed that their initiative was intended to assist the federal minister (I-41). Because Röttgen was a good speaker and was publicly well known due to his national executive office, his supporters assumed that he could win a primary (I-38), whereas Laschet, who had spent much of his recent political life at the regional level, was believed to have a better network among the convention delegates (I-31; I‑41). However, this view of a tactical use of the primary to circumvent a delegate decision may be the product of an ex post rationalisation (I-35). That is, only retrospectively did Röttgen’s supporters construct the success story of the primary that had helped their favourite candidate become party chair.

In the last step of the ex ante theorised tactical use mechanism, it was assumed that the candidate whose chances to win the primary would be smaller would try to prevent the membership ballot from happening. This last step is especially hard to trace because the opponents—for strategic reasons—might not admit to being against the primary: It is harmful to be against more democracy (I-14; I‑21; I‑48). There is some evidence that supporters of Krautscheid (the third candidate who later decided not to run in the membership ballot), namely the outgoing party chair Rüttgers and the party’s youth wing, tried to prevent the primary (Rheinische Post 2010; Spiegel Online 2010). However, the evidence for this is weak, e.g., because it cannot be triangulated with the meeting minutes, which provide an anonymous summary of the discussion only (CDU Nordrhein-Westfalen 2010). Also, the interviews suggest that proponents and opponents of a membership ballot existed in both camps (I-21; I‑41). Thus, the overserved causal mechanism follows the logic of the ex ante theorised mechanism to circumvent the party’s delegates (Table 1). The empirical evaluation of the mechanism’s traces can be found in the supplementary material (Table S. 7). Because tactical considerations could not be operationalised as explanatory factors in the QCA, it becomes clear why the CDU North Rhine-Westphalia emerged as deviant case coverage in the QCA.

6.2 Case Study: SPD Lower Saxony 2011

In the run-up to the 2013 regional elections, in which the SPD in Lower Saxony succeeded in regaining government responsibility, the party nominated its top candidate in a membership ballot. The then mayor of Hanover, Stephan Weil, won the primary against the incumbent party chair, Olaf Lies, with 53% of the votes. The SPD won the election 1.5 years later, and Weil became the regional prime minister. The decision to conduct a membership ballot was motivated by a combination of an electoral shock, tactical motives, and a desire for a peaceful conflict settlement (Fig. 2).

Fig. 2
figure 2

Social Democratic Party Lower Saxony: schematic summary of causal process(es) leading to membership ballot

Full size image

As can be seen in Fig. 2, the decision of the SPD branch in Lower Saxony to hold a primary to select its top candidate for the regional election in 2013 can be ultimately traced back to its poor election result in 2008. That year, the party suffered its worst ever election result, with 30.3% of the vote (2003: 33.4%; 1998: 47.9%) (Klecha 2016). Following the disappointing election result (the party did not achieve its office-seeking goal), a renewal mechanism can explain why the primary was written into the party’s statute in 2010 but not why a membership ballot was actually conducted in 2011.

After its 2008 electoral result forced the party to remain on the opposition bench, an investigation into the reasons for its poor electoral showing was launched. Next to the so-called future commission (Zukunftskommission; an ad hoc commission charged with investigating the party’s poor electoral performance), the mayor of Hanover and the head of the SPD-Landesgruppe (the group of SPD Bundestag members from Lower Saxony) as well as the Braunschweig district drafted their own analyses and reform proposals (Oppermann and Weil 2008; SPD Braunschweig 2008). In this process, the SPD in Lower Saxony identified its organisational structure as one of the reasons for its electoral weakness. As a result, a controversial debate between the powerful districts started about how the organisational structure should be reformed. As part of this debate, the Braunschweig district proposed the idea of opening up and democratising the party structures (SPD Braunschweig 2008). The “future commission” also recommended the introduction of a primary to select the party’s top candidate (Zukunftskommission 2008). However, after it first appeared in 2008, the issue of the selection method vanished from the party’s agenda, only to reemerge in early 2010 after the party chair (and likely top candidate for 2013) Garrelt Duin announced his resignation.

As a consequence, the party elite made two decisions: (1) to have a membership ballot in case there would be more than one candidate for the position of top candidate, and (2) to involve rank-and-file members in the selection of the new party chair (which was the first decision to be taken). The two most promising candidates for party chair were Stefan Schostok and Olaf Lies. However, both had been elected as newcomers to the regional parliament only 2 years previously. Their political seniority was not regarded as sufficient to become prime minister and beat the CDU incumbent (I-10; I‑44). Therefore, the decision to hold a membership ballot for the office of top candidate (the so-called Achim resolution) ensured that the selection of the new party chair would not predetermine the nomination of the top candidate—it is an informal rule that the party chair enjoys privileged access (so-called Erstzugriffsrecht) to the position of top candidate (Berger 2010). The overall result of the reform process was that the party congress in the summer of 2010 decided to add the primary for the office of top candidate to the party’s statute—a decision motivated by a combination of renewal and tactical motives (see Fig. 2).

The German parties usually prefer the “coronation” of a single candidate by the party elite. Thus, in order to explain why the SPD Lower Saxony actually held a primary, it has to be shown why the agreement on a single candidate—who was preferred by the party elite (I-10; I‑37; I‑44)—failed.

Lies, the winner of the membership ballot for the new party chair, would have had privileged access to the position of top candidate (I-34). However, large parts of the party elite did not regard him suitable. As party chair, he had travelled through the local district associations (he visited approximately 200 local party branches) and became convinced that he would have good chances of winning a membership ballot. In contrast, he would not be able to win the informal nomination from the party elite (or at a delegate conference). The reason for this was his lack of political seniority (I-10; I‑44; interview with Lies; Seng 2011). The prospect of having a leadership primary motivated Olaf Lies to pursue his candidacy, against all attempts by his party colleagues to convince him not to do so. There is weak evidence that Weil (the other candidate and the party board’s and district chair’s favourite) did not want the primary (e.g., I‑45; Seng 2011).

As it became clear that neither Weil nor Lies would renounce their ambition to run for top candidate, the party’s executive board reached the conclusion that a membership ballot was the only way to settle the leadership dispute peacefully: “[I]t could also tear a party apart to decide this at a party congress, so we give the members the say” (I-34, similarly I‑44). The higher democratic legitimacy of the primary is a frequently cited reason for this: “[T]he party elite was aware that the level of discord had become so great that their own legitimacy would not have been sufficient for such an important decision” (e.g., I‑27). Also, the acceptance of the primary’s result was regarded as higher (I-34; I‑37; I‑45). This follows the logic theorised ex ante in Table 1. Also, as theorised, the SPD in Lower Saxony was cautious about providing a fair procedure (I-16; I‑44). However, there are alternative explanations for this caution (I-12; I‑19). For the empirical evaluation of the mechanism’s traces, see the supplementary material (Table S. 8).

6.3 Case Study: CDU Baden-Wuerttemberg 2014

The primary to select the top candidate was conducted in late 2014 and saw two candidates: Thomas Strobl, the party chair, and Guido Wolf, who was a district chief executive in the Tuttlingen district (Landrat). Wolf won the primary but lost the regional election in 2016. As depicted in Fig. 3, the decision to ballot the party’s members was caused by the interaction of a renewal mechanism and tactical motives to create a window of opportunity for suitable candidates for the position of top candidate.

Fig. 3
figure 3

Christian Democratic Party Baden-Wuerttemberg: schematic summary of causal process leading to membership ballot

Full size image

The shock caused by the severe electoral defeat in 2011 (the CDU lost the office of prime minister after almost 60 years) led to a renewal mechanism. There were discussions involving the party’s rank and file (most notably, a conference with 1000 rank-and-file members in April 2011). Also, the defeated prime minister immediately resigned as party chair. Moreover, the party launched the so-called future workshop (Zukunftswerkstatt), inviting (organisational) reform proposals from below. The decision to select the top candidate for the 2016 regional election in a membership ballot can be partially linked to this renewal process, as it was repeated in the Zukunftswerkstatt’s motion for the party convention in 2012. There were, however, other motives as well.

There was a strong desire for a personal renewal at the party’s base. After the electoral defeat, there was discontent with the (at that point) only candidate for the office of party chair, Thomas Strobl, who was blamed for the electoral outcome (e.g., Müller 2011). The party’s base was also dissatisfied with how Strobl should become head of the party branch: without a real choice (I-17; I‑40; I‑49). This feeling of discontent with the overall process and the lack of alternatives can explain the initiative by the Tuttlingen local branch and the Südbaden district to demand a primary to select the party chair. Thus, demands for democratisation were also motivated by instrumental concerns: to prevent Strobl as party chair (CDU Tuttlingen 2011; I‑29). This initiative for a primary from below was then blocked by the party board at its meeting in June (Stuttgarter Zeitung 2011). Instead, it was transformed into a primary to select the top candidate (several years later). What caused this interesting twist?

Neither the outgoing party chair Mappus (because he wanted to leave the office as soon as possible) nor the secretary-general Strobl (he feared losing) wanted a lengthy membership ballot to select a party chair (I-15; I‑49). However, after the loss of government, and facing the severe discontent of the party’s rank and file, the party executive could not simply ignore the demands from below. For other parts of the party elite, this decision was motivated by the desire to prevent Strobl from automatically becoming top candidate (an informal privilege of the party chair). According to the proponents of this tactical use of the primary, the membership ballot was, thus, used to open a “room of opportunities” (I-49) to find an alternative top candidate for the 2016 election. Again, other members of the party board supported the primary because they did not see a “natural” top candidate for the next regional election—a situation they had never experienced before, since, usually, the prime minister was their top candidate (I-22; I‑28).

As in the case of the SPD in Lower Saxony, the “coronation” of a single candidate would have been preferred in 2014. This, however, failed because the prospect of a membership ballot motivated Guido Wolf to challenge Strobl, as he could hope to win the members’ approval (interview with Wolf).

Parts of the party later blamed Wolf for the poor electoral result. This—and an already bad experience with a primary in 2004 that had deepened existing divisions within the party—led to the belief of several high-ranking actors that primaries are a “poisonous” instrument never to be used again.

6.4 Case Study: SPD Bremen 2016

None of the ex ante theorised causal mechanisms can explain why the Social Democrats in Bremen conducted the fourth primary in their party’s history to select a new party chair in 2016. Instead, what we see in Bremen is that membership ballots have already become a tradition or “almost a normal process” (I-30; similarly, I‑24; I‑18). This means that a primary will always be the leadership selection method unless there is a “natural” candidate. Because of its higher legitimacy (I-30; I‑42) and positive experiences with its previous membership ballots (e.g., I‑13; I‑18; I‑24), this selection method has replaced the delegate convention as the new “default method” in the event of more than one candidate.

7 Discussion and Conclusion

Several attempts have been made to explain the introduction and use of party primaries (e.g., Astudillo and Detterbeck 2020; Cross and Blais 2012; Kemahlioglu et al. 2009). This research has provided some valuable insights. However, existing studies failed to operationalise subjective factors (e.g., intraparty conflict or electoral defeat) in a satisfactory way and to account for causal complexity. Therefore, in this article, I reexamined the role of factors identified in previous work: electoral defeat, intraparty conflict, and tactical considerations in explaining membership ballots by using a novel, mechanism-centred approach to gain new insights.

In the case of the CDU North Rhine-Westphalia, electoral defeat was an important contextual condition that made the party elites susceptible to giving the rank and file a say. However, the causal mechanism explaining the membership ballot resembles the ex ante theorised ideal–typical mechanism to circumvent the conference delegates. In the case of the SPD in Lower Saxony, the electoral defeat had an enabling function. The primary can be causally explained by a combination of tactical motives and the demand to solve an internal conflict peacefully. In the case of the CDU Baden-Wuerttemberg, demands for organisational and personal renewal interacted with instrumental motives to create a window of opportunity for future (and, hopefully, more suitable) candidates for the office of top candidate (a similar process could be witnessed regarding the SPD Lower Saxony). In the case of the SPD in Bremen, a previously not specified mechanism was detected whereby past experiences with the democratic selection method turned membership ballots into the new “normal.” However, our case study of Baden-Wuerttemberg suggests that next to a contagion effect (see, e.g., Cross and Blais 2012), there seem to exist instances of deterrence effects as well. This offers a potential explanation of why primaries might not become standard (nonoptional) under some circumstances.

To summarise: Usually, different mechanisms interact to result in a primary. This novel finding shows the merit of using an approach suitable for dealing with causal complexity. Moreover, my study shows that electoral defeat is an important motive for change and thereby confirms insights from the existing literature, e.g., Cross and Blais (2012), Gauja (2017), or Panebianco (1988). However, electoral defeat is linked to the membership ballot in different ways—sometimes via a renewal mechanism, or on other occasions as a contextual condition. Via a mechanism-centred approach, we could thus enhance our understanding of the causal mechanisms linking electoral defeat to a change of the selection rules. In two cases, demands from below decisively influenced the decision to hold a primary, confirming earlier findings by Cross and Blais (2012). Finally, the qualitative approach reveals that tactical considerations seem to play a more significant role than suggested by previous research (e.g., Astudillo and Detterbeck 2020).

In the Spanish context, both major parties started to use primaries more frequently at the regional level in order to create a more democratic image. This was not triggered by an electoral defeat, but can rather be seen as a strategic response to the rise of successful challenger parties such as Podemos that had a more open and democratic organisational structure (Barberà and Rodríguez-Teruel 2021; Debus and Navarrete 2020). The emergence of these challenger parties posed the danger of drawing support away from the established parties. Because democratisation in this case can be attributed to the parties’ anticipation of a future decline in their vote share, the emergence of more democratic challenger parties could be integrated as a cause into the theorised ideal–typical renewal mechanism in further tests of the theory.Footnote 12

Using in-depth process tracing, however, comes with the limitation of focussing only on four cases within a single country. Nevertheless, the findings could possibly be generalised to all instances in which primaries are not made permanent but in which democratisation is used occasionally. Moreover, only catch-all parties were studied, which means the results might not be transferable to newer parties or antiestablishment parties that could choose to adopt membership ballots as their candidate selection mechanism in order to differentiate themselves from the “established” parties and to present a more open and transparent approach to politics (see, e.g., the Spanish case, Barberà and Rodríguez-Teruel 2021).

Although the conflict resolution mechanism stipulates that primaries are used to peacefully end an internal conflict, this does not imply that holding a primary will actually do so (for more on this subject, see Barberà and Rodríguez-Teruel 2021). Furthermore, when a primary is collectively remembered as having caused divisions (e.g., CDU Baden-Wuerttemberg 2004), primaries cease to be seen as a means to settle internal conflicts. The case of Baden-Wuerttemberg additionally points to another important phenomenon: Primaries can serve as “critical turning points” (Barberà and Rodríguez-Teruel 2021, p. 48) for future organisational development. If the initial membership ballot is remembered for deepening existing conflicts or creating new ones, this can delay or even halt the use of future primaries, the Spanish case being a further example (Barberà and Rodríguez-Teruel 2021; Debus and Navarrete 2020). In contrast, my case study on the SPD in Bremen points in the opposite direction and demonstrates how positive experiences with primaries can become a critical juncture in establishing this inclusive selection method as the new default mechanism.

Another potential limitation of this study is that it cannot explain the choice of open primaries over closed primaries. This could, however, be a fruitful area for future research. It is easily imaginable how a mechanism to use the primary in order to circumvent a decision by the midlevel elites could lead to an open primary instead of a closed one. This scenario might arise if a candidate or their supporters believe that the candidate, would perform better among the general population than in a vote by all members. Likewise, the desire for renewal following an electoral defeat could also motivate the choice for an open primary over a closed one. This decision may stem from the party’s perception that its members are not representative of the wider electorate (for example, if men or older individuals are overrepresented among its members) or if it believes that its members are ideologically more extreme than its voters (as stipulated by May’s law; see May 1973). These perceptions may lead to the conclusion that the party members are not capable of selecting a candidate who resonates well with the broader population and with whom the party can win elections. I would, however, not expect these demands to be raised by the rank and file (as theorised in my ex ante mechanism), as open primaries devalue party membership. For a similar reason, it appears questionable whether an open primary would be perceived as a way to peacefully end an internal conflict. Dues-paying long-term party members might not be inclined to attribute more legitimacy to a choice made by supporters or voters not closely affiliated with the party.

Last but not least, this study has shown that the combination of QCA and process tracing seems particularly useful for future investigations of intraparty decision-making. A larger number of cases and thus greater generalisability can be achieved via QCA. At the same time, process tracing can reveal the interaction of different drivers (in typical cases) or the breaking off of mechanisms (in deviating cases of the “consistency” type).