I have relatives who work for 3M Corp, and what they tell me is that 3M has a very cruel and absurd policy toward its employees: The lowest-performing 10% of workers is laid off every year, forcing the remainder to be in a constant state of job insecurity that (apparently successfully) forces better job performance. While this is a twisted policy to apply to people just trying to make a living and look out for their families, I think the general principle could potentially be very useful - not to mention morally justified - to apply something similar to elected leadership in the Democratic Party.
At the moment, decisions about whether to primary a leader are very ad hoc, and based on an intersection of jilted single-issue constituencies combined with the availability of talented upstart candidates to issue the challenge. As a result, credible primary challenges against incumbents are unusual, and successful ones even rarer despite what most of us can agree is the pervasive under-performance of our Party in delivering progressive change.
We can and should systematize the process, and I think doing so could deliver not only more contested primaries, but produce a consistent and sustained sense of job insecurity among elected Democrats from the base: One far more effective than the threat of mere lack of turnout, since losing a general election still keeps them juiced-in with the Party, while losing a primary drastically undercuts their political relevance for future campaigns.
Another advantage of systematic, percentage-based primary challenges is that it basically grades on a curve so that allowances are made for political reality rather than attempting to quixotically apply ideals in ignorance. So, for instance, if 90% of our leaders get something right and 10% get it wrong, then the problem is with those 10% and primarying them will have tangible benefits. But if 90% get it wrong, then there is something wrong with the overall context in which they operate, and just changing the roster would have limited benefits. However, going after 10% of the lowest-performers every single cycle would, eventually, create a sea change in the culture that could ultimately address even systemic problems.
Now, why 10%? Why not 20%, or 50%? Quite simply, because progress requires a stable foundation at any given moment, and even if by some miracle a large proportion of the Party's elected leadership were replaced in one fell swoop, the result would be organizational chaos that would be swiftly and effectively exploited by the Republicans. Those competent insiders who survived such a purge would, rather than helping to stabilize the situation, back off and allow the naive upstarts to destroy themselves so that they could later move into the vacuum and restore the status quo.
But if only 10% or so are systematically and repeatedly targeted, you remove the least productive (or most counter-productive) element, effectively warn those who survive the challenges and those who are adjacent to the challenged bloc, and leave the remainder more mindful of what their job is and grateful to have it. This is especially true if positive changes in attitude on the part of the survivors of challenges are rewarded rather than treated as more evidence of their unworthiness, and also if intransigence is guaranteed to face perpetual challenge every single cycle until it changes or until they're removed.
Such a program would not preclude a challenge outside of the 10% bloc, since there are other reasons to primary a leader - e.g., if some scandal or other thing compromises their ability to win a general election, but they insist on running anyway. But it does create a focused emphasis and pool of resources for dismissing those leaders who drag down or obstruct the progressive agenda in office, placing resources where they are most needed for the overall improvement of the Party rather than just better representing any given district.
That said, it is important to recognize that calculating performance is no trivial thing: It must be weighted according to the constituency. A centrist in a profoundly liberal district may be considered a more egregious under-performer than a center-right Democrat in a center-right district, but at the same time failing to be tangibly more progressive than one's constituency at all can be considered under-performance to whatever extent applies. The job of a leader is to lead, not just passively reflect in exchange for a paycheck and perks.
So the ideal that leaders must be convinced to pursue is not to perfectly reflect their constituencies, since nothing ever changes for the better that way, nor should we demand that they be so far ahead of the people they represent that their likelihood of winning elections is too badly compromised. Rather, the ideal should be to guide their constituencies toward better politics than they currently have, taking into account where they are right now.
The 10% of leaders who best exhibit this standard should be rewarded in ways substantially over and above the middle bloc who are neither challenged nor rewarded. I'm thinking this could take the form of a committed, prefab bloc of organized volunteers and donor bases similar to (if not actually) OFA that would put their reelection as a top priority. We would say to these leaders, "You get money and volunteers before anyone else. We will slap down any attempt by other Democrats to challenge you, and steamroll your Republican opponents." The middle bloc who only get second-tier priority for funding and volunteers, and may face maverick primary challenges even if they're not in the lowest 10%, should be made to aspire to the rewarded bloc and be shown exactly how to compete for a spot in it.
Of course the standards for calculating performance are subjective, but once consensus is achieved on what they are, rigorously and quantitatively applying them would be useful. However, it must be stated that the calculation cannot simply be based on interest groups' issue ratings: It would do little good to replace a bunch of centrist Democrats who pass moderate legislation with a bunch of impotent rhetoricians who never pass anything at all while making nice-sounding speeches about progressive ideals.
Performance must be strongly weighted in favor of actual achievement, so quixotics who never do anything more than talk the talk should either figure out how to accomplish something or make way for people who will. Being right and passing legislation are not mutually exclusive, and officeholders who think they are do not help us. Still, the overwhelming majority of those targeted for primaries under a 10% doctrine would be conservatives and centrists, and to the extent any progressives ended up being targeted their challengers would likely also be progressive - only more focused on applying their principles rather than just pontificating on them.
Such a program could specifically involve a dedicated fund that would automatically go to such challenges every cycle, and perhaps also draw from a "draft pool" of potential primary challengers who could bring an A game to a primary election. So, if a given office-holder is in the lowest 10% of performers based on their aggregate performance since their previous election, the process is automatically triggered at some appropriate point with funds going into the organization of a challenge and potential contenders sought out from the draft pool. A contender who is settled on and decides to run would have a campaign apparatus already organized for them, just as the incumbent does, rather than having to create one from scratch while the machinery of the Party works effortlessly for an incumbent that undermines it. If multiple credible people want to issue the challenge, their own performance ratings could be used quickly and decisively to choose the best.
So through a rigorous, systematized program of primary challenges, complemented with an equally systematized program of rewards for progressive leadership, we can in fact change both the Party and thereby the government on a sustained basis.
8:25 PM PT: Perhaps there could be a more subtle approach in some cases: Give leaders in the worst 10% an ultimatum: Run from a different constituency where their shortcomings are net advantages over whoever currently holds the seat (be it a Republican or an even worse Bad Dem), or else be primaried.