Why Stuart Stevens Wants to Defeat Donald Trump

Many Republicans have joined the Never Trump camp, but few have previously been more important to the Party than Stuart Stevens. After working on George W. Bush’s two Presidential campaigns and serving as a consultant for several major congressional candidates, Stevens was Mitt Romney’s top strategist in 2012. He began speaking out against Donald Trump in 2016 and today is an adviser to the Lincoln Project, a political-action committee formed by current and former Republicans to prevent Trump’s reëlection. In Stevens’s new book, “It Was All a Lie: How the Republican Party Became Donald Trump,” he tells the story of his long career in politics and how the party he once worked for has been subsumed by nativism, bigotry, and cruelty. But Stevens doesn’t believe that the G.O.P. was taken over by someone entirely alien to its ideology or behavior. As he writes, “How do you abandon deeply held beliefs about character, personal responsibility, foreign policy, and the national debt in a matter of months? You don’t. The obvious answer is those beliefs weren’t deeply held.”

I recently spoke by phone with Stevens, who was in Vermont. During our conversation, which has been edited for length and clarity, we discussed what a Romney Presidency might have looked like, the ethics of campaign consultants, and how we should evaluate the George W. Bush years.

Your book differs from others in arguing not that the Republican Party has gone wrong with Donald Trump, but that the Republican Party has been going wrong for a very long time. What is the difference in your mind between those two critiques, and why did you think it was important to make the latter one?

I think I wanted to believe for a long time that when Donald Trump came along he was hijacking the Party. A lot of people were wrong about Trump in 2016, but it’s hard to find somebody more wrong than I was. I predicted he wouldn’t win the primary, and I predicted he wouldn’t win the general election. In retrospect, a lot of that was that I didn’t want to believe it. He says he has ninety-five per cent Republican approval, and that’s probably an exaggeration, but let’s say it is eighty-nine per cent or so. You look at what Trump is saying, and the degree to which the Party is comfortable with it, and I don’t know what conclusion to come to other than that Trump very well suits the Republican Party. In the book, I trace the history of the Party from the post-World War Two era, and Trump is one direction the Party could have gone in. I don’t know how else to say it, but it did go in that direction.

Without getting into a question about cause and effect, and how the universe functions, and whether a different future was possible, what do you think held the Republican Party together pre-Trump, and what was attractive to you about it?

What appealed to me was a party that believed in personal responsibility, that character counts, and that was strong on Russia and free trade, and strongly pro legal immigration. It’s not just that the Party has drifted away from those principles, like parties do. As far as I can tell, the Party is actively against every one of them. We’re the “character doesn’t count” party. We’re the anti-personal-responsibility party. We’re the pro-Putin party. I think the only conclusion is that a party that said it believed in these things didn’t really believe in them. If a George Wallace type had run in the Democratic primary in 2016, would that person have won? No. Had that person by some freak accident won, would the Party establishment have coalesced around him? No. They would have said, “This isn’t what the Democratic Party stands for.”

Your book has a huge chapter about racism and the degree to which it’s always been a part of the modern Republican Party. How do you see the effect it had before Trump?

There are always these tensions. There was Eisenhower; there was McCarthy. We look at William F. Buckley now and mourn him for the loss of this erudite voice. Instead we have Sean Hannity. But we forget Buckley started out as a stone-cold racist arguing for segregation. If you go to the Bush campaign that I worked on in 1999 and 2000, when Bush called himself a compassionate conservative and was criticized on the right, he tried to articulate a different vision. Had Bush not become a wartime President, I think it’s fascinating to think what would have happened.

But the Party has existed predominantly as a white party. I think if you’re a business, and you spend sixty years appealing to one segment of the market, you get good at that and not very good at the other. What I think is really striking is that we used to admit this was a failure and talk about it. We used to talk a lot about a big tent. I go back to 2005, when Ken Mehlman, the chairman of the Party, went before the N.A.A.C.P. and apologized for the Southern strategy.

This was the same Ken Mehlman who helped run the Bush reëlection campaign, which used anti-gay-marriage initiatives to turn out Republicans in key states.

What I say about this is that we were far from perfect in Bush world. We played too much to the dark side. But we had an aspiration to be better than we were. I think that’s important. With Trump, he takes your worst self and validates that as your best self. That part of you, we all have it, that feels aggrieved, that feels cheated—the side of you that, when someone cuts you off in traffic, you have that little spirit of road rage. Trump said, That’s your best self. You should empower that. I think that that’s an important distinction.

Bush clearly is a nice guy in certain ways that Trump is not a nice guy. But when you look back at the Bush Administration, and you look at what’s going on now—when you talk about contempt for science and you look at the way the Bush Administration dealt with global warming, when you look at spurning expertise and making decisions and how Iraq ended up playing out and Katrina played out, do you think that even if we see that the intentions were different and that cruelty was not the point of the Bush Administration, to take a phrase from another writer, that there were more commonalities there? And that it’s not a coincidence that the last two Republican Presidencies are ending in failure, assuming that Trump’s Presidency is ending soon?

I think the answer is, yes, those elements were there. Definitely. Look, Bush gets elected. If you look at that picture of him signing [No Child Left Behind] with Ted Kennedy behind him, I mean, today that would be submitted in a war-crimes trial in the Republican Party. It’s unimaginable that that would happen with Trump. There was that side of Bush. What happened after 9/11? Did he demonize Muslims? No, he didn’t. He defended Muslims. There’s that side of him. Obviously, Iraq was a debacle, and we can argue about how that happened, why it happened, what they believed when they went in—but it was a disaster, undoubtedly, one of the great disasters in American history.

I think we played too much on the social-conservative side, particularly, with the same-sex-marriage referendum. I think that’s regrettable. So I don’t think it’s one or the other. I think both. What I’m describing is the tension within that party, that both elements existed. Now that’s not uncommon for a party. You can look at Joe Biden and you can look at Bernie Sanders, and they exist in the same party. There’s an argument to be made that diversity is a strength, though I think the sort of know-nothingness of anti-science is not a strength. I don’t think that’s diversity. I think it’s ignorance.

I think that Bush being there as a leader made a huge difference in an ability to at least assert values of commonality that were not our worst selves. I think with Trump that has been abandoned. Then the question becomes, Do you accept that? In 2015, when I went out against Trump, I can’t tell you how many people in the Republican Party, high levels, were e-mailing me and saying, Thank you for doing this. I can’t do it because of X, Y, and Z, right up to about ten o’clock on Election Night. Then I started getting e-mails saying, Could you maybe erase that e-mail I sent? I think in many ways everything that a lot of us said about Trump in 2015 turned out to be overly optimistic.

If I had said in 2016 that it’s going to be July, 2020, and we’re going to have the worst economy ever in the history of America, the greatest decline in G.D.P. in American history, over a hundred and fifty thousand Americans dead in a pandemic in the last four months, and Donald Trump is talking about suspending elections, people would have said I’m crazy.

It’s not ideal. Let me ask you, though, about your mental state. You write about the deficit and “out of control” federal spending being a phrase Republicans use. And then you say, “But no one really believes in it any more than communicants believe they are actually eating and drinking the body and blood of Christ.” Putting the latter aside for a minute, when you say no one really believes it, are you implying that no Republicans really believe it, or that you didn’t believe it when you were working in Republican politics?

I’ve said before that I probably represented the worst of the American political system. I was a guy who was drawn to politics because of campaigns and not government. You know, I didn’t think I’d be very good at government. I briefly worked on the Hill and I was probably the world’s worst staffer, which, there’s a lot of competition for that. The process of government, which at its base level is usually pretty boring and tedious, just didn’t appeal to me. I really didn’t think about this stuff a lot. I should have thought about it more. You’re not powerless. You can have an impact on this.

The deficit line is just something people say, in the way you say when you see someone you don’t like, “It’s nice to see you.” It’s sort of like a social nicety. If you said to them, “Are you for major deficits?” They would say, “No,” and they would have passed that lie-detector test. But if you had said, “Are you willing to do what it takes to end the deficit?” The answer would have been what happened, “No.” It’s really even worse than that. If you go and you look at the last time that the deficit got wrestled under control, it was under Clinton. In part, that was because Clinton passed a tax increase. If you go back and you look at what we all predicted, and I made a million spots, we predicted economic Armageddon after the Clinton days. Instead, we had the beginning of the greatest period of growth in American history. We were wrong. I think we have to admit that and look at what happened.

Would Mitt Romney have made a good President if he had been elected in 2012?

I think Mitt Romney would have been a great President, and every day I wish that he were President.

Maybe this gets to the fundamental issue we’ve been going back and forth on. I think a lot of people acknowledge that Romney is in many ways a pretty decent, honorable man. Certainly, I think a lot of liberals think this now, and a lot of centrists think this now. At the same time, you’ve just finished talking about how Iraq was a disaster and how Republican economic policies don’t really make much sense. Romney was, essentially, running on “doubling Guantánamo,” and an aggressive foreign policy, and the Paul Ryan budget. If Romney had been elected, would his have been another Republican Presidency that ended the way the Bush Presidency did? Is that sort of more important than the fact that Romney is an honorable guy?

I don’t think that all Republican Presidencies end badly. I don’t think that the Bush Forty-one Presidency ended badly. I think that the W. Bush Presidency is the tragedy of Iraq. I think Mitt Romney is someone who has a proven record of being able to bring people together and being able to solve problems in a practical way not driven by an extreme ideology. I believe that we would have had more control on deficit spending, because you would have had someone who actually could have done something about it, who would have actually believed in it. Unlike Donald Trump who says, “I’m the king of debt,” and can’t pass anything because he can’t bring people together.

It’s interesting why Mitt Romney picked Paul Ryan, and it’s a very personal choice and I don’t want to speak for the senator, but I think he picked him as a governing choice. I mean, he said as much. I think it’s correct. He had no experience on Capitol Hill. He thought that having that experience would be important. Paul Ryan was someone who was well liked across the aisle. I think, together, they could have brought a lot of economic sanity to the country. I think they would have tried. Is it possible in America today to bring the budget down? I think so. I don’t know anyone on the conservative side that can do the same with any credibility.

But Paul Ryan helped pass the Trump tax cuts, which, as you say, were deficit-exploding. He was the one who shepherded it through the House.

Yes, yes. I think when not combined with cuts in the budget, it creates huge deficits that we have. If cutting taxes was a unified theory for Republicans, which it is in a certain class of Republicans, I think it’s proved to be a very false religion. I don’t think that what we invested in cutting taxes has proved to be accurate.

I’m less convinced that Mitt Romney and Paul Ryan and a Republican Congress would have gotten responsible about the federal budget in this alternate universe we’re talking about. Bush also had a budget-busting tax cut. It just seems very integral to what the Republican Party is, completely independent of Donald Trump.

Well, you can’t argue with that, O.K.? Whether or not Romney could have taken the Party in a more productive, rational fiscal policy, we’ll never know. I say this in the book, and I think it’s pretty much inarguable, the idea of so-called trickle-down economics has proved to be nonsense.

Do you think Paul Ryan and Mitt Romney and George W. Bush think it’s nonsense?

I can’t speak for them.

How do you understand his 2012 campaign—that Romney met with Trump—and things like this?

Listen, both those candidates had fifty per cent favorables. Romney got a lot of criticism for not going far enough on stuff. If you go back to the foreign-policy debate, he was criticized for not trying to make Benghazi the center of all evil. In the Republican primary, I remember this well, there was one of those awful raise-your-hand questions: Who believes Barack Obama is a socialist? Mitt Romney was the only person who didn’t raise his hand.

He did write a book called “No Apology,” presenting Obama as some sort of foreign-policy radical.

I think there were huge failures with Barack Obama’s foreign policy. I really don’t know how anybody would argue with that. You could argue that those failures were inevitable, but you can’t look at Syria, the greatest human-rights tragedy in the post-World War Two world, and not think that it was a tremendous, horrible failure. You can sustain two beliefs. I can. One, that Barack Obama is a decent human being, and if Barack Obama were President today the country would be better off. I say that without hesitation, while admitting that there were failures. I think that there was a failure on Obamacare not to be able to come to some greater ability to have both parties support part of it. And probably what we should have done on Obamacare is divide it into pieces. I think it’s governmentally problematic when you have one party pass it.

He did pass a version of Mitt Romney’s health-care plan, but let me ask you, Are you in touch with Romney?

I talk to Senator Romney some. Mainly about books we’re reading and just stuff going on in mutual friends’ lives. You know, the impeachment, for example, I had no idea how he would vote. The stuff that I talk to him about is more like personal stuff.

Do you think that Romney or Bush will endorse Joe Biden? Do you think that they should?

I would be very surprised if they do. I think, particularly, for an ex-President the roles that they play—I mean, there’s a lot of talk about why didn’t President Obama weigh in more in the primary. I think it is a unique role that has serious gravity.

I would be very surprised if Senator Romney endorsed him. My experience with Romney is pretty much that on the record and off the record is the same these days. What he said is that he intends to do what he did last time. I think that’s what he’ll do.

Does that disappoint you in any way?

No. I think Mitt Romney has been heroic. I think the eternal shame of most of these Republicans is going to be, why didn’t we follow Mitt Romney? I mean, Mitt Romney went out [against Trump] in March of 2016, and, had the Republican Party rallied around him, we could have saved ourselves from this incredible debacle and disgrace and humiliation.

Did you ever figure out what his whole Secretary of State flirtation thing was? Do you think that was just trying to be an adult in the room?

I pretty much know what it was about, because I talked to him then. I think it’s pretty much what you would think—that there’s a greater duty to the country, and I think he thought that, out of respect for the office and the process, he should talk to Donald Trump. I think he’s very glad that he was never asked to be Secretary of State, because I think it would have been the shortest Secretary of State tenure in history. I don’t think he would have lasted two months.

Why are Southerners attracted to Donald Trump?

It’s a question I’ve asked myself a lot, because in many ways Donald Trump is the caricature of the rich Yankee that we’re always warned about, who has no manners, no respect for anyone, including women, who is crass, and values money over everything. That’s Donald Trump, and he’s pretty much wildly popular with a lot of polite Southerners. I think that there is a perception of Donald Trump as a fighter that appeals to a Southern Scotch-Irish tradition that loves to fight. I think there was a sense that he was politically incorrect and would tell the establishment, “Screw you.” At the same time, in a lot of these states he did worse than Mitt Romney did. In many ways, Mitt Romney was an unusual fit being a Mormon and also being from Massachusetts. I think that there’s a lot of reluctance.

I’ve found the Roy Moore thing both the most inspiring and depressing event, because you say, “What would it take to get white Republicans to vote for a Democratic moderate?” You say, “O.K., what if the Republican was a child molester?” [In 2018, the Democrat Doug Jones defeated Moore in the race for a U.S. Senate seat, after Moore denied reports that he had made sexual advances toward underage girls in the seventies.] The positive is—and not for the first time—that Alabama was saved by African-Americans and saved by, particularly, African-American women. And saved by evangelicals. You know, one of the things that drives me absolutely crazy is we talk about evangelicals and say Trump is popular with evangelicals, and that’s not true. Trump is popular with white evangelicals.

Same with the way we talk about the working class.

Exactly.

The Lincoln Project ads have obviously been very effective in getting a lot of press. Do you think that they’re aimed more at élites to signal the Republican opposition to Trump, or are you really trying to get voters to switch? I’m not saying the first is necessarily unimportant, but what’s the goal?

I think there are multiple purposes here, and I think the purposes are shifting as the Lincoln Project starts spending more money on television. I think, in part, they’ve existed to give permission to others to say what needs to be said that isn’t being said. Look, I’m just kind of a backup singer in the Lincoln Project. It wasn’t my creation. I’m playing rhythm guitar on this thing. But not having a client is very liberating. You don’t have to worry that if you go too far, it’s going to blow back on your client, because you don’t have a client. I think that part of the role that the project has been able to play is to say what people are thinking, but you really can’t say that out loud. You say it out loud, and I think that’s positive, and I think it moves the needle of discussion. I think that it helps the Biden campaign.

I know that Trumpworld gives every indication of being obsessed with the Lincoln Project, and then it’s a joke when they attack us. Are you crazy? I mean, we’re not running for President. That’s a day that they’re not focussed on the Biden campaign.

Let’s say I’m a little bit of a cynical person, and let’s say I’m, like, look, these Lincoln Project ads have been really good, but the people who are doing it are these former Republican operatives like Steve Schmidt, who was working for Howard Schultz a year ago, and John Weaver, who got in trouble for agreeing to do lobbying for Russia a year ago. It’s all these operatives, and they’re coming together to make money off anti-Trumpism. Why is that too cynical?

There are none of us that couldn’t be making a gazillion dollars working for Trump. The idea that you’re going to go out and run against your own party and that’s going to be beneficial financially is nonsense. Look, we’re political consultants. We even joked about this. “How is it that we became the conscience of the Party?” We’re supposed to be the hacks. We’re supposed to be the people who say, do anything. We don’t confuse ourselves with role models. We don’t confuse ourselves with being candidates. We’re operatives. We’re all appalled at what’s happened in the Republican Party. We have a few skills that we developed over the years, and our choice is one of three things: support Trump or say nothing; O.K., not going to do that. Be quiet; really don’t like that. Or try to use these skills that we have to make a difference.

You sound like Liam Neeson in “Taken.”

Yeah. Well, look, I mean, I don’t think society should look at the total gestalt of the political system and say, “You know, the people I really admire are consultants.” That would be a pretty sick society. You could argue whether or not we’re a necessary evil, but I don’t think that people are going to look at Jeffersonian democracy and say the problem with that Constitution is it didn’t have enough about political consultants. We’re an aberration of a system that has evolved, but it is our system, and we are good at some stuff.

Either Donald Trump is going to be elected President or he’s not. I hope he’s not. We are trying to employ that which we know how to do to achieve that goal. We’re not confusing it with personal nobility. We’re just guys and women that know how to do some stuff, and we’re trying to do it to beat Donald Trump.

Jeffrey Epstein Conspiracies and the Mysterious Deaths of the Rich and Ruined

On the evening of November 5, 1991, a Spanish fisherman spotted the body of Robert Maxwell, a controversial British press baron, floating in the Atlantic Ocean near the Canary Islands. The crew of Maxwell’s luxury motor yacht had been searching for him all day, after he vanished, that morning, with no explanation. Almost immediately, conspiracy theories emerged. Maxwell, who came to Britain as an impoverished Eastern European émigré and turned himself into a larger-than-life figure and confidant of political leaders, hadn’t ended his own life: he had been murdered. The rumored perpetrators included agents of the K.G.B. or M.I.6, or a team of frogmen from the Mossad. In support of this theory, it was pointed out that Maxwell had long been rumored to have ties to various intelligence agencies, especially the Israeli one. Maybe he had been silenced to prevent him from spilling the beans.

Almost thirty years later, some people cling to these confabulations, despite the existence of a simpler and more convincing explanation for Maxwell’s death. When he set out on his boat, he knew that the debt-burdened business empire which he had spent decades building, Maxwell Communication Corporation, was on the brink of collapse. He also knew that, in a desperate and failed effort to prevent such an outcome, he and his associates had taken hundreds of millions of pounds from M.C.C.’s employee pensions and used the money to try to prop up the company’s share price. After the inevitable bankruptcy occurred, this illegal scheme would be revealed. Maxwell would be ruined, shamed, and, most likely, sent to jail. To a man who was eaten up by pride and insecurity even as he became a well-known figure on two continents—that year, he had purchased the Daily News—the prospect of financial ruin and public humiliation was too much to take. So he jumped overboard.

Having followed Maxwell’s career closely as a financial writer and editor for the London Sunday Times, I believed at the time, and continue to believe, this version of events. It doesn’t clear up all the mysteries surrounding Maxwell’s death, such as the lack of a suicide note and the fact that a team of coroners couldn’t agree conclusively on the cause, leaving open the possibility of heart attack or accidental drowning. But suicide is intuitively plausible, and it satisfies the principle of Occam’s razor, which says that when choosing between various theories we should choose the one that provides the simplest explanation and requires the fewest auxiliary hypotheses to be true.

In a remarkable quirk of history, the stories of Robert Maxwell and Jeffrey Epstein are linked, through Maxwell’s daughter, Ghislaine. The motor yacht on which Maxwell took his last steps was called Lady Ghislaine. Shortly after his death, Ghislaine Maxwell moved to New York, where she met Epstein, becoming his girlfriend, and, according to some accounts, his procurer. (She has vigorously denied these claims.) Like Maxwell, Epstein was a self-made figure—he hailed from Coney Island and didn’t graduate from college—who lived by his wits. Like Maxwell, he cultivated prominent people even though the source of his fortune was opaque. And, like Maxwell in 1991, at the time of Epstein’s death everything was being taken away from him.

A decade ago, Epstein used his money and influence to emerge from a two-year F.B.I. investigation pleading guilty to just two state charges of soliciting prostitution, one involving a minor. This time, however, he was trapped. In July, a team of federal prosecutors from the Southern District of New York had accused him of running a sex-trafficking scheme involving dozens of underage girls. A judge had denied his plea for bail. New witnesses had come forward. The case had attracted enormous publicity. Virtually everyone associated with Epstein had turned on him, including Leslie Wexner, the retail billionaire who appears to have been a primary source of Epstein’s fortune. (Last week, Wexner claimed that Epstein “misappropriated vast sums of money from me and my family.”)

At sixty-six, Epstein was facing the prospect of languishing for months in a nightmarish jail that had housed the likes of John Gotti and El Chapo; facing his accusers in a criminal trial; losing his fortune in civil suits; and spending the rest of his life in a federal pen, this time without the work release he’d been granted during his first incarceration. He had lost what sociopaths like him value most: control. Based on what we know now, it appears that Epstein killed himself, in his cell at the Metropolitan Correctional Center, on Saturday morning, and that no one else was involved. In some ways, this isn’t a very satisfying explanation, and it raises important questions about why Epstein wasn’t being supervised more closely. But it fits the facts that have been revealed so far. It also fits what we know about Epstein’s psychological profile. And it doesn’t require the involvement of Mossad frogmen, or their equivalent, to be true.

But how was he allowed to do it? According to the Wall Street Journal, Epstein’s own attorneys were the ones who requested that he be taken off suicide watch. This doesn’t explain why the authorities acceded to this request when Epstein, only weeks earlier, had been found unconscious in his cell, with bruises on his neck. Similarly, we don’t know why Epstein was left alone in his cell last Friday night, or why the guards didn’t check on him at regular intervals, as the jail’s standard procedure demanded. “It remained unclear why that procedure was not followed in Mr. Epstein’s case,” the Times reported on Sunday. Bob Hood, a former senior official at the Federal Bureau of Prisons, which runs the Metropolitan Correctional Center, told the Times, “The Bureau of Prisons dropped the ball. Period.”

That explanation won’t satisfy many people, of course—not with the President and members of his Administration spreading defamatory conspiracy theories about the Clintons. On Saturday, Trump retweeted a video from a conservative comedian, Terrence Williams, in which Williams suggested that Bill Clinton and Hillary Clinton were responsible for Epstein’s death. (Earlier on Saturday, Lynne Patton, an official at the Department of Housing and Urban Development, posted a headline about Epstein’s death along with the word “Hillary’d!!” and the hashtag “#VinceFosterPartTwo.”) What the forty-fifth President appears to be intimating is that an assassin, working for the forty-second President, broke into one of the most secure jails in the country, hanged Epstein, and left without disturbing the guards or being caught on internal cameras. And perhaps the most remarkable thing is that no one is really surprised to see Trump doing this—disinformation and incitement are two of his trademarks.

Of course, Trump isn’t the only one raising questions. As I pointed out in a column last month, the Epstein saga, in addition to being a sickening sex-crime story, is really about wealth, privilege, and the ability of the super-rich to circumvent the rules that bind ordinary people. Over the weekend, Mayor Bill de Blasio said, “Something’s way too convenient here, and we need to get down to the bottom of what happened.” De Blasio, along with Republican Senator Ben Sasse, has demanded an independent probe into the circumstances surrounding Epstein’s death. Could someone working for Epstein have got to the warden, or whoever made the suicide-watch decision, and to the guards? Given the way the earlier case was resolved, in 2008, and the list of names that have been associated with Epstein, such a possibility, outlandish as it sounds, needs to be investigated. Right now, though, the simplest explanation seems like the most persuasive one: Epstein wanted out, and a series of screwups allowed him to beat the system, again.

Will Hunter Biden Jeopardize His Father’s Campaign?

Joe Biden’s son is under scrutiny for his business dealings and tumultuous personal life.

In today’s political culture, people running for President may announce their candidacy on the steps of their home-town city hall or on “The View,” but the full introduction comes with their book. Some candidates’ memoirs tell stories of humble beginnings and of obstacles overcome; some describe searches for identity; some earnestly set out detailed policy agendas. Nearly all are relentlessly bland. In 2017, Joe Biden, a longtime senator from Delaware, Barack Obama’s Vice-President for eight years, and now a candidate for the Democratic Presidential nomination, published an unusually raw memoir about the death, two years earlier, of his forty-six-year-old son, Beau, describing how it had threatened to undo him but ultimately brought his family closer. Beau, his father writes, was “Joe Biden 2.0,” a war veteran, a prosecutor, and a promising politician who “had all the best of me, but with the bugs and flaws engineered out.”

How to run a multi-tenant WordPress platform on Google Kubernetes Engine

As a service provider running WordPress sites, it is all about density, density, density

WordPress is said to run 28% of all websites on the Internet. That is a phenomenal installed base of some 75 million sites. While some of these are massive sites like TechCrunch or The New Yorker, the vast majority of WordPress sites are much smaller.

That means as a WordPress hoster, your business probably follows the 80-20 rule. 80% of your revenue comes from 20% of your sites. Or said another way, 80% of your sites only account for 20% of your traffic.

That means that you need to think about your business in two ways:

  1. You need to provide a reliable service to a large number of low-traffic sites while minimizing infrastructure costs since your margins come in large part by placing more sites on the same physical infrastructure.
  2. You need to provide a white-glove, highly performant and reliable experience to a small number of sites that make up the bulk of your revenue.

At the same time, you need a migration path for some sites to move from low-volume to high-volume plan, without disrupting the customer or your own internal operations teams.

 

..“Our clusters are highly dense, meaning we run a lot of containers per host. On AWS, we use huge instances. The recommendation from Kubernetes is 100 pods per VM. Already, we’re running 200-300 pods per host. Also, since most of the apps that we run are stateful, we can easily have 200-300 volumes per host as well. And we’re working to push these limits even further. Because of these densities enabled by Kubernetes and Portworx, we’re easily saving 60-90% on our compute costs. Portworx itself was between 30-50% cheaper than any other storage solution we tested.”

 

.. If you categorically knew which 20% of your customers would account for 80% of your traffic at all times, solving the noisy neighbor problem would be a one-time migration. But, because traffic patterns change over time, this is a hard problem to solve. Portworx does a few critical things to help.

First, in addition to using Kubernetes to limit pod resources like Memory and CPU, you can use Portworx to automatically place different workloads on different storage hardware for different classes of service. For instance, you might sell your customers a premium “performance” plan if they are expecting heavy usage and they are performance sensitive. Alternatively, cost-conscious customers might opt for a “budget” plan that offers reliability but doesn’t guarantee blazing fast performance. On the backend, these plans can be mapped to Portworx “storage classes” that automatically place high-end plans on SSDs and low-end plans on HHDs.

 

.. Often a hosting customer will call their service provider the day before they are going to be on a national TV and say “I really need my site to work tomorrow.” This often leads to a lot of scrambling around and manual tuning, but with PX-Motion, moving the customer to an environment with more resources is as easy as kubectl apply -f wp-migration.yaml.

The above described moving one heavy load site off a multi-tenant cluster. This is often the best option if you have some advance warning before a large traffic spike. However, in the middle of a large traffic event, it is often better to move low traffic sites away from the heavy traffic site, instead of vice versa. This is also possible with PX-Motion.