How presidential campaigns are building the perfect website

Which picture of Bernie Sanders makes you most likely to sign his birthday card? Is a limited-edition lithograph a more tempting thank-you gift than a Ben Carson for President hat? Are you more likely to click a donation button if it’s Green Bay Packers green?

As they optimize their websites to collect every possible donation and email address, these are the types of questions that the presidential campaigns are asking.

They’re using a technique called A/B testing—a way of making marketing and design decisions that grew out of the technology and culture of Silicon Valley startups. Rather than having a human select an option based on instinct or experience, an A/B test treats the decision like a scientific experiment: define a measure of success (like total donations or email signups), present each option to a randomly selected group of visitors, and see which performs best.

For instance, before he suspended his campaign, Scott Walker ran this interesting experiment: If the donation buttons themselves showed as much Green Bay Packers spirit as the Wisconsin governor, would people be more likely to click and give? Slide the bar left or right to compare the two versions.

While the results of this experiment weren’t dramatic enough to make a difference for Walker, who dropped out of the race in September, they illustrate the extreme attention to detail by campaigns desperate for an edge.

Campaigns run these experiments all the time. Since September, we’ve been tracking them using Optimizely, a popular A/B testing tool. We can see the experiments being run and review the variations being tested.

Because of the way the software works, we don’t have access to the results. Even so, it’s an interesting look at how campaigns are tweaking their websites to get the most out of them. In short, it’s all about the money.

Gifted hands

Ben Carson, noted for his focus on small donations, has been one of the most prolific users of Optimizely. He’s used it to test different thank-you gifts and donation amounts.

Hats remain a common gift on Carson’s donation pages, indicating either that they performed better or that the campaign ran out of limited-edition lithographs.

First person

In the days before the Iowa caucuses, Carson compared two differently phrased appeals to voters. One featured a picture of two supporters, and a statement of fact: “Ben Carson would like your vote.” The other, illustrated with a candid photo of Carson himself, was written from his perspective. “I desperately need to know if you are with me,” it said. “Will you please use the form below and let me know if I can count on your vote?”

Carson is now offering a similar pledge for South Carolina voters, featuring the more personal language and photo.

Happy birthday

Bernie Sanders has used his website to raise record numbers of small-dollar donations. Here, his campaign tested which photo would persuade more people to sign his birthday card:

The message

Some of the experiments are mostly cosmetic, but others are more consequential. In late November, Sanders started testing different headlines on his splash page, the initial pitch that appears for new visitors. At first, there were eight possible headlines being tested, across a wide range of topics:

• Nobody who works 40 hours a week should be living in poverty.
• We need to keep fossil fuels in the ground and save our planet.
• It’s time the wealthiest people in America paid their fair share of taxes.
• A woman has the right to control her own body.
• We need pay equity for working women.
• We need a president who will get big money out of politics.
• It’s time to put an end to the private for-profit prison business.
• We should break up the “too-big-to-fail” banks.

After about a day of testing, the campaign whittled the list to the first five. After further testing, they tried a different set:

• Nobody who works 40 hours a week should be living in poverty.
• Our economy is rigged and our political system is corrupt.
• The Koch brothers shouldn’t be allowed to buy our politicians.

Finally, in mid-December, the campaign tried a set focused on child poverty:

• Nobody who works 40 hours a week should be living in poverty.
• As the wealthiest country in the history of the world, it’s disturbing that we have one of the highest rates of childhood poverty.
• The United States is rich enough to make sure none of our children live in poverty.
• We’ve got to eradicate childhood poverty.

The page currently reads, “Nobody who works 40 hours a week should be living in poverty.”

Your Name Here

Hillary Clinton’s campaign has also experimented with changes to the donation page, like this version, which puts your name on the final Donate button. The screen appears after you’ve entered your name on the previous page.

The personalized button did not survive the experiment.

Moving Pictures

Clinton is currently experimenting with video backgrounds on the donation page:

When Bill Clinton and Bob Dole launched the first presidential campaign websites 20 years ago, they didn’t even accept online donations. Visitors could download screensavers or fill out an intimidating form to get on the mailing list, but the sites looked nothing like those of modern campaigns, with their eye-catching graphics and immediate calls to action.

Campaigns like Howard Dean’s in 2004 and Barack Obama’s in 2008 showed the potential of websites as tools for organizing and fundraising. Now social platforms do some of what websites once did—candidates share videos on Facebook, post pictures from the trail on Instagram and Snapchat, and converse on Twitter—leaving websites ever more focused on raising money.

Optimizely itself represents a melding of Silicon Valley and electoral politics. One of its eventual founders worked at Google and was director of analytics for the 2008 Obama campaign. While most of Optimizely’s business comes from commercial clients like Spotify and Fusion, it’s not surprising that campaigns find its tools useful.

We found Optimizely in use on the websites of Hillary Clinton, Bernie Sanders, Jeb Bush, Marco Rubio, and Ben Carson, plus the doomed campaigns of Rand Paul, Mike Huckabee, Bobby Jindal, Scott Walker, and Rick Perry.

According to Federal Election Commission filings, the Sanders campaign has spent at least $54,000 on Optimizely’s services. The Clinton campaign has spent at least $33,420, the Carson campaign at least $23,650, Rand Paul at least $13,000, and Scott Walker at least $5,000. (This data is not necessarily comprehensive because it only includes direct expenditures by a campaign.)

These are just a sampling of the experiments the candidates have run so far. We’ll keep tracking them as the campaign progresses.

Daniel McLaughlin is a creative technologist exploring the 2016 presidential election. Before joining Fusion, Daniel worked at the Boston Globe and graduated from MIT with a BS in urban studies and planning.

Kate Stohr is a data journalist and community builder based in San Francisco, CA.