One of the most useful applications of measuring analytics online is the ability to iterate – or change – your site’s design, structure and content in ways that lead directly to your goals. If you want your customers to complete an action (buy a product, fill out a form, request more information) then you ignore testing at your own peril. Major e-commerce sites like Amazon run hundreds of tests per month. There are potentially several items on a single page being tested at any given time.

The A/B Test
One method for analyzing consumer behavior is the A/B test. Two versions of a page or element (A and B) are compared, identical except for one variation. Half of the visitors receive the A version, half receive the B version. The “variation” is tested against the “control” to determine if the variation created a statistically significant outcome. If it did, the less-effective variation is dropped for all users. If that sounds like your high school science class, it should. This is a simple lab experiment, and your lab is the world wide web.

Why Use A/B Testing
The ability to test and learn makes a reliance on “best practices” a bad idea. To suggest a simple list of “always” or “never” practices regarding page layout or color or navigation is sure to fail. While there are basic concepts of design and functionality that are helpful starting points, only user testing will arm you with actual proof of a good or bad decision on your site. This requires developing a culture where the team (from all disciplines) regularly forms and tests hypotheses. These can be as simple as “I think users are more likely to sign up using a green button” or as complex as a complete re-structure of your site’s navigation. Because your users will each be served a completely functional version of your site, there is no down-time, no waiting on the perfect solution right out of the gate. Many of your tests will provide little or no usable insight, that’s to be expected. Testing requires time and resources, but when you hit a home run, the business results can be worth it.

Wired has a more in-depth look at the methods and history of A/B testing, but here are some quick-hit stories of tests that made a big difference to the bottom line:

A company that develops event-management software, Lyyti hypothesized that they could get more click-throughs from their pricing page by clearly outlining the features of each pricing plan, and by emphasizing “Free Trial” over “Request a Quote.” Result: The variation led to almost 98% more click-throughs.

Express Watches
A test was developed to understand some of the factors other than the product itself that lead to a purchase. In the control test, a module on the page included a guarantee from the brand: “Never beaten on price.” The variation replaced that guarantee with a “Seiko Authorized Dealer” badge.  Result: The variation led to a 107% increase in conversions. This led the team to the conclusion that their customers valued authenticity over the best price. 


California Closets
Headline and text are another area where testing can be extremely useful. It might be more difficult to spot the visual differences in this case study, so here is a bit of background first. The team wanted to test whether an acknowledgment of the user’s path to arrive at this landing page would make a difference in the effectiveness of the destination itself. The “You’ve Arrived” headline lets the user know that the brand knows they’ve come from a banner ad on another site. Even though the headline and sub-head might be a bit more clear on the variation, the results told a different story.  Result: The control version was 115% more successful than the variation. Why? When the user feels as though they are making actual progress through the sales funnel, they are more likely to compete the final step.

Testing: Make it a Habit
Beware anyone who suggests they’ve got the right strategy, design or platform for your business without a “test & learn” mindset. Intuition and experience are valuable, but in the online space, they should only serve as starting points. A website should be a living, breathing tool that does not get left in the dust by your online customers, who have zero tolerance for speedbumps. With so much data available, there should be very little guesswork, educated or otherwise!

(Thanks to Unbounce and VWO for the case studies.)