Scroll To Top

By Dick Maggiore and Mark Vandegrift

A/B Testing Deming Marketing Innis Maggiore

Deming’s gift to marketers: Test. Optimize. Repeat.

William Edwards Deming is credited with launching the Total Quality Management movement that is the groundwork for our modern emphasis on efficient and effective manufacturing.

Among his many accomplishments, Deming popularized statistical testing and analysis, which, it turns out, today helps marketers gain confidence in our work.

An early data geek, Deming was known for his quips. "In God we trust. Everyone else, bring data," he said.

Deming's work led to ISO 9000, lean manufacturing, Six Sigma, the Malcolm Baldrige National Quality Award, the notion of continuous improvement and just about everything else related to helping companies to get better at what they do.

Marketers can get better at what they do, too, by learning from Deming, a statistician known for developing sampling techniques.

A/B split testing

Deming did not invent A/B testing, but he did embrace data and how it is used to increase the efficiency and effectiveness of marketing and thereby increase return on investment.

A/B testing is a systematic method of comparing two versions of a marketing campaign or tactic. Testing would result in optimizing an email campaign's open, click or conversion rates, increasing response to a direct mail campaign, reducing a website's bounce rate, increasing an advertisement's online or offline response rate, reducing an e-commerce site's shopping cart abandonment rate, increasing the number of leads coming from a website and more.

Consider a traditional direct mail piece — snail mail.

Your knowledge and intuition of your products, your customers and the marketplace is enough to develop the first version for testing — the "control" version.

Next, we need to make a version to test against the control — the "challenger" version. To make it a true test, only one variation is allowed when developing the challenger. In the case of direct mail, the variation can be made in one of three areas: the message (the package), the mailing list (who you're mailing to) or the offer (the call-to-action).

For this example, let's pick the offer. The control version is a $5 off coupon offer. The challenger version is a BOGO (buy one, get one free) offer. Randomly, half the recipients will receive a $5 off coupon while the other half will receive the BOGO offer. Everything else remains exactly the same. If the BOGO offer wins, we have a new control.

The goal is for the challenger version to beat the control. If it does, the challenger version becomes the new control. Then, a new challenger version is developed and tested with the goal of beating the control.

Testing never ends. The goal is to keep optimizing — increasing the efficiency and effectiveness of the marketing campaign.

Testing in digital world

Testing digital marketing activities is a natural. Almost anything can be tested in the digital world. Marketing automation tools do most of the work for you.

Consider an email campaign in which the advertiser wants to improve the open rate. Several variables can be tested, including the emailing list itself, the time of day, week, month or year to blast, or the subject line.

We'll test the subject line. Remember, nothing else can be changed, otherwise, we wouldn't know what caused the lift or decline in the open rate.

If the test results showed the challenger subject line had 14 percent higher open rate compared to the control's subject line, we would have a new control. Then we could continue testing to increase the open rate. Or devise a test to increase the click-through rate. Or test to increase the conversion rate.

Many companies, particularly in the business-to-business (B2B) space, depend on leads coming from their websites for new growth. Google Analytics is a great place to start. It's free. It can help uncover a weakness in your conversion funnel.

Your website's conversion rate is key. It's so critical that a new term has been coined: conversion rate optimization, or CRO. Optimizing the conversion rate is greatly facilitated through A/B testing.

You can test a webpage or landing page (control) against a new webpage or landing page (challenger). At random, some of your visitors will go to the existing page with the others going to the new challenger page. The data will pick the winner.

If you are selling through an e-commerce site, you should test regularly. Small and easy changes can make significant improvements to your bottom line. It could be as simple as testing different colors of the "buy now" button, moving it to a different location or changing its size.

The technology side of the equation has caught up with the needs of advertisers. Testing should become standard practice with your organization. You and your marketing team provide the insights and strategic and creative thinking to drive the process.

As Edwin Land once said, "Industry is best at the intersection of science and art."