Showing posts with label Optimization. Show all posts
Showing posts with label Optimization. Show all posts

Wednesday, November 12, 2014

Deodorant Goes Viral! Test Intelligently.



In my last post, we took a look at a framework for optimization that creates results that will inform future programs and tactics. Using Scientific method, we created control and test groups to prove out our hypotheses. We also looked at ways to increase our testing flexibility using test matrices allowing us to test multiple factors simultaneously to get the best combination of variables. Now let’s look at a real-world example that might challenge our assumptions.

Deodorant goes Viral!                                                  
Let’s say our good friends at Acme, a Consume Packaged Goods company, is releasing a new mens' deodorant brand. The marketing brain trust has spent plenty of time, resources and money testing and validating the brand positioning and customer value proposition. Acme has settled in on a particular brand voice that is best presented in a humorous light.

Testing on several control groups have settled on a particular series of video clips that have shown the potential to “go viral” and spread the new brand’s message over the Internet for very little money. The demand gen team packages and presents the idea to brand leadership and everyone agrees the videos are engaging and hilarious and will definitely spread like wildfire over the Internet, so the project gets the green light. Time to light ‘em up! The Demand Gen team posts the videos on You Tube and eagerly waits for the results.

Within hours the video does, indeed, go viral as expected. Views are in the millions, and rapidly increasing. Within days the videos are at the very top of the view lists and are making the rounds via Social Media channels. All is good, right?

And now, a word from our sponsor…
Expectations are high, but word from the distribution team is the product is staying on the shelves in droves. Expected backfill orders are non-existent. The videos are obviously a smash hit, spreading the new brand across the Internet at record speed. Why no product sales as a result? Quick, somebody check their Twitter account to see what’s going on!

Houston, we have a problem. While everyone seems to absolutely love the humor in the video, the reaction is not exactly what product planners have in mind.

#worldsdumbestdeodorant
#dumberthandirtdeodorant
#lmaodumbdeodorant
#smellslikestupiddeodorant
#canyousmellthatgoodandbethatstupid

 Yes, the videos are hilarious, but nobody wants to be associated with a product whose promotional videos depict its users as complete idiots. And it’s too late for those tens of millions of viewers to “un-see” those videos.

What should Acme have done?
This is a clear example of where spending a little time and money on a limited pilot could have saved a lot of embarrassment later. Executing tests in a controlled environment with limited, but representative audiences can provide insight into those unintended consequences. These pilots should concentrate on understanding not only what will happen, but why it happens.

Change the conversation.
Your pilot should be conducted with the end objective in mind. These pilots are good at bridging assumptions you may have made in causal relationships. In this case, ACME assumed that millions of Internet views would cause millions of product sales. Instead, those views led to brand ridicule and corporate embarrassment.

Pilots can also be used to test specific audience segment reactions against the whole. It is a great way to determine how message/offer variants track in one segment vs. another. Do customized messages really improve responses? Better to pilot and know than spend additional time and money only to find out after the fact you generated no additional lift for your efforts.

Notes:

Pilot before committing to an untested plan.

Pilot with the end objective in mind.

Pilot to correct causal assumptions.
                                    
We’re now piloting testing and optimizing every bit of success out of every program we execute. It’s time to tell your C-Suite about the fantastic work you are doing. Unfortunately, Marketing historically has done a less-than-stellar job of reporting on its success. In our next installment, we’re going to begin a series on reporting to overcome that legacy: Create Marketing Reports That Will Thrill Your C-Suite!

Tuesday, November 4, 2014

Three Ways to Use your Demand Funnel Part 3. Optimize Your Lead Processes.



We just explored way to properly use your Demand Funnel to diagnose and fix problems with your Lead processes. Using this diagnose-and-fix methodology, you should be able to get leads moving and plug and any leaks in your process. This should create a positive lead flow with negligible leakage or stuck leads and an “always-on” diagnostic to keep you from developing more problems. So you’re good, right? Not so fast! You now need to use the Demand Funnel infrastructure you’ve created to optimize your process. Here’s how.

Understanding Optimization
Optimizations makes some assumptions, so let’s make sure we have properly aligned on those.

1.       Optimization assumes your process works. If your process is broken, optimization will not help. It may make matters worse.
2.       Optimization should be built in to your Lead Management process. It is not an optional thing you do every once in a while for specific processes or parts of your process. To be really successful, optimization needs to become a part of your Demand Gen culture – something that is an expected part of the way everyone does their jobs.
3.       Optimization requires a framework and methodology. You cannot optimize against a moving target.
                                                           
We have already covered many ways to perfect your process, so we’ll spend little time here addressing that subject. At this point we need to assume your processes are defined and properly implemented.

Building Optimization into your process
If we think of Demand Generation as a series of actions taken to achieve success against a given strategy, we would normally execute these actions with a plan (as opposed to random acts of marketing). Organizations manage those projects in a variety of ways with a variety of tools – most often referred to as project management. The three pillars of project management are: 1) Scope, 2) Resources and 3) Time. To build optimization into your project plan, you are likely affecting all three of those pillars, with scope being the key. Let’s look at a simplified example.

For every Demand Generation project, no matter which tactics or media are used, major project categories remain relatively constant. Starting with the idea that we would like to do X by Y date, you may see a project plan similar to this:

1.       Initiation and Planning: What do we want to do? Who is our target audience? What defines success in Demand Funnel terms?  How will we measure it?
2.       Discovery: What are the requirements, resources and constraints to doing it?
3.       Ideation: Based on our discovery, how should we do it?
4.       Design: Based on ideation results, what do the deliverables look like?
5.       Development: Based on approved design concepts, build the deliverables.
6.       Testing: Based on the design requirements, make sure the deliverables meet requirements.
7.       Deployment: Execute the delivery of tactics to the target audience.
8.       Reporting: How did we do against our plan?

As a Demand Gen professional, you probably recognize project plans that look similar to this. Within each of these major categories will be many sub-categories, each dependent on the scope, time and resource availability. Of course, your organization will have its unique twist to this structure.

What’s missing? Optimization. Where should optimization fit in this plan? Let’s take a look at a revised project plan including optimization.

1.       Initiation and Planning: What do we want to do? Who is our target audience? What defines success in Demand Funnel terms?  How will we measure it? How will we optimize it?
2.       Discovery: What are the requirements, resources and constraints to doing it?
3.       Ideation: Based on our discovery, how should we do it? What should be optimized?
4.       Design: Based on ideation results, what do the deliverables look like? What components of the deliverables should be optimized and compared to other similar programs.
5.       Development: Based on approved design concepts, build the deliverables. Build the optimization components.
6.       Testing: Based on the design requirements, make sure the deliverables meet requirements. Make sure the optimization components and tests work per specifications.
7.       Deployment: Execute the delivery of tactics to the target audience. Implement the testing components.
8.       Optimization: Evaluate test components on the prescribed cycle. Report and record results within the framework. Deploy winning components.
9.       Reporting: How did we do against our plan?

Notice we only added one major step to the plan, but optimization has become completely integrated into the planning, design and execution of the overall plan.

Optimization Framework
Optimization on a particular program is great. Great for that particular program. But how do we scale optimization so we can leverage optimization learnings across Demand Gen programs for the entire organization? For that, we need a framework.

A framework is simply a method by which we can organize and execute our testing and optimization such that results can be compared to other, similar tests. Think back to high school science classes where you learned the Scientific Method. Your testing framework should emulate this methodology to maximize your results and create the ability to measure them across multiple programs of similar – and, in some cases, dissimilar – types of programs. To get to an operational framework, let’s start with objectives and work backwards.

Objective: Create 1,000 MQLs in Q1

Let’s break this down. (Although your number is different, this is probably a very common objective for Demand gen teams, so it will be easy to relate.) We have two real options here. The first is to create MQLs from your unknown TAM – those contacts outside your database, but fit your target market. The second is to convert current Inquiries (those contacts already in your database who have previously interacted with your brand) to MQLs. This sounds like the basis for an optimization test, so what would we want to understand from this program test?

1.       Total Volume of conversions: did more Inquiries or Unknowns convert?
2.       Speed of conversions: Which converted faster – Inquiries or Unknowns?
3.       Cost of conversions: Which converted at a lower cost - Inquiries or Unknowns?

As a result of this optimization framework, we would begin to understand how best to apply our marketing resources to the challenge of creating MQLs. However, we need to dig deeper into optimization because we need to understand why these conversions happen. Without that understanding, we could wind up applying the wrong tactics for the wrong reasons. This is where our use of Scientific Method gains us an advantage in optimization.

We start with a hypothesis and devise a method to test that hypothesis against a baseline. Let’s say our proposed program is designed to execute a very typical multi-touch mix of tactics, including an email and a direct mail message. We have looked at the previous arguments and decided to address current Inquiries in our database as our target audience, with the objective of creating our 1,000 MQLs. It have been determined that conversions (via form submit) from this program will generate sufficient scoring points to “auto-MQL” any Contact already in Inquiry status. Let’s also assume our benchmark conversion rate is 5%, so we require a total audience of 20,000 Inquiries for this program. Our optimization objective is to convert at least 6%, beating our benchmark rate by 20%. How, then should we design our optimization tests?

Let’s start with our hypothesis. We believe offer messages work better than product messages because our product is well-known and comparable to other products in the industry. Price drives conversions.

The conversion points for this program are:

Data and list hygiene, along with good emailing practices, are the keys to deliverability. We don’t test for that at a program level, so let’s assume static deliverability for this program.

Open rates are a direct result of the combination of sender and subject lines (for email) or packaging for direct mail. We’re going to test two specific messages:
1.       Better Widgets, Better Price
2.       20% off Widgets until Friday.

Once the email or direct mail is opened, we need to get the recipient to read far enough into our message to respond to our call to action (CTA). We’ve already established that our products are well known, so we don’t need a lot of description, other than why the reader should respond now. Here, we are going to test two messages and two CTAs in a multivariate matrix to understand what combination provides the best results. This test covers both the Read and Respond steps in our conversion path.

Message
1.       Our great widgets do more for you.
2.       Our great widgets are on sale.
CTA
A.      Better Widgets, Better Price.
B.      20% Off this week only

The matrix is very simple, covers every possible combination, and looks like this:

                1A           2A
                1B           2B

Conversion happens once the recipient has responded, and is the final measure of how committed the recipient is to your message and offer. Whether the response is to purchase or to engage more deeply with your brand, this commitment is the critical step. Do not assume that once your recipient has clicked the CTA, he or she will go ahead and execute the conversion. Abandonment here indicates you have not solidified your “closing message.” Let’s try something to test that commitment here – I call it the bonus close.

For three of the four message/offer combinations, a sale price is expected. Only one expects just a “better” price. Let’s align each of the four combinations with a bonus close test.

·         1A expects only a better price.  Bonus Offer = 20% off through Friday
·         1B expects 20% off this week only. Bonus Offer = 22% off if you order by Wednesday
·         2A expects an unidentified sale price. Bonus Offer = 20% off through Friday
·         2B expects 20% off this week only. No Bonus Offer = 20% off through Friday

These tests will help us understand the best mix of offers and bonus offers as they meet or exceed expectation when the recipient hits the conversion form.
                                                      
Change the conversation.
Optimization is a conversation that needs to be a part of every program execution. As you have seen here, it is a lot more than a few random tests thrown into the mix to get a few extra conversions; it is an entirely new way of thinking about what drives those conversions. The conversation should not be if we test, it should be how we test. Smarter tests lead to smarter programs.

Notes:

Test against solid hypotheses to provide scalable optimization.

Optimization is a culture, not an action.

Test every conversion step in your program process.
                                    
Are you all in on optimization? I hope you are. One last thing to consider: what if one of your tests goes wildly wrong? (This is not a bad thing – it is why we test!) Is there a way to mitigate risk and not put your Demand Funnel in jeopardy? Excellent question, and we’ll look at ways to do that next week in: Deodorant Goes Viral! Test intelligently.

Monday, August 25, 2014

Three Ways A-B Testing Will Improve Your Marketing. (Part 3) Into the Vortex.


Automating your Demand Generation functions is often referred to as a journey. However, your journey need not be aimless and without a destination. In fact, if you don’t have a destination in mind, your journey will take much longer then necessary and, in fact, may never end. As the saying goes, “If you don’t know where you’re going, how will you know when you get there?”

While your ultimate destination may be something like “a world-class Demand Center,” you will need to establish some milestones along the way to measure your progress. Testing will help you to establish your progress toward (and beyond) those milestones. When it relates specifically to A-B testing, there are some great milestones against which you should always be measuring your progress! Those milestones may be found in your Demand Waterfall (or Demand Funnel).

Principle #1: Programs should be measured against movement
Marketers often fall into the trap of trying to generate activity. Activity is fine, but if that activity does not result in movement, it has both cost you money and gained you nothing. Let me explain.

Let’s say you create an outbound marketing program, sending 50,000 emails to your target audience. You have written incredibly compelling content with a CTA pointing to a wildly popular whitepaper. Your Subject Line drew a 50% open rate and your content prompted 50% of the opens to download your fabulous whitepaper, resulting in 12,500 downloads at a 25% conversion rate. My guess is you’d be jumping up and down at your success after posting these metrics. Not so fast! You’ve generated lots of activity, but have you generated any movement?

Unless you got paid for all of those whitepaper downloads, you’ve actually spent time and money to send out a free piece of information to your prospects, customer and, likely, your competitors. Here’s the real question: what did the recipients do as a result of reading the whitepaper? What was the goal of your program? Was it awareness, engagement or conversion? Did the readers react according to your goals?

Principle #2: Movement should be measured in terms of Waterfall Stages
You have a Demand Waterfall for a reason, which is to determine your prospects’ stage in their buyers’ journeys. Assuming your Waterfall accurately reflects that buyer’s journey, each and every program you deploy should have the specific purpose of moving the prospect from one stage to the next in the funnel.

In our previous example, let’s assume our whitepaper program was an engagement program, with the purpose of moving Inquiries to AQL (Automation Qualified Lead) stage. Based on this goal, we now have a movement objective against which we can measure the success of our program. We can determine the success or failure of the program in terms of how many Inquiries convert to AQLs as a results of the program.

Principle #3: Movement should indicate a significant interaction with your brand
Is downloading the whitepaper enough to indicate a significant interaction? Just because a reader downloaded it does not guarantee he or she even read it. How many downloads do you have just sitting on your hard drive waiting to be read at some future date? We can differentiate between insignificant and significant by creating a definition for those actions.

An insignificant interaction is a download with no continuation of the interaction. Sometimes I download something simply because I want the information and somebody has offered it for free. I have absolutely no intention of purchasing anything, but the information seems interesting or useful.

A significant interaction is a download with a continuing action. For example, a well-designed program might have a secondary CTA within the downloaded asset, prompting readers to engage more deeply with your brand. As an example, after reading mare about a specific A-B Testing solution, another CTA could prompt the reader to try a free testing tool or interact with a free testing framework generator. This continuation action would indicate the reader’s real interest in your solutions.

How do we measure movement?
Waterfall movement is a matter of understanding changes to a Contact record in your MAP. This change has a number of components you need to understand in the context of many other potential changes simultaneously affecting that same Contact record. So how do you isolate the movement you need to measure? The first priority is to make sure you are capturing the required data behind the metrics you want to measure!

1.     You should systematically link your Waterfall program to your response measurement. There are a variety of ways to do this; the goal is to attribute Waterfall movement to a specific response – almost always the most recent significant interaction. This is different from Campaign attribution, in which most organizations want to attribute closed revenue across all campaigns that touched the Contact during the Lead lifecycle.
2.     You need to record your program responses so they are actionable by your Waterfall program. This means you will not only need to understand which program, but when the Contact responded.
3.     In most cases, you will also need to record how the Contact arrived at the program CTA in order to convert. This is a great basis for testing which routes to the CTA are most effective.

Change the conversation.

Optimization should be conducted with the program movement goal in mind. If we go back to our original whitepaper marketing program, we would likely make significant changes to the way we determine success if our goal is Inquiry-to-AQL conversion. IN that case, 12,500 whitepaper downloads that don’t result in AQL conversion might look like a tactical success, but is actually a strategic failure. How would we change the conversation?

First, we would change our target audience to include only Contacts in the Inquiry stage. If the goal is to convert Inquiries to AQLs, what is the purpose of sending to any other than Inquiries? Second, we would insure our infrastructure was pre-built to capture the data points necessary to measure Inquiry-to-AQL conversion. There is no sense setting a goal we cannot measure. Thirdly, we set up our A-B testing points to measure the things we can both control and change, such as traffic sources. Did outbound emails work better than banner ads, or did paid SEM work better than purchased media?

Notes:

Testing must be performed with an overall objective in mind.

You should be testing movement, not activity.

Your infrastructure must be pre-built to capture the data necessary to your metrics.

Next week, we will take a look at how your MAP platform might be doing a whole lot of processing that accomplishes nothing. With SaaS and cloud-based solutions come a plethora of cloud-based add-ons. They do all sorts of nifty stuff, but do you really need or want them? Maybe, but there is a right and wrong way to look at cloud connectors and plug-ins. We will look at them in next week’s edition: Head in the Clouds? Why Less may be More.