There continues to be a lot of buzz around marketing performance optimization and testing. Justifiably so. Technology is helpful with this, of course, but it’s non-technical expertise that really makes a difference.
I do a lot of performance testing with my clients, and I always follow scientific method. Here’s a primer on scientific method, along with a deep dive into one of its core elements: hypotheses.
Scientific method
Using scientific method offers a structured approach to performance optimization. Critically, it involves developing and testing hypotheses about what strategies or tactics will improve outcomes, like revenue, conversions, or engagement.
Scientific method:
- Provides data-driven insights to inform your decisions.
- Reduces risk by testing changes on a small scale before they the-coolest-youtube-banners-ive-ever-seen">are fully implemented.
- Enables continuous improvement by helping you identify not just what works, but why it works.
This approach allows you to ground you working in marketing testing in evidence, rather than assumptions.
There are eight stages to scientific method:
Researching and developing your hypothesis is a critical part of scientific method. Here are some tips and tricks for doing so successfully.
What is a hypothesis?
In this context, “hypothesis” is a fancy word for an idea of what might boost performance — but it’s also more than that.
You could say: “I want to test the color of our call-to-action buttons. Right now they are red, let’s try making them green and see if that boosts performance.”
That’s not a hypothesis. There’s no proposed explanation of why that change might be effective.
Case study: Building a hypothesis
Here’s a real-life example of how I built a hypothesis for a test years ago.
I was at my local Barnes & Noble bookstore, reading about the psychology of color in a book that I found in the discount rack.
It discussed why red was the color of stop signs — that subconsciously it could be sending a signal to stop. I realized that I had a promotional email with red buttons, because the brand colors were dark purple (it was almost black) and red.
The product was a financial advisory publication (we provided advice on what stocks to buy to enhance your financial portfolio), which is when I got to thinking about what red meant in the financial world. ‘In the red’ is a bad thing there — it means that you own more money that you have.
So, I thought, maybe the red buttons are depressing response.
But what to test against the red?
I thought of traffic lights, where red means stop and green means go. Then I thought about the financial world. Green is the color of money, which is what we were promising the stock recommendations would earn them.
The hypothesis that came from this was: “Changing the color of the CTA buttons from red to green will increase response and revenue for the reasons outlined above.”
See the difference? Tests based on hypotheses backed by sound reasoning are more likely to perform well.
Getting inspiration for hypotheses
Inspiration can come from internal or external sources. Here are a few ideas for finding inspiration for your tests.
Internal inspiration
Case study 1: Failed performance tests
Even if a test fails, you should look for learnings you can leverage in the future. For instance, years ago we pitted a control email against a version where we changed a number of different elements. Our KPI was conversions; the recipients needed to fill out a form to convert.
The control won soundly. But while the test version lagged in conversions, the click-through rate (CTR) on its top CTA button was nearly double that of the top CTA button in the control. As a result, we went on to re-test just the elements around the CTA button — including location and a message about this being an exclusive offer.
The hypothesis here: “These things appear to have increased CTR in the previous test; perhaps if we isolate them from the other elements in that test and apply them to the control, they will increase not just CTR but also conversions.”
This time the test won.
Case study 2: Campaign metrics
A lot of my client work involves multi-effort email campaigns, where we are sending a series of two or more emails to the same list over a period of time. I often get inspiration from the data, for instance…
In this example, we saw that effort 5, the last effort in the series, still generated over $18,000 — so perhaps we could garner an additional $9,000 or $10,000 by adding an effort 6. They hypothesis here “Since Effort 5 did well, we should be able to garner additional revenue by adding an effort 6.”
We also saw that effort 4 did well with $0.45 in revenue generated per email address, compared to just $0.32 for effort 3. Efforts sent earlier in the series tend to perform better. So we did a test with the hypothesis “Since earlier efforts tend to perform better, we should be able to get a lift in revenue by switching the order of efforts 3 and 4, since effort 4 generates a higher revenue-per-email-address.”
Dig deeper: Why we care about performance marketing
External inspiration
Your inbox
Your inbox can be a treasure trove of inspiration for performance tests. Do you get any testing ideas from this Walgreens email?
Here are the hypotheses that I derived from this email:
- “Including first-name personalization at the top of the email should pull more people in to read it, increase clicks. and drive more revenue.”
- “Putting the offer behind a ‘scratch off’ will increase engagements and conversion rates.”
- “Visually showing recipients their rewards status will motivate them to want to continue to earn reward points, which will increase revenue.”
Articles, blog posts, webinars, presentations and other resources
Any online resource can be a great source of inspiration; even better if it’s presenting case studies of performance tests others have done.
The last one on this list, Really Good Emails, is a swipe file. They have screenshots of more than 15,000 email messages; you can search on a number of different variables. Browsing swipe lists like this one are a great way to get inspiration for hypotheses.
Now’s the time to start performance testing, or to raise your performance testing game. I hope this primer on developing hypotheses helps.
The post Applying scientific method to performance appeared first on MarTech.
