Episode 251: The Psychology of A/B Test and Measure

Welcome to HubShots Episode 251: The Psychology of A/B Test and Measure

This edition we dive into:

• Psychology studies that have been ‘reversed’ (ie debunked)
• What this means for marketers
• And how this can motivate us to do A/B testing
• How to do A/B testing in HubSpot
• Is HubSpot testing whether they can make us do more testing?
• Be wary of your biases
• The importance of your website’s visual presentation
• GA4 guide

You can listen to this episode of the show here.

Did a colleague forward this episode to you? Sign up here to get yours every Friday.

Please forward this on to your work colleagues.

Recorded: Wednesday 30 June 2021 | Published: Friday 02 July 2021

Shot 1: Growth Thought of the Week

The Psychology of Test and Measure

I was reading this article (from January 2020) about Reversals in Psychology recently and was fascinated to learn how many psych studies have been ‘reversed’ (which is really just a kind way of saying debunked).

 You may recognise some of the studies - such as the famous ‘power pose’ finding (that launched a viral TED talk) and the debates that followed (the latest seems to be that it might actually have some validity). Most of the reversals are based on lack of replicability - that is, when other psychologists try to replicate the studies, they can’t get the same results. And thus the initial study is brought under suspicion.

In Shot 8 we drill into why this is interesting for marketers, and how we’re fortunate to not only have the tools that allow us to test and measure repeatedly but also the channels to drive enough testing to meet statistical confidence levels.

Shot 2: Quick Shots of the Week

Here’s a few quick items of interest we noticed:

• Workflows tool has twelve fresh new templates that’ll make it easier to automate your most common use cases like, nurturing first conversions, send a notification when a meeting is scheduled, sending confirmation emails for events,  celebrate anniversaries plus more.
• Sync Companies with Data Sync
• Collaboration Sidebar added in Website Pages, Landing Pages, and Blog for Marketing Professional and Enterprise users.

Shot 3: HubSpot Marketing Feature of the Week

A/B Testing Website / Landing pages

Firstly what is A/B Testing?

A/B testing (also known as split testing) is a process of showing two variants of the same web page to different segments of website visitors at the same time and comparing which variant drives more conversions.

The ability to test variations of pages and get clarity on how the page is performing is relatively easy to do in HubSpot with a little thought.

In the screenshot below you can see we started the test recently and are testing version B with less content.  Although there is not a significant amount of data you can see the time per pageview is vastly different.  This may be an indication that people who are on this page are further into their journey and are looking for specific information as this page is very location based.

Here are 2 resources to learn more about A/B testing:

1. How to Do A/B Testing from HubSpot
2. A/B Testing Guide from VWO

Shot 4: HubSpot Sales Feature of the Week

Looking at property history to understand why/how a task got created

Especially useful when you have workflows and sequences creating tasks! So our tip is to look through the property history and see how the task is being created.  

Tip: by also naming tasks appropriately is a great way to track it back!

Shot 5: HubSpot Email Marketing Feature

A/B Testing Marketing Emails

One of the simplest tests you can do to increase the open rates of your email is to test the Subject line.

What is also great is that HubSpot makes it easy to run the test.

To see more detailed instructions on how to run a test you can see step by step instructions in the knowledgebase.

Shot 6: HubSpot Test of the Week

Test Status Column

It’s always illuminating to see what HubSpot tests in their interface…

Have you (or your clients) asked why this ‘Test Status’ column was promoted to most default listing views for Landing pages and Web pages?

We wonder if HubSpot are ‘testing’ whether showing the ‘test status’ column will nudge users to do more ‘testing’ (it’s getting a bit Inception like at this point sorry).

BTW you can quickly filter your listing using the Filters:

And then:

Shot 7: Marketing Tip of the Week

Beware of your Biases

In the next shot we discuss replicability of testing.

But before we even get to that, we need to be aware of the biases that might kill a good test before it even gets started.

This sketch explains it better than I could hope to:

(via Sketchplanations)

Common biases I catch (in myself and) with clients:

• Our customers aren’t on [insert channel], they are on [insert other channel]
• Our customers don’t want to read about [insert topic], they are only interested in [insert other topic]
• We want to rank for [insert broad term]

The best way to manage these when chatting through is to simply note them as ‘assumptions’ or ‘hypotheses’ and test when appropriate(from your testing budget).

If it turns out they are true - great. You confirmed your assumptions. If it turns out they aren’t true - even better, you got some success that you wouldn’t have previously. And bonus: if you had that assumption, your competitors might have as well - and if they haven’t tested, then perhaps the field is less crowded there for you.

Shot 8: Insight of the Week

Confidence in Test and Measure

As I noted in Shot 1, this article by Gavin Leech on Reversals in Psychology is a fascinating read (it’s been doing the rounds on Twitter this past week).

(By the way, worth highlighting up front, I have no idea on the veracity of the author’s work, and whether the critique itself is reliable - but perhaps that’s not actually a problem for my point.)

(I’ll spare you the Twitter threads with people disagreeing with it, and then others disagreeing with the disagreeing - because… Twitter).

Instead, it’s the concept that intrigues me. Let’s assume that many of the studies are in fact not replicable. How does that relate to digital marketing?

The point to highlight is this: no matter what field we’re in, we can’t simply assume that ‘controlled’ tests are valid. In marketing we often read ‘studies’ by marketing ‘experts’ that highlight findings that are tempting to take on board as fact.  But we need to be thoughtful about how  we interpret these findings - use them as ideas to test sure, but don’t take them as facts.

And that’s one (of many) wonderful benefit(s) of digital marketing - we can easily set up tests to check replicability. We can take an idea and set up our own testing.

HubSpot (and many other platforms) provide sophisticated tools for testing ideas (A/B and multivariate) and reporting on them scientifically. And even better, the many channels available (email, organic, paid ads, social) allow us to economically generate enough traffic in enough forms (visitors, conversions, devices, etc) to meet statistical confidence requirements.

But perhaps there’s an even simpler takeaway from this: Have you actually run any tests?

If not, then at least make a start - the tools are there. Run a test, and then worry about whether it’s statistically confident later. After all, there’s no harm in adopting a power pose (even if it does turn out to be bollocks).

Shot 9: HubShots Throwback of the Week

Via the HubSpot product updates blog.

This time a year ago HubSpot was updating workflows to allow cloning and reordering of branch conditions. I totally missed this!

Shot 10: Resource of the Week

Google Analytics 4 Overview

Another gem from Olga, in this post she walks through a number of Google Analytics 4 reports and tips.

Shot 11: Quote of the Week

“Passion is Interest plus Mastery.”

• Tom Bilyeu

Shot 12: Bonus Links of the Week

Regular Expressions in Google Search Console

One for the nerds to bookmark for use later: a thorough guide to using RegEx in GSC by Jean-Christophe.

Shot 13: Big Shots of the Week

Website Visual Presentation is Important

Google’s John Mueller suggests that the visual presentation of a website can impact its visibility in search results.

But there’s always the exceptions eg here’s the Berkshire Hathaway site. At least it works on mobile now.

Shot 14: Follow Us on the Socials

Connect with HubShots here:

• HubShots YouTube channel 
• HubShots Spotify channel 
• HubShots iTunes
• HubShots Facebook group 
• HubShots Twitter 
• HubShots Twitter Show Notes
• HubShots Instagram 
• HubShots LinkedIn 

Connect with Ian Jacob on LinkedIn and Craig Bailey on LinkedIn 

HubShots, the podcast for marketing managers and sales professionals who use HubSpot, hosted by Ian Jacob from Search & Be Found and Craig Bailey from XEN Systems.

HubShots is produced by Christopher Mottram from Podcastily.

Please share this with colleagues - it helps us improve and reach more marketers.