Ab Testing Ideas 3 Data Driven Ways For Good Testing Hypotheses
Management Summary
What is a good testing hypothesis?
A well-thought-out test hypothesis follows the scheme:
IF… THEN… BECAUSE…
“Thoughtful” because in order to formulate such a hypothesis, the following questions must be answered in advance:
- WhichConversion goaldo I want to increase?
- Where is there a (possible)problem?
- What is aSuggested solution?
- WhichResultsDo I expect change?
Example of an A/B test hypothesis
A formulated test hypothesis could be as follows:
“IF we remove the mandatory “Address” field in the ebook download form THEN more users will fill out the form to the end BECAUSE filling it out is quicker and less personal data has to be given out.”
It is based on these basic considerations:
- Conversion goal: more downloads for my ebook
- problem: high abandonment rate in the form
- Suggested solution: Shorten form (remove address field)
- Expected result: Increase form completion rate
How do I identify problems? How do I develop testing ideas?
Sounds logical so far? Wonderful :) But how do you find potential problems/vulnerabilities on your own website, so-called “neuralgic” points? And how do you come up with suggested solutions or concrete testing ideas?
I recommend: let your data help you. Below I would like to introduce 3 data-driven ways to generate test ideas and develop hypotheses.
1. Analytics Tool: How are my pages performing?
A good place to start is to take a look at the analysis tool. For example, check the following points:
- At which step in the conversion funnel do many visitors drop off?
- Are there landing pages with particularly high bounce rates?
- Do conversion rates differ by device, demographics or source?
All of these – and of course many more – can be answered with the analytics data and can reveal possible problems.
2. Behavior Analytics: How do users behave on my site?
Analyzing user behavior on your own site offers even more insights. These include, among others:
- Scroll maps:What percentage of users scrolled to a certain area.
- Heatmaps:An overlay shows how often certain elements on a page are clicked.
- Form analysis:How many visitors fill out the individual form fields, how long do they take per field, how often is the same field filled out multiple times, etc.
- Visitor recordings:Recordings of real visitors that show how an individual user actually moves on the website.
Some testing tools already offer these analyses, for exampleVWO with its Behavior Analysis module. But there are also special tools for this behavior analysis, including Crazy Egg and Hotjar, to name just two.
3. User Tests: What do my visitors think?
Can be particularly insightfulUser or usability testsbe. Here, people from the desired target group are given specific tasks on the website. By “thinking out loud” the users explain their approach, their thoughts and possible ambiguities.
The big advantage of user tests: In the best case scenario, you not only identify stumbling blocks, but are also provided with suitable solution suggestions. From the statement of a tester“The site doesn’t seem trustworthy to me. I don’t have any known test seals.”For example, one could derive an optimization idea from the use of quality seals as a trust enhancer.
Bonus: Feedback from customer service
Another great way to identify problems is to interview your customer service colleagues. You are in daily contact with your website visitors and probably know better than anyone where common confusion arises.
Conclusion – without analysis there is no good test hypothesis
I can’t repeat it often enough: testing without a well-founded hypothesis is usually worthless and anything but professional. Invest the time in analysis, your results will be even better.
