Case study with Denver Community Credit Union
We ran an experiment that showed the impact a product page layout can have on overall application completion rates.
Denver Community Credit Union wanted to increase the number of auto loan applications they get.
We looked at the auto loan funnel and found that the best opportunity was to improve the click-through rate on the auto loan page itself. Heatmaps showed us that specific elements on the page got more engagement than others, such as the rates, calculator and frequently asked questions. Our hypothesis was that eliminating elements with less engagement and rearranging more popular content toward the top of the page would yield more applications.
The a/b test
We ran an a/b test to see how our changes to the layout would impact applications. In the control, we left the contents of the auto loan page as-is. In the experiment, we added a link to see rates in the hero next to the primary call to action. We also brought the “Here’s how it works” and “Frequently Asked Questions” sections higher up on the page. Then, we reduced the number of frequently asked questions to showcase only the most popular. We also removed the information about getting financed at a dealership. Finally, we added repeated calls to action in the middle and at the end of the page.
Let’s compare the results of the control vs the experiment.
The control version (i.e., what Denver Community started with) got an overall conversion (i.e., amount of completed applications) of 5.53% and the experiment got 9.19% (that’s +66% higher).
Note: These results scored 95.36% statistical confidence.
From this a/b test, we learn two big takeaways:
- The experiment layout we created can produce a 9% application completion rate (hint: this is very good!).
- Usability testing can help you learn the truth about a page. As they say, “Don’t guess, test.”