Case study with Alta Vista Credit Union
We ran an experiment with Alta Vista Credit Union that showed the impact a required calculator can have on overall application completion rates.
The goal
Alta Vista wanted to collect more auto loan applications.
The hypothesis
First, we looked at how many visits to the auto loan page Alta Vista was getting, and how many people were starting applications. We saw that these metrics were healthy, but the number of people completing their application was lower than we’d wanted. In a study, we found that lead capture itself can increase application completion rate. Furthermore, a set of five studies on GoodUI.org show that gradual engagement increases conversion by a median of 20%. So, we decided that requiring the user to calculate their loan during lead capture might increase the user’s motivation to complete the full application.
The a/b test
We ran an a/b test to see how requiring a loan calculation would impact application completions. In the control, people are not required to calculate their loan during lead capture. In the experiment we required the calculation before proceeding.
The results
Let’s compare the results of the control vs the experiment.
The control version (i.e., what Alta Vista started with) got a 56.83% click-through rate and the experiment got 26.54%. The control performed 114% better than the experiment.
Note: These results scored 98% statistical confidence.
The takeaways
From this a/b test, we learn two big takeaways:
- Calculations should be optional.
- Usability testing can help you learn the truth about a page. As they say, “Don’t guess, test.”