Lots of people have been asking for a screencast on how to use SaaS Compass to find your key metric. This short screencast (4:30) shows how to fill in the spreadsheet and analyze the created forecasts.
SaaS Compass shows customer life-time values and CLTVs are something you’ll want to keep to yourself, so I’m not using real business data here. As Rob Walling said in his 2012 MicroConf talk “If your competitors know your customer life-time value, they’ll eat you alive”.
If you just want to see how to read the forecasts, start watching at 2:15.
Find Your Key Metric with Small Targets, Verify the Results with Large Targets
There’s one thing that I want to point out from the video. I’m setting small targets to both conversion rate and churn and there’s a pitfall in doing that.
It is easier to compare the metrics and learn about their dynamics when the targets are small. But when I set out to improve those two metrics, I set BIG targets.
I set big targets because I can’t verify the small targets. I have no way of knowing if the results happened because I did something, or just because of random chance.
This old blog post by Jason Cohen presents an easy formula to check if your A/B testing results are statistically significant. It’s a good read, and not only because of the adorable Hammy the Hamster. In the post he says: “when N is small it’s hard — or even impossible — to be statistically significant”.
Both churn and conversion rate are metrics that easily range in 1%-10%. Especially if your customer base size and traffic are still small, you’ll end up having targets that include moving just small amount of persons around. Often just the error margin makes a bigger difference.
So plan big, or at least keep that statistic significance formula at hand when verifying your results.