When you’re running a lean experiment, one of the key decision points is setting your minimum success criteria: the breakpoint at which you consider the experiment to have validated or invalidated your hypothesis.
Make sure you explicitly set the criteria before you run the experiment, and make a record of it. It’s too easy to fudge the numbers later and rob yourself of any valuable insight.
There are two methods I recommend for determining the minimum success criteria for your experiments.
The ‘business school’ method of setting minimum success criteria is to put together your pro forma spreadsheet with projections of what numbers you need to hit for the business to be financially viable and then reverse engineer the conversion rate you need to hit to make those numbers.
The approach that I find tends to work better, particularly for companies that are in the early stages, is to ask your team. What conversion rate would you have to see for you and your team to still be excited about the opportunity? This is the half-life of enthusiasm approach (H/T Frank DiMeo for the terminology).
In early-stage companies (we’re talking pre-product/market fit), funding is tight, but usually not the limiting factor. Enthusiasm and the will to continue is usually in much shorter supply. So, optimize for your scarcest resource. If it’s truly funding, make the numbers work. If it’s enthusiasm, make sure the problem opportunity still excites the team.