There is a lot of talk about experimentation in startups.

There is violent agreement about its importance to momentum. The question is how do you tell the difference between interesting experiments and those which help advance company missions.

I decided to write about this today because a colleague recently noticed that I use a tactic I learned when I was a young soldier. I didn’t realise I was doing it and it took someone familiar with SMEAC (not surprisingly a military acronym pronounced smee-akk) to call me on it.

I will dissect SMEAC as it relates to experimentation in startups later in the post. Ahead of that though, I think it helps to understand how serious you and your culture is about experimentation.

Some know experimentation better than others

There are founders who know how to structure, execute and learn from experiments. They use experimentation as a means to accelerate knowledge and keep their instincts in check. I put my friends Nic Hodges, Gautam Mishra and Dee Deng in this camp.

And then there are those who talk about it without really knowing how it all works. Break these founders down further and two more groups appear.

The first are founders who really want to learn how to experiment and create a culture of experimentation. They have read everything there is about experimentation speed, how to identify the riskiest assumptions and how to pare down bloated tests. Although their level of experimentation expertise is academic, these founders are walking the talk. Launching experiments as often as possible and trying to gather data, they fossick for insights and learn as they go.

That was and to some extent still is me.

The second group love the idea of experimentation but hate the process. They have decided to stick with their intuition but at some point, their perspective changes and they turn to experiments to learn.

And those who don’t often end up needing to find a job.

Mission Driven Vs Interesting Experimentation

When I move past entrepreneurs who understand experimentation, I often find less experienced founders running rafts of experiments they find interesting.

And ‘interesting’ can be dangerous for the simple reason that interesting equals subjective.

Here’s an example. You discover something interesting about a customer. You share it with your team or community and do an incredible job of selling it to them. They buy what you’re selling and before long a feature or change is taking place just because you found it interesting.

No supporting data.

No objective corroborating insights.

Just an observation.

And now, thanks to your selling skills, a wholesale distraction for your already resource-constrained team.

I have caused these types of distractions in the past as have most first-time founders.

With the benefit of hindsight, I think there are four questions you can ask your team to make sure experiments remain mission focused.

Perhaps most importantly, these questions are self-reinforcing in a way that encourages teams to maintain mission-focused experimentation.   

Q1. What number are you trying to move?

I owe this one to Gautam. He always asks it and rightly so. This question helps people focus on the challenge to be solved or the opportunity to be captured. Whether you’re trying to understand a behaviour, accelerate growth or stem attrition, this question helps clear away the noise that can cloud and overcomplicate mission-critical objectives.

And there is a 1:1 relationship between the number to be moved and the experiment. In other words, if you need to move two numbers, each number needs its own experiment.

Q2. What is your hypothesis?

A hypothesis is your best guess with limited data.

I provide this simple definition whenever a team member or founder tenses up because they think they need to present a more scientific explanation.

That said, there needs to be limited data to accompany the best guess.  Otherwise what you’ve got is a subjective observation. If that’s what you’re presented with, ask for limited data so you can have a conversation about the future instead of a conversation about religion.

This question also builds on the first one.

Q3. How does this experiment support the core growth metric?

The core growth metric of most startups is (and should be) rate of revenue growth. Revenue solves a lot of problems and the faster it grows, the more leverage a company possesses. And this is the truest when young companies are seeking investment from angel investors and venture capital firms who seek more and more concrete evidence that founders are onto something.

I can almost hear people thinking that growth comes in different forms. That is correct and there is a time when investment-fuelled growth can be traded-off against revenue growth. But the days of companies raising rounds of financing in the hope that one day they will break even are over.

Moving the dial of a mission-critical metric using hypothesis-driven experimentation that ultimately ties to revenue growth can only put you on a winning path.

This then leaves the final question.

Q4. How does mission-driven experimentation become part of your culture?

Here is where SMEAC comes in. Also known as the ‘Five Paragraph Order’, SMEAC is the standard way in which teams in the military formulate and distribute instructions for exercises and missions. You can learn more about SMEAC here

In the context of building businesses, SMEAC presents a replicable format that addresses points of a plan that are often missed due to the cut and thrust of startup.

Here is the format. The descriptors are how I apply SMEAC to startup experimentation.

  1. Situation: This is the context. The background for why the team is thinking about this experiment is discussed here. The number you are trying to move, the hypothesis, and how this ties into the core growth metric are also mentioned here as are potential challenges and unknown factors.
  2. Mission: Who, What, Where, When, and Why about the experiment. In other words, if this mission statement is clear, specific, time-bound and measurable, you’re on the right track. For good measure, the mission statement is said twice.
  3. Execution: This is the ‘how’ and it can be the lengthiest part of SMEAC. Team members are presented with their tasks, contingencies are discussed and the desired end-state is articulated.
  4. Admin & Logistics:  Usually the tools and communication methods (including regular reporting) to be used during the experiment.
  5. Command & Signal: Who is accountable for experiment success and how they will feed progress and results back to the company.

For the record, my office isn’t a war-room.

I don’t have my team standing around mud maps and I don’t often use each heading (ie Execution) as I walk through each section.

This is a framework to help establish clarity on why we are investing time in an experiment.

By the time you reach SMEAC, three other important questions have been answered. This step is all about closing the loop on a hypothesis that (hopefully) increases your rate of learning.

One last thing…

If you are serious about increasing the speed at which your team learns, mission-driven experimentation is essential. And if you feel your team is not quite hitting the mark with their experimentation, think about asking them these four questions.

Finally, SMEAC may be new to you or you might think its relevance is restricted to the military. I can assure you that it works well in nearly all turbulent environments, largely due to how easy it is learned, understood and transmitted between leaders.

Like everything in a startup, getting good at experimentation takes practice and clarity on why you are testing in the first place. I hope you find these frameworks useful.