Building a culture of Experimentation
by Byron, director of PM at Optimizely
Why Experimentation Matter?
How do you know that a product, or a product change, is successful or not? Different people, different places, different experiences, it’s impossible to control for all of the externalities that make it difficult to explain the story your data is trying to tell you.
Experimentation is the only way to systematically determine the effect that your product is going to have.
Building these “Six Habits of Experimentation” will help your company gain usable data to make better decisions and improve behavior quickly.
The Six Habits of Experimentation
1. Culture starts from the top
So PMs must be available and attentive to their team members. This shows that ideas are valued, giving them the confidence to work harder knowing they will be heard.
2. teams can “protect” their ideas by framing them as experiments
This changes the conversation from risk and investment to testing and data, allowing teams to have more opportunities to capitalize on promising ideas.
3. Leaders should always celebrate wins and losses
While wins increase profit or have other positive effects, losses allow for teams to re-evaluate, re-test, and find a solution.
4. Teams must hire for an experimentation mindset
This can be achieved by requiring interviewees to present a data set, and then, look for the applicant who can best use context to tell a story about the truth of the data.
5. Choosing the right metrics for your experiments
Experimentation allows teams to remove guesswork and objectively make decisions, so, you have to make sure your metrics are suitable for your optimization goals.
6. The more experiments that are conducted, the more ideas that can see the light of day
A few ways to achieve a higher experiment velocity is through investing in education and training, holding experiment review meetings, simplifying experiment launches and their requirements, and automating experiment tracking. With these actions, you will see an increase in experiments and an increase in successful decisions.
If you want to listen to Byron Jones’ full ProductCon SF talk, watch it here.
