The Web is the greatest and most accessible research lab that ever existed. The ability to rapidly deploy and iterate and reiterate (often in randomized experiments) at mass scale is one of the most significant platforms for innovation in the field of technology today.
In the pre-digital world, we test marketed an idea or feature with a working group. In the digital world, you have the opportunity to think differently and create experiments at massive scale.
Large Internet companies are already doing this. Amazon, eBay, Etsy, Facebook, Google, Groupon, LinkedIn, Microsoft, Netflix, Yahoo, and Zynga are using Experimentation at Scale to drive innovation and improve their online products and services.
Whether you have 10 or 100 million subscribers, you can add a new button or feature to your website and see what happens. Instantly, all your subscribers can see it. Don’t like where it’s going? Turn it off. Instantly.
Experimentation at Scale helps organizations make data-driven decisions about product development based on consumer behavior. It’s especially valuable when it’s used with Agile Development. Highly scalable systems can be engineered to process data, support millions of users, and run hundreds of simultaneous experiments. This is a new approach to engineering; standard testing and debugging techniques don’t belong here. It’s about immediate causation and correlation.
The goal is to conduct a large number of experiments, rapidly, to evaluate and learn at a faster pace and a larger scale. Running thousands of experiments at the same time can produce results that typical approaches—based on iterative product releases, fixes, and customer feedback—do not.
The outcomes are focused around larger effects based on data and the sheer number of users involved. Experimentation at Scale can make huge leaps forward in innovation. Then there’s the bottom line. Imagine if just a percentage of a percentage of revenue was increased by experimentation, and then multiply this by hundreds of thousands. It could mean increases in revenue by hundreds of millions of dollars.
The key to effective Experimentation at Scale is the ability to admit that mistakes will happen, to iterate, and consider each idea.
Digital experimentation has awesome potential. As the boundaries between reality and digital blur, Experimentation at Scale will play a critical role in the evolution of science, technology, and medicine. And it can change the way you do business.
In my next post, I’ll look at how a modernized platform approach will enable organizations to harness the potential of digital while capitalizing on the opportunities it presents.
For more thoughts ON DIGITAL, download the book.
