experiment-driven-development

Experiment-Driven Development In A Nutshell

Test-Driven Development (TDD) and Behavior-Driven Development (BDD) are popular agile development techniques. However, they don’t measure application usage or provide guidance on gaining feedback from customers. Experiment-Driven Development (EDD) is a scientific, fact-based approach to software development using agile principles.

Understanding Experiment-Driven Development

While TDD and BDD help developers enhance code quality and ensure that it behaves according to spec, EDD helps identify the features that should be developed. In other words, what will become the spec.

EDD is driven by split A/B testing, where a baseline (control) sample is compared to several single-variable samples to determine which of the two choices improves response rates. 

This form of feedback collection avoids the need to conduct user surveys, which are often time-consuming for both parties and can be prone to bias.

Implementing Experiment-Driven Development

To implement EDD, it is a matter of following these four steps:

Start with a hypothesis

Instead of beginning with a user story, the project team starts by defining a hypothesis related to customers, problems, solutions, value, or growth.

For example, a growth hypothesis may be “A virtual shoe fitting station in every store will increase shoe sales by 30%.” 

Identify the experiment

In the second step, take the highest-priority hypothesis and define the smallest experiment that will prove or disprove it.

The shoe store may decide to install a virtual fitting station in five stores to begin with and measure the impact on sales.

Run the experiment

This may include creating a minimum viable product (MVP) and then measuring progress based on validated learning from the end-user.

minimum-viable-product
As pointed out by Eric Ries, a minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort through a cycle of build, measure, learn; that is the foundation of the lean startup methodology.

Here, many businesses choose to run experiments based on the Build/Measure/Learn (MVPe) loop. 

product-market-fit
Marc Andreessen defined Product/market fit as “being in a good market with a product that can satisfy that market.” According to Andreessen, that is a moment when a product or service has its place in the market, thus enabling traction for the company offering that product or service.

Debrief

For example, what are the observations?

How were the validated learnings used? Would more time spent on planning have helped?

Based on the results, the team may choose to pivot to a new hypothesis.

Alternatively, they may choose to persevere with the current hypothesis or discard it entirely and move to the next one.

Experiment-Driven Development Benefits

When a business incorporates EDD to complement an existing approach such as TDD or BDD, it can realize several benefits.

These include:

Structure

EDD allows project teams to ask and answer questions in a structured, measurable process.

Since ideas are validated by hypotheses, teams also avoid the testing of ideas simply to validate individual egos or hunches. 

Versatility

Although its scientific foundations may suggest otherwise, Experiment-Driven Development can be used across any business in any industry.

It is not specifically designed for use by R&D teams. 

Objectivity and efficiency

All agile methodologies dictate that value to the end-user is the primary goal.

However, the hypothesis-driven approach of EDD forces teams to define value through validated learning and not assumption alone.

Efficiency is also increased by building an MVP instead of focusing on superfluous features that provide little benefit to the end-user.

Key takeaways

  • Experiment-Driven Development is a hypothesis-driven approach to software development that is based on fact.
  • Experiment-Driven Development incorporates A/B testing, where a baseline sample is compared to a single-variable sample to determine which sample delivers a better outcome. This allows the business to formulate, test, and evaluate hypotheses.
  • Experiment-Driven Development complements approaches such as TDD and BDD, but it does not replace them. EDD can be used in any industry or department as an efficient and (most importantly) objective means of agile software development.

What are the steps to implement experiment-driven development?

The steps to implement experiment-driven development are:

What are the benefits of experiment-driven development?

The benefits of experiment-driven development are:

Connected Agile Frameworks

AIOps

aiops
AIOps is the application of artificial intelligence to IT operations. It has become particularly useful for modern IT management in hybridized, distributed, and dynamic environments. AIOps has become a key operational component of modern digital-based organizations, built around software and algorithms.

Agile Methodology

agile-methodology
Agile started as a lightweight development method compared to heavyweight software development, which is the core paradigm of the previous decades of software development. By 2001 the Manifesto for Agile Software Development was born as a set of principles that defined the new paradigm for software development as a continuous iteration. This would also influence the way of doing business.

Agile Project Management

agile-project-management
Agile project management (APM) is a strategy that breaks large projects into smaller, more manageable tasks. In the APM methodology, each project is completed in small sections – often referred to as iterations. Each iteration is completed according to its project life cycle, beginning with the initial design and progressing to testing and then quality assurance.

Agile Modeling

agile-modeling
Agile Modeling (AM) is a methodology for modeling and documenting software-based systems. Agile Modeling is critical to the rapid and continuous delivery of software. It is a collection of values, principles, and practices that guide effective, lightweight software modeling.

Agile Business Analysis

agile-business-analysis
Agile Business Analysis (AgileBA) is certification in the form of guidance and training for business analysts seeking to work in agile environments. To support this shift, AgileBA also helps the business analyst relate Agile projects to a wider organizational mission or strategy. To ensure that analysts have the necessary skills and expertise, AgileBA certification was developed.

Business Model Innovation

business-model-innovation
Business model innovation is about increasing the success of an organization with existing products and technologies by crafting a compelling value proposition able to propel a new business model to scale up customers and create a lasting competitive advantage. And it all starts by mastering the key customers.

Continuous Innovation

continuous-innovation
That is a process that requires a continuous feedback loop to develop a valuable product and build a viable business model. Continuous innovation is a mindset where products and services are designed and delivered to tune them around the customers’ problem and not the technical solution of its founders.

Design Sprint

design-sprint
A design sprint is a proven five-day process where critical business questions are answered through speedy design and prototyping, focusing on the end-user. A design sprint starts with a weekly challenge that should finish with a prototype, test at the end, and therefore a lesson learned to be iterated.

Design Thinking

design-thinking
Tim Brown, Executive Chair of IDEO, defined design thinking as “a human-centered approach to innovation that draws from the designer’s toolkit to integrate the needs of people, the possibilities of technology, and the requirements for business success.” Therefore, desirability, feasibility, and viability are balanced to solve critical problems.

DevOps

devops-engineering
DevOps refers to a series of practices performed to perform automated software development processes. It is a conjugation of the term “development” and “operations” to emphasize how functions integrate across IT teams. DevOps strategies promote seamless building, testing, and deployment of products. It aims to bridge a gap between development and operations teams to streamline the development altogether.

Dual Track Agile

dual-track-agile
Product discovery is a critical part of agile methodologies, as its aim is to ensure that products customers love are built. Product discovery involves learning through a raft of methods, including design thinking, lean start-up, and A/B testing to name a few. Dual Track Agile is an agile methodology containing two separate tracks: the “discovery” track and the “delivery” track.

Feature-Driven Development

feature-driven-development
Feature-Driven Development is a pragmatic software process that is client and architecture-centric. Feature-Driven Development (FDD) is an agile software development model that organizes workflow according to which features need to be developed next.

eXtreme Programming

extreme-programming
eXtreme Programming was developed in the late 1990s by Ken Beck, Ron Jeffries, and Ward Cunningham. During this time, the trio was working on the Chrysler Comprehensive Compensation System (C3) to help manage the company payroll system. eXtreme Programming (XP) is a software development methodology. It is designed to improve software quality and the ability of software to adapt to changing customer needs.

Lean vs. Agile

lean-methodology-vs-agile
The Agile methodology has been primarily thought of for software development (and other business disciplines have also adopted it). Lean thinking is a process improvement technique where teams prioritize the value streams to improve it continuously. Both methodologies look at the customer as the key driver to improvement and waste reduction. Both methodologies look at improvement as something continuous.

Lean Startup

startup-company
A startup company is a high-tech business that tries to build a scalable business model in tech-driven industries. A startup company usually follows a lean methodology, where continuous innovation, driven by built-in viral loops is the rule. Thus, driving growth and building network effects as a consequence of this strategy.

Kanban

kanban
Kanban is a lean manufacturing framework first developed by Toyota in the late 1940s. The Kanban framework is a means of visualizing work as it moves through identifying potential bottlenecks. It does that through a process called just-in-time (JIT) manufacturing to optimize engineering processes, speed up manufacturing products, and improve the go-to-market strategy.

Rapid Application Development

rapid-application-development
RAD was first introduced by author and consultant James Martin in 1991. Martin recognized and then took advantage of the endless malleability of software in designing development models. Rapid Application Development (RAD) is a methodology focusing on delivering rapidly through continuous feedback and frequent iterations.

Scaled Agile

scaled-agile-lean-development
Scaled Agile Lean Development (ScALeD) helps businesses discover a balanced approach to agile transition and scaling questions. The ScALed approach helps businesses successfully respond to change. Inspired by a combination of lean and agile values, ScALed is practitioner-based and can be completed through various agile frameworks and practices.

Spotify Model

spotify-model
The Spotify Model is an autonomous approach to scaling agile, focusing on culture communication, accountability, and quality. The Spotify model was first recognized in 2012 after Henrik Kniberg, and Anders Ivarsson released a white paper detailing how streaming company Spotify approached agility. Therefore, the Spotify model represents an evolution of agile.

Test-Driven Development

test-driven-development
As the name suggests, TDD is a test-driven technique for delivering high-quality software rapidly and sustainably. It is an iterative approach based on the idea that a failing test should be written before any code for a feature or function is written. Test-Driven Development (TDD) is an approach to software development that relies on very short development cycles.

Timeboxing

timeboxing
Timeboxing is a simple yet powerful time-management technique for improving productivity. Timeboxing describes the process of proactively scheduling a block of time to spend on a task in the future. It was first described by author James Martin in a book about agile software development.

Scrum

what-is-scrum
Scrum is a methodology co-created by Ken Schwaber and Jeff Sutherland for effective team collaboration on complex products. Scrum was primarily thought for software development projects to deliver new software capability every 2-4 weeks. It is a sub-group of agile also used in project management to improve startups’ productivity.

Scrum Anti-Patterns

scrum-anti-patterns
Scrum anti-patterns describe any attractive, easy-to-implement solution that ultimately makes a problem worse. Therefore, these are the practice not to follow to prevent issues from emerging. Some classic examples of scrum anti-patterns comprise absent product owners, pre-assigned tickets (making individuals work in isolation), and discounting retrospectives (where review meetings are not useful to really make improvements).

Scrum At Scale

scrum-at-scale
Scrum at Scale (Scrum@Scale) is a framework that Scrum teams use to address complex problems and deliver high-value products. Scrum at Scale was created through a joint venture between the Scrum Alliance and Scrum Inc. The joint venture was overseen by Jeff Sutherland, a co-creator of Scrum and one of the principal authors of the Agile Manifesto.

Read Also: Business Models Guide, Sumo Logic Business Model, Snowflake

InnovationAgile MethodologyLean StartupBusiness Model

InnovationAgile MethodologyLean StartupBusiness Model InnovationProject Management.

Main Free Guides:

Scroll to Top
FourWeekMBA