Evidence-Based Portfolio Management (E-B PfM) applies agile principles to the process of deciding where to invest funds for maximum benefit to the business. Traditional portfolio management tends to focus on activities and outputs, with less consideration given to outcomes that are often poorly defined.
Element
Description
Concept Overview
Evidence-Based Portfolio Management is an approach that relies on empirical data and evidence to make informed decisions about project portfolios. It emphasizes data-driven insights over intuition or gut feelings, helping organizations allocate resources effectively and achieve strategic goals.
Empirical Data
Empirical data forms the foundation of Evidence-Based Portfolio Management. It involves collecting and analyzing real-world data, such as project performance metrics, customer feedback, and market trends, to gain actionable insights and make informed decisions.
Decision-Making Framework
Evidence-Based Portfolio Management provides a structured decision-making framework. It involves defining clear objectives, collecting relevant data, conducting rigorous analysis, and making portfolio decisions based on empirical evidence rather than subjective judgment.
Continuous Improvement
Continuous improvement is a fundamental aspect of this approach. Organizations use ongoing data collection and analysis to refine their project portfolios continuously. By learning from past experiences, they optimize resource allocation and enhance portfolio performance over time.
Implications
Evidence-Based Portfolio Management has several implications: – Informed Decision-Making: Relies on data for objective decision-making. – Resource Allocation: Allocates resources based on data-driven priorities. – Risk Mitigation: Identifies and mitigates risks using empirical evidence. – Performance Optimization: Ensures continuous portfolio improvement.
Benefits
– Informed Decisions: Enhances decision-making with data-driven insights. – Resource Efficiency: Optimizes resource allocation for better results. – Risk Reduction: Identifies and addresses risks proactively. – Strategic Alignment: Aligns portfolios with organizational goals effectively.
Drawbacks
– Data Quality: Relies heavily on data, requiring high data quality and accuracy. – Implementation Challenges: May require changes in organizational culture and processes. – Resource Intensive: Data collection and analysis can be resource-intensive. – Resistance to Change: Stakeholders may resist data-driven decisions.
Use Cases
Evidence-Based Portfolio Management is applied in various domains: – Project Management: Optimizes project portfolios for better outcomes. – Product Development: Enhances product portfolios based on customer feedback and market trends. – Investment Management: Informs investment decisions using data-driven insights. – Risk Management: Identifies and mitigates risks in portfolios.
Table of Contents
Understanding Evidence-Based Portfolio Management
Annual budgeting processes, for example, restrict the ideation process to the point where ideas falling outside of budgetary constraints are discarded entirely.
When managers are asked to estimate the cost of delivering a solution, these estimates often come attached with several caveats. These caveats are typically ignored in favor of meeting hard, non-negotiable schedules and deadlines.
Ultimately, this results in funding decisions being made by people who are far removed from the actual work. These rather optimistic decisions cause the scope of the work to expand once knowledgeable individuals are recruited, resulting in budget blowouts and delays.
Evidence-Based Portfolio Management applies lean and agile principles to the challenge of deciding where to invest funds for maximum ROI. By enabling businesses to quickly test ideas and rapidly deliver benefits in small increments, E-B PfM avoids the bloated, non-collaborative, and over-specified aspects of traditional portfolio management.
Indeed, E-B PfM replaces expensive and inefficient project meetings with direct evidence to continuously evaluate and adapt strategy where necessary.
Principles of Evidence-Based Portfolio Management
The structure, roles, responsibilities, and processes of every organization are different. E-B PfM is thus based on seven general principles that form an agile philosophy.
This philosophy can be used to determine how the business identifies opportunities and considers which of those opportunities to pursue. It also strongly advocates the role of experimentation in guiding whether to increase, continue, or cease investment in those opportunities.
Following is a look at each of the seven principles:
1 – Separate budgeting for capacity from investing for innovation
An organization that takes on new work must add new teams or enable existing teams to be more effective. E-B PfM recognizes that there will always be more ideas than teams, so proper portfolio management is largely about deciding what not to work on.
2 – Make the best decision based on the evidence available
Evidence is often incomplete and unreliable, but an empirical approach makes allowances for this fact by testing assumptions and seeking better evidence. When making important decisions, the amount of money invested should be proportional to the quality of the evidence.
3 – Invest in improving business impacts using hypotheses; don’t just fund activity
Cost, schedule, and output are three variables that drive traditional portfolio management. But each has little relevance to value. E-B PfM instead equates value with delivering products and services that help customers achieve better outcomes.
4 – Continuously (re)evaluate and (re)order opportunities
As new opportunities are discovered, the relative attractiveness of existing opportunities will fluctuate. This means that the business will need to refine the list of opportunities according to their relative importance and invest accordingly. Relative importance should always be evaluated when new evidence comes to hand.
5 – Minimize avoidable loss
To minimize loss, the business must determine which ideas will not work. Project teams can perform experiments designed to actively prove that certain solutions don’t work, thereby providing direction for future development.
For example, a company that is unsure of how a new product feature will be received can run a customer focus-group to gauge initial reaction.
In keeping with agile principles, solution viability should be tested in the simplest, fastest, and most cost-effective way possible.
6 – Let teams pull work as they have capacity
When a business attempts to work on ideas for which it does not have the capacity, it creates a Work In Process (WIP). A high amount of WIP causes a loss of efficiency, project delays, and impedes the flow of work.
By ensuring that teams pull the most valuable opportunity only once they are ready, WIP is reduced. Free to make their own decisions and focus on one opportunity at a time, the motivation and subsequent performance of the project team increases.
7 – Improve status reporting with increased engagement and transparency
Traditionally, portfolio investment is monitored through status reporting that lacks transparency because it is people outside the team that prepare the reports. By replacing this uninformed and subjective approach with E-B PfM, status reports are based on frequent, iterative product deliveries that contain useful, actionable data.
Updated estimates of unrealized value and measures of current value are two such examples. Both help project teams reliably verify assumptions and allow them to reassess priorities with respect to organizational goals and strategies.
Case Studies
Tech Company: Prioritizing New Features for a Software Product
Challenge: A tech company is developing a software product and needs to decide which new features to prioritize for the next release.
E-B PfM Thinking Process:
Separate budgeting for capacity from investing for innovation:
The company identifies that it has a limited development team capacity.
They decide to allocate a specific budget for innovation, ensuring they don’t overburden their team.
Make the best decision based on the evidence available:
The product team collects user feedback, conducts market research, and analyzes competitor features.
They prioritize features based on the quality of evidence, focusing on those that address user needs and align with their product strategy.
Invest in improving business impacts using hypotheses; don’t just fund activity:
Instead of simply allocating resources to feature development, the company formulates hypotheses about how each feature will impact user satisfaction and revenue.
They fund features that have well-defined hypotheses with the potential for significant business impact.
Continuously (re)evaluate and (re)order opportunities:
As they collect more user data and feedback, the company regularly reevaluates the priority of features.
Features that show promising results are invested in further, while those with less impact are deprioritized.
Minimize avoidable loss:
To minimize potential loss, the company conducts A/B testing on new features.
They actively disprove ideas that do not show the expected improvements in user engagement or revenue.
Let teams pull work as they have capacity:
The development team pulls in new feature work only when they have the capacity to do so.
This reduces work in process and ensures that features are developed with a focus on quality and thorough testing.
Improve status reporting with increased engagement and transparency:
The company replaces traditional progress reports with regular product releases.
These releases contain actionable data on user engagement, allowing teams to make informed decisions and adapt their strategy.
Outcome: By applying E-B PfM principles, the tech company makes data-driven decisions, prioritizes features based on user needs and business impact, and maintains transparency throughout the development process. This approach results in a software product that continually evolves to meet user expectations and drive business growth.
Challenge: A financial institution manages multiple investment portfolios for its clients and needs to optimize the allocation of assets to maximize returns.
E-B PfM Thinking Process:
Separate budgeting for capacity from investing for innovation:
The institution recognizes that each portfolio has a limited capacity for diverse assets.
They allocate separate budgets for portfolio management and innovative investment strategies.
Make the best decision based on the evidence available:
The institution analyzes historical investment data, market trends, and economic indicators.
They make investment decisions based on the quality and reliability of available evidence.
Invest in improving business impacts using hypotheses; don’t just fund activity:
Instead of blindly investing in various assets, the institution formulates hypotheses about the potential returns and risks of each investment.
They allocate funds to investments with well-defined hypotheses and expected positive impacts.
Continuously (re)evaluate and (re)order opportunities:
As market conditions change, the institution regularly reevaluates the composition of each portfolio.
They adjust asset allocations based on new evidence and changing market dynamics.
Minimize avoidable loss:
To minimize potential losses, the institution actively manages risk through diversification and hedging strategies.
They actively disprove high-risk investment ideas through scenario analysis and stress testing.
Let teams pull work as they have capacity:
Portfolio managers make investment decisions based on portfolio capacity and asset availability.
This ensures that each portfolio is managed efficiently, minimizing unnecessary asset overlap.
Improve status reporting with increased engagement and transparency:
The institution provides clients with transparent and real-time access to their portfolio performance.
Clients can see the evidence-based decisions behind asset allocations and investment strategies.
Outcome: By applying E-B PfM principles, the financial institution optimizes its investment portfolios, maximizes returns while managing risks, and provides clients with a transparent and data-driven approach to wealth management.
Key takeaways
Evidence-Based Portfolio Management is an empirical, principles-based approach to agile portfolio management.
Evidence-Based Portfolio Management replaces the rigid and over-specified nature of traditional portfolio management with collaboration, autonomy, and continuous improvement.
Evidence-Based Portfolio Management is based on seven principles. These combine to allows management approaches to be adapted to the specific needs of any business.
Key Highlights
Agile Approach to Investment: E-B PfM applies agile principles to investment decisions, emphasizing evidence, experimentation, and continuous improvement.
Outcome-Focused: Unlike traditional approaches, E-B PfM prioritizes outcomes and results over activities and outputs.
Challenges with Traditional Approaches: Traditional portfolio management can lead to misaligned expectations, budget overruns, and scope expansions due to optimistic estimates and lack of real-time evidence.
Seven Principles of E-B PfM:
Separate budgeting for capacity from investing for innovation.
Make decisions based on available evidence.
Invest in outcomes using hypotheses, not just activity.
Continuously re-evaluate and re-order opportunities based on new evidence.
Minimize loss by actively disproving unproductive ideas through experimentation.
Allow teams to pull work based on capacity to reduce Work In Process.
Improve status reporting through transparency and engagement.
Benefits of E-B PfM:
Replaces rigid practices with collaboration and agility.
Adaptable principles for different business needs.
Informed Investment Decisions: E-B PfM helps organizations make investment decisions based on empirical evidence, reducing risks and enhancing outcomes.
AIOps is the application of artificial intelligence to IT operations. It has become particularly useful for modern IT management in hybridized, distributed, and dynamic environments. AIOps has become a key operational component of modern digital-based organizations, built around software and algorithms.
Agile started as a lightweight development method compared to heavyweight software development, which is the core paradigm of the previous decades of software development. By 2001 the Manifesto for Agile Software Development was born as a set of principles that defined the new paradigm for software development as a continuous iteration. This would also influence the way of doing business.
Agile Program Management is a means of managing, planning, and coordinating interrelated work in such a way that value delivery is emphasized for all key stakeholders. Agile Program Management (AgilePgM) is a disciplined yet flexible agile approach to managing transformational change within an organization.
Agile project management (APM) is a strategy that breaks large projects into smaller, more manageable tasks. In the APM methodology, each project is completed in small sections – often referred to as iterations. Each iteration is completed according to its project life cycle, beginning with the initial design and progressing to testing and then quality assurance.
Agile Modeling (AM) is a methodology for modeling and documenting software-based systems. Agile Modeling is critical to the rapid and continuous delivery of software. It is a collection of values, principles, and practices that guide effective, lightweight software modeling.
Agile Business Analysis (AgileBA) is certification in the form of guidance and training for business analysts seeking to work in agile environments. To support this shift, AgileBA also helps the business analyst relate Agile projects to a wider organizational mission or strategy. To ensure that analysts have the necessary skills and expertise, AgileBA certification was developed.
Agile leadership is the embodiment of agile manifesto principles by a manager or management team. Agile leadership impacts two important levels of a business. The structural level defines the roles, responsibilities, and key performance indicators. The behavioral level describes the actions leaders exhibit to others based on agile principles.
The andon system alerts managerial, maintenance, or other staff of a production process problem. The alert itself can be activated manually with a button or pull cord, but it can also be activated automatically by production equipment. Most Andon boards utilize three colored lights similar to a traffic signal: green (no errors), yellow or amber (problem identified, or quality check needed), and red (production stopped due to unidentified issue).
Bimodal Portfolio Management (BimodalPfM) helps an organization manage both agile and traditional portfolios concurrently. Bimodal Portfolio Management – sometimes referred to as bimodal development – was coined by research and advisory company Gartner. The firm argued that many agile organizations still needed to run some aspects of their operations using traditional delivery models.
Business innovation is about creating new opportunities for an organization to reinvent its core offerings, revenue streams, and enhance the value proposition for existing or new customers, thus renewing its whole business model. Business innovation springs by understanding the structure of the market, thus adapting or anticipating those changes.
Business modelinnovation is about increasing the success of an organization with existing products and technologies by crafting a compelling value proposition able to propel a new business model to scale up customers and create a lasting competitive advantage. And it all starts by mastering the key customers.
A consumer brand company like Procter & Gamble (P&G) defines “Constructive Disruption” as: a willingness to change, adapt, and create new trends and technologies that will shape our industry for the future. According to P&G, it moves around four pillars: lean innovation, brand building, supply chain, and digitalization & data analytics.
That is a process that requires a continuous feedback loop to develop a valuable product and build a viable business model. Continuous innovation is a mindset where products and services are designed and delivered to tune them around the customers’ problem and not the technical solution of its founders.
A design sprint is a proven five-day process where critical business questions are answered through speedy design and prototyping, focusing on the end-user. A design sprint starts with a weekly challenge that should finish with a prototype, test at the end, and therefore a lesson learned to be iterated.
Tim Brown, Executive Chair of IDEO, defined design thinking as “a human-centered approach to innovation that draws from the designer’s toolkit to integrate the needs of people, the possibilities of technology, and the requirements for business success.” Therefore, desirability, feasibility, and viability are balanced to solve critical problems.
DevOps refers to a series of practices performed to perform automated software development processes. It is a conjugation of the term “development” and “operations” to emphasize how functions integrate across IT teams. DevOps strategies promote seamless building, testing, and deployment of products. It aims to bridge a gap between development and operations teams to streamline the development altogether.
Product discovery is a critical part of agile methodologies, as its aim is to ensure that products customers love are built. Product discovery involves learning through a raft of methods, including design thinking, lean start-up, and A/B testing to name a few. Dual Track Agile is an agile methodology containing two separate tracks: the “discovery” track and the “delivery” track.
eXtreme Programming was developed in the late 1990s by Ken Beck, Ron Jeffries, and Ward Cunningham. During this time, the trio was working on the Chrysler Comprehensive Compensation System (C3) to help manage the company payroll system. eXtreme Programming (XP) is a software development methodology. It is designed to improve software quality and the ability of software to adapt to changing customer needs.
Feature-Driven Development is a pragmatic software process that is client and architecture-centric. Feature-Driven Development (FDD) is an agile software development model that organizes workflow according to which features need to be developed next.
A Gemba Walk is a fundamental component of lean management. It describes the personal observation of work to learn more about it. Gemba is a Japanese word that loosely translates as “the real place”, or in business, “the place where value is created”. The Gemba Walk as a concept was created by Taiichi Ohno, the father of the Toyota Production System of lean manufacturing. Ohno wanted to encourage management executives to leave their offices and see where the real work happened. This, he hoped, would build relationships between employees with vastly different skillsets and build trust.
GIST Planning is a relatively easy and lightweight agile approach to product planning that favors autonomous working. GIST Planning is a lean and agile methodology that was created by former Google product manager Itamar Gilad. GIST Planning seeks to address this situation by creating lightweight plans that are responsive and adaptable to change. GIST Planning also improves team velocity, autonomy, and alignment by reducing the pervasive influence of management. It consists of four blocks: goals, ideas, step-projects, and tasks.
The ICE Scoring Model is an agile methodology that prioritizes features using data according to three components: impact, confidence, and ease of implementation. The ICE Scoring Model was initially created by author and growth expert Sean Ellis to help companies expand. Today, the model is broadly used to prioritize projects, features, initiatives, and rollouts. It is ideally suited for early-stage product development where there is a continuous flow of ideas and momentum must be maintained.
An innovation funnel is a tool or process ensuring only the best ideas are executed. In a metaphorical sense, the funnel screens innovative ideas for viability so that only the best products, processes, or business models are launched to the market. An innovation funnel provides a framework for the screening and testing of innovative ideas for viability.
According to how well defined is the problem and how well defined the domain, we have four main types of innovations: basic research (problem and domain or not well defined); breakthrough innovation (domain is not well defined, the problem is well defined); sustaining innovation (both problem and domain are well defined); and disruptive innovation (domain is well defined, the problem is not well defined).
The innovation loop is a methodology/framework derived from the Bell Labs, which produced innovation at scale throughout the 20th century. They learned how to leverage a hybrid innovation management model based on science, invention, engineering, and manufacturing at scale. By leveraging individual genius, creativity, and small/large groups.
The Agile methodology has been primarily thought of for software development (and other business disciplines have also adopted it). Lean thinking is a process improvement technique where teams prioritize the value streams to improve it continuously. Both methodologies look at the customer as the key driver to improvement and waste reduction. Both methodologies look at improvement as something continuous.
A startup company is a high-tech business that tries to build a scalable business model in tech-driven industries. A startup company usually follows a lean methodology, where continuous innovation, driven by built-in viral loops is the rule. Thus, driving growth and building network effects as a consequence of this strategy.
As pointed out by Eric Ries, a minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort through a cycle of build, measure, learn; that is the foundation of the lean startup methodology.
Kanban is a lean manufacturing framework first developed by Toyota in the late 1940s. The Kanban framework is a means of visualizing work as it moves through identifying potential bottlenecks. It does that through a process called just-in-time (JIT) manufacturing to optimize engineering processes, speed up manufacturing products, and improve the go-to-market strategy.
Jidoka was first used in 1896 by Sakichi Toyoda, who invented a textile loom that would stop automatically when it encountered a defective thread. Jidoka is a Japanese term used in lean manufacturing. The term describes a scenario where machines cease operating without human intervention when a problem or defect is discovered.
The PDCA (Plan-Do-Check-Act) cycle was first proposed by American physicist and engineer Walter A. Shewhart in the 1920s. The PDCA cycle is a continuous process and product improvement method and an essential component of the lean manufacturing philosophy.
RAD was first introduced by author and consultant James Martin in 1991. Martin recognized and then took advantage of the endless malleability of software in designing development models. Rapid Application Development (RAD) is a methodology focusing on delivering rapidly through continuous feedback and frequent iterations.
Retrospective analyses are held after a project to determine what worked well and what did not. They are also conducted at the end of an iteration in Agile project management. Agile practitioners call these meetings retrospectives or retros. They are an effective way to check the pulse of a project team, reflect on the work performed to date, and reach a consensus on how to tackle the next sprint cycle. These are the five stages of a retrospective analysis for effective Agile project management: set the stage, gather the data, generate insights, decide on the next steps, and close the retrospective.
Scaled Agile Lean Development (ScALeD) helps businesses discover a balanced approach to agile transition and scaling questions. The ScALed approach helps businesses successfully respond to change. Inspired by a combination of lean and agile values, ScALed is practitioner-based and can be completed through various agile frameworks and practices.
The SMED (single minute exchange of die) method is a lean production framework to reduce waste and increase production efficiency. The SMED method is a framework for reducing the time associated with completing an equipment changeover.
The Spotify Model is an autonomous approach to scaling agile, focusing on culture communication, accountability, and quality. The Spotify model was first recognized in 2012 after Henrik Kniberg, and Anders Ivarsson released a white paper detailing how streaming company Spotify approached agility. Therefore, the Spotify model represents an evolution of agile.
As the name suggests, TDD is a test-driven technique for delivering high-quality software rapidly and sustainably. It is an iterative approach based on the idea that a failing test should be written before any code for a feature or function is written. Test-Driven Development (TDD) is an approach to software development that relies on very short development cycles.
Timeboxing is a simple yet powerful time-management technique for improving productivity. Timeboxing describes the process of proactively scheduling a block of time to spend on a task in the future. It was first described by author James Martin in a book about agile software development.
Scrum is a methodology co-created by Ken Schwaber and Jeff Sutherland for effective team collaboration on complex products. Scrum was primarily thought for software development projects to deliver new software capability every 2-4 weeks. It is a sub-group of agile also used in project management to improve startups’ productivity.
Scrumban is a project management framework that is a hybrid of two popular agile methodologies: Scrum and Kanban. Scrumban is a popular approach to helping businesses focus on the right strategic tasks while simultaneously strengthening their processes.
Scrum anti-patterns describe any attractive, easy-to-implement solution that ultimately makes a problem worse. Therefore, these are the practice not to follow to prevent issues from emerging. Some classic examples of scrum anti-patterns comprise absent product owners, pre-assigned tickets (making individuals work in isolation), and discounting retrospectives (where review meetings are not useful to really make improvements).
Scrum at Scale (Scrum@Scale) is a framework that Scrum teams use to address complex problems and deliver high-value products. Scrum at Scale was created through a joint venture between the Scrum Alliance and Scrum Inc. The joint venture was overseen by Jeff Sutherland, a co-creator of Scrum and one of the principal authors of the Agile Manifesto.
Six Sigma is a data-driven approach and methodology for eliminating errors or defects in a product, service, or process. Six Sigma was developed by Motorola as a management approach based on quality fundamentals in the early 1980s. A decade later, it was popularized by General Electric who estimated that the methodology saved them $12 billion in the first five years of operation.
Stretch objectives describe any task an agile team plans to complete without expressly committing to do so. Teams incorporate stretch objectives during a Sprint or Program Increment (PI) as part of Scaled Agile. They are used when the agile team is unsure of its capacity to attain an objective. Therefore, stretch objectives are instead outcomes that, while extremely desirable, are not the difference between the success or failure of each sprint.
The Toyota Production System (TPS) is an early form of lean manufacturing created by auto-manufacturer Toyota. Created by the Toyota Motor Corporation in the 1940s and 50s, the Toyota Production System seeks to manufacture vehicles ordered by customers most quickly and efficiently possible.
The Total Quality Management (TQM) framework is a technique based on the premise that employees continuously work on their ability to provide value to customers. Importantly, the word “total” means that all employees are involved in the process – regardless of whether they work in development, production, or fulfillment.
The waterfall model was first described by Herbert D. Benington in 1956 during a presentation about the software used in radar imaging during the Cold War. Since there were no knowledge-based, creative software development strategies at the time, the waterfall method became standard practice. The waterfall model is a linear and sequential project management framework.
Gennaro is the creator of FourWeekMBA, which reached about four million business people, comprising C-level executives, investors, analysts, product managers, and aspiring digital entrepreneurs in 2022 alone | He is also Director of Sales for a high-tech scaleup in the AI Industry | In 2012, Gennaro earned an International MBA with emphasis on Corporate Finance and Business Strategy.
