In the previous post, we discussed how Jobs to be Done (JTBD) works as a great “exchange” currency to facilitate strategy discussions between designers, business stakeholders, and technology people.
In this post, I’ll show you a few ways to use Jobs to be Done (JTBD) to facilitate the two-way negotiations between leadership and product teams that allows for managing by outcomes.
- Facilitating Investment Discussions around Value
- The Balancing Act of Managing by Outcomes, Feasibility, Viability, and Desirability
- Outcomes over Outputs
- Managing by Outcomes and the Product Trio
- Managing by Outcomes at the Right Level of Altitude
- Coordination for Managing by Outcomes
- Exploration for Managing by Outcomes
- Psychological Safety, Experimentation and Permission to Fail
- Outcomes and Metrics
- Jobs to be Done as a Common Strategy Vocabulary for Managing by Outcomes
- Managing by Outcomes as a Two-way Negotiation
- Roadmaps and Two-way Negotiations
- Outcomes and Roadmaps Objectives
- Outcomes and Roadmaps Themes
- Facilitating Two-way Negotiations through Visual Thinking
- Anti-patterns for Managing by Outcomes
- The Right Time for Managing by Outcomes Discussions
- The Double Diamond Framework
- Facilitating discussions for Managing by Outcomes
- Recommended Reading
- Using outcomes creates focus and alignment. It eliminates needless work. And it puts the customer at the center of everything you do.
- Managing by outcomes requires answering the following question: What measurable changes in human behavior drive business results?
- While managing by outcomes, we give our teams the autonomy, responsibility, and ownership to chart their own path. Instead of asking them to deliver a fixed roadmap full of features by a specific time, we ask them to solve a customer problem or address a business need.
- Managing by outcomes is only as effective as the outcomes themselves. We’ll still get the wrong outputs if we choose the wrong outcomes.
- When considering outcomes for specific teams, it helps to distinguish between business outcomes, product outcomes, and traction metrics.
- A business outcome is a metric that moves the business forward, while a product outcome is a metric that helps us understand if the product is moving the business forward.
- In the outcome-driven paradigm, the focus is not on the customer but on the job: the job is the unit of analysis. When companies focus on helping the customer get a job done faster, more conveniently, and less expensively than before, they are more likely to create products and services that the customer wants.
- Encouraging a two-way negotiation between the product leader and the product trio ensures we capture the proper organizational knowledge required for selecting an outcome.
- Even if product teams “know better” what the desired outcomes from their customers’ perspective are, it will be difficult to have genuine two-way negotiation around managing outcomes with product leaders if their roadmaps are fixed.
- Your roadmap should be a high-level view of what needs and problems your product should solve while also helping you confirm why, while your release plan should detail how you will solve them.
- The product roadmap should not be about developing solutions or defining what your product will look like. Leave that to the release plan!
Facilitating Investment Discussions around Value
As I mentioned in a previous post, designers must become skilled facilitators that respond, prod, encourage, guide, coach, and teach as they guide individuals and groups to make decisions that are critical in the business world through effective processes. Few decisions are harder than deciding how to prioritise.
I’ve seen too many teams that a lot of their decisions seem to be driven by the question “What can we implement with the least effort” or “What are we able to implement,” not by the question “what brings value to the user.”
From both a user-centered and a market fit perspective, the most crucial pivot that needs to happen in the conversation between designers and business stakeholders is the framing of value:
- Business value
- User value
- Value to designers (sense of self-realization? Did I impact someone’s life in a positive way?)
The mistake I’ve seen many designers make is to look at prioritization discussion as a zero-sum game: our user-centered design tools set may have focused too much on the needs of the user, at the expense of business needs and technological constraints.
That said, there is a case to be made that designers should worry about strategy because it helps shape the decisions that not only create value for users but value for employees. And here is why.
Therefore, a strategic initiative is worthwhile only if it does one of the following (Oberholzer-Gee, F. (2021). Better, simpler strategy. 2021):
- Creates value for customers by raising their willingness to pay (WTP): If companies find ways to innovate or to improve existing products, people will be willing to pay more. In many product categories, Apple gets to charge a price premium because the company raises the customers’ WTP by designing beautiful products that are easy to use, for example.A value-focused company convinces its customers in every interaction that it has their best interests at heart.
- Creates value for employees by making work more appealing: When companies make work more interesting, motivating, and flexible, they are able to attract talent even if they do not offer industry-leading compensation. Paying employees more is often the right thing to do, of course. But keep in mind that more-generous compensation does not create value in and of itself; it simply shifts resources from the business to the workforce. By contrast, offering better jobs not only creates value, it also lowers the minimum compensation that you have to offer to attract talent to your business, or what we call an employee’s willingness-to-sell (WTS) wage. Value-focused businesses think holistically about the needs of their employees (or the factors that drive WTS).
- Creates value for suppliers by reducing their operating cost: Like employees, suppliers expect a minimum level of compensation for their products. A company creates value for its suppliers by helping them raise their productivity. As suppliers’ costs go down, the lowest price they would be willing to accept for their goods—what we call their willingness-to-sell (WTS) price—falls. When Nike, for example, created a training center in Sri Lanka to teach its Asian suppliers lean manufacturing, the improved production techniques helped suppliers reap better profits, which they then shared with Nike.
The Value Stick is an interesting tool that provides insight into where the value is in a product or service. It relates directly to Michael Porter’s Five Forces, reflecting how strong those forces are: Willingness to Pay (WTP), Price, Cost, and Willingness to Sell (WTS). The difference between Willingness to Pay (WTP) and Willingness to Sell (WTS) — the length of the stick — is the value that a firm creates (Oberholzer-Gee, F., Better, simpler strategy, 2021)
This idea is captured in a simple graph, called a value stick. WTP sits at the top and WTS at the bottom. When companies find ways to increase customer delight and increase employee satisfaction and supplier surplus (the difference between the price of goods and the lowest amount the supplier would be willing to accept for them), they expand the total amount of value created and position themselves for extraordinary financial performance.
Organizations that exemplify value-based strategy demonstrate some key behaviors (Oberholzer-Gee, F., “Eliminate Strategic Overload” in Harvard Business Review, 2021):
- They focus on value, not profit. Perhaps surprisingly, value-focused managers are not overly concerned with the immediate financial consequences of their decisions. They are confident that superior value creation will improve financial performance over time.
- They attract the employees and customers whom they serve best. As companies find ways to move WTP or WTS, they make themselves more appealing to customers and employees who like how they add value.
- They create value for customers, employees, or suppliers (or some combination) simultaneously. Our early understanding of success in manufacturing holds that costs for companies will rise if they boost consumers’ willingness to pay—that is, it takes more-costly inputs to create a better product. But value-focused organizations find ways to defy that logic.
Instead, we need objective ways to value design solutions to justify the experience investments. We need to look at the different points in the strategy creation and execution and identify the discussions strategists should facilitate while tracking and tracking its implementation to ensure we bring value for customers and business.
For such a conversation to pivot to focus on value, designers will need to get better at influencing the strategy of their design project. However, some designers lack the vocabulary, tools, and frameworks to influence it in ways that drive user experience vision forward.
I don’t know about you, but I’m not in this (only) for the money: I want my work to mean something, create value, and change people’s lives! For the better! We need to bring users’ needs to the conversation and influence the decisions that increase our customer’s Willingness to Pay (WTS) by — for example — increasing customers’ delight so that we can create products and services we are proud to bring into the world!
The Balancing Act of Managing by Outcomes, Feasibility, Viability, and Desirability
As a design manager, I always strive to help stakeholders define the Product Design vision to ensure cohesive product narratives through sound strategy and design principles.
That said, I’ve always found that how priorities are defined can quickly create a disconnect from vision, especially when we need to make tough choices around the scope. We must facilitate discussions around priorities, so we can confidently decide while taking into account not just feasibility, viability, and desirability.
To understand the risk and uncertainty of your idea you need to ask: “What are all the things that need to be true for this idea to work?” This will allow you to identify all three types of hypotheses underlying a business idea: desirability, feasibility, and viability (Bland, D. J., & Osterwalder, A., Testing business ideas, 2020):
- Desirability (do they want this?) relates to the risk that the market a business is targeting is too small; that too few customers want the value proposition; or that the company can’t reach, acquire, and retain targeted customers.
- Feasibility (Can we do this?) relates to the risk that a business can’t manage, scale, or get access to key resources (technology, IP, brand, etc.). This isn’t just technical feasibility; we also look need to look at overall regulatory, policy, and governance that would prevent you from making your solution a success.
- Viability (Should we do this?) relates to the risk that a business cannot generate more revenue than costs (revenue stream and cost stream). While customers may want your solution (desirable) and you can build it (feasible), perhaps there’s not enough of a market for it or people won’t pay enough for it.
Here is — yet again — another advantage of Jobs to be Done: when the team engages in endless discussions around which customer/user problems we should be focusing on, Jobs becomes a unit of analysis that helps teams have facilitated discussions around finding ways to remove (or at least reduce) subjectivity while assessing value, especially while facilitating two-negotiations for managing by outcomes.
Outcomes over Outputs
In traditional planning, the solution provider commits to delivering specified deliverables (the scope) at a specified cost within a given time frame. This approach doesn’t work when requirements are volatile because it locks all parties into predetermined specifications that are likely to be outdated by the time the product is delivered (Podeswa, H., The Agile Guide to Business Analysis and Planning: From Strategic Plan to Continuous Value Delivery, 2021).
Instead of focusing on predetermined deliverables, agile enterprises focus on desired outcomes, such as increased revenues and increased customer loyalty (Podeswa, H., The Agile Guide to Business Analysis and Planning: From Strategic Plan to Continuous Value Delivery, 2021).
You might be asking, “what do you mean by outcomes?” Joshua Seiden defines outcome as “a change in user behavior that drives business results.”
You can help the team and leaders to start thinking in terms of outcomes by asking three simple questions (Seiden, J., Outcomes over Output, 2019):
- What are the user and customer behaviors that drive business results? I’ve suggested in another post that facilitating discussions around Jobs to be Done can be a great way to get the team to align.
- How do we get people to do more of these things?
- How do we know we’re right? The easiest (and the hardest) way to answer that question is to design and conduct tests.
Business thought leaders have been advocating for managing by outcomes for decades. A renowned managerial thought leader, Peter Drucker wrote about its benefits countless times. Andy Grove, the former CEO of Intel, utilized the practice at Intel and wrote about it in his best-selling book High Output Management. More recently, Google, Google Ventures, and John Doerr, a venture capital partner at Kleiner Perkins, have popularized the topic again with their advocacy for objectives and key results (OKRs), one flavor of managing by outcomes. You’ll hear from prominent thought leaders in most industries and broadly across the technology sector (including me) that shifting from dictating outputs to managing outcomes is critical to a company’s success. (Torres, T., Continuous Discovery Habits, 2021).
Managing by Outcomes and the Product Trio
Managing by outcomes communicates to the team how they should be measuring success. A clear outcome helps a team align around the work they should be prioritizing, it helps them choose the right customer opportunities to address, and it helps them measure the impact of their experiments. Without a clear outcome, discovery work can be never-ending, fruitless, and frustrating (Torres, T., Continuous Discovery Habits, 2021).
The key distinction between this strategy over traditional roadmaps is that we are giving the team the autonomy to find the best solution. If they are truly a continuous-discovery team, the product trio has a depth of customer and technology knowledge, giving them an advantage when it comes to making decisions about how to solve specific problems. (Torres, T., Continuous Discovery Habits, 2021).
Additionally, this strategy leaves room for doubt (Torres, T., Continuous Discovery Habits, 2021):
- A fixed roadmap communicates false certainty. It says we know these are the right features to build, even though we know from experience their impact will likely fall short.
- An outcome communicates uncertainty. It says, we know we need this problem solved, but we don’t know the best way to solve it. It gives the product trio the latitude they need to explore and pivot when needed.
The keyword here is uncertainty, which sounds scary to a lot of people! Uncertainty draws people to the conversation by admitting you don’t have all the answers, inviting others to figure it out!
While we should “use” uncertainty to leave some room for exploration when it comes to the solutions, we should provide clarity around the outcomes we expect from teams!
Without a clear outcome, discovery work can be never-ending, fruitless, and frustrating (Torres, T., Continuous Discovery Habits, 2021).
Outcomes, Objectives, and Key Results (OKRs)
Identifying business objectives that tie to your vision is critical to making that vision a reality. It’s great to say, “we imagine a world where instant teleportation from one location to another makes travel effortless.” However, if you don’t have specific objectives to hit along the way, that vision will be challenging to implement because there will be too many different product, technology, and business directions to take to get there (Lombardo, C. T., McCarthy, B., Ryan, E., & Connors, M., Product Roadmaps Relaunched, 2017).
Here is — yet again — another advantage of using Jobs to be Done to frame our value proposition, especially with regards to product vision: because the product vision communicates why you are building something and what the value proposition is for the customer, if we focus on the customers’ outcomes can deliver the greatest value we can be confident to push our vision horizon far out, knowing that the outcomes our vision are trying to address are stable over time.
There are a few ways of helping teams measure their success through outcomes, including John Doerr’s Management by Objectives using Objectives and Key Results (Doerr, J., Measure what matters, 2018).
- Objective: Get real traction for our app
- Key Result 1: Increase paying customers to x per month
- Key Result 2: Increase monthly active users (MAU) by y%
- Key Result 3: Increase revenue to $z MRR
A team (or a whole organization) can have 3-5 top-level OKRs that together reflect the most important outcomes the company is pursuing. In larger organizations, OKRs may “cascade” from these top few down to child OKRs that divide up the focus across teams and levels of hierarchy. Key Result 1 from the above Objective could become a Parent Objective with its own KR children below (McCarthy, B., How should product teams use OKRs?, 2019):
- Objective: Increase paying customers to x per month
- Key Result 1: Increase self-sign-up to x per month
- Key Result 2: Increase free-trial conversion by y%
- Key Result 3: Reduce churn to < z% per month
Objectives and Outcomes
Often reffered as Objectives and Key Results (OKRs) are a great way to pair your business objectives with scuess criteria. The premise of the OKR framework is that objectives are especific qualitative goals, and key results are quantitative measures of progress towards those objectives (Lombardo, C. T., McCarthy, B., Ryan, E., & Connors, M., Product Roadmaps Relaunched, 2017).
Asking what is our product vision or team mission is also really helpful as a starting place for OKRs. When asked to write objectives, lots of executives default to “grow by x%” or “generate y revenue.” Then they generally run out of ideas for objectives or start quickly thinking in terms of deliverables like “ship the new thingie by Q1” or “launch the whackamole initiative.” (McCarthy, B., How should product teams use OKRs?, 2019).
Sure, you probably still have some growth metrics in mind, but you could start by measuring the improvement you are seeking in customers’ lives. Measuring the value you are generating as directly as you can is a great way to align your team on what matters (McCarthy, B., How should product teams use OKRs?, 2019).
Key Results and Outcomes
There are two fundamental principles for allows for teams to switch to managing by outcomes mindset (Cagan, M., Inspired: How to create tech products customers love, 2017):
- Never tell people how to do things. Tell them what to do, and they will surprise you with their ingenuity.
- Performance is measured by results. The idea here is that you can release all the features you want, but if it doesn’t solve the underlying business problem, you haven’t really solved anything.
The key here is — going back to Joshua Seiden’s “Outcomes” — to help the team connect the dots between what are the changes in human behaviour that drives business results?
Here is — yet again — another advantage of using jobs to be done as a common exchange currency between leadership, designers, product managers and developers while having managing by outcomes negotiations: jobs described what users are trying to get done (the “why”s), which — in turn — provides a good framing to discuss objectives for the team (the “what”‘s the solution, not the “how”s).
OKRs can quickly drive you to a narrow fixation on business results, unethical behavior, and short-term outcomes that eventually undermine your very business. But if your OKRs begin with an inspiring vision of how awesome it will be for your customers when you succeed, then you can break that down into whatever is required to achieve that vision or carry out that mission (McCarthy, B., How should product teams use OKRs?, 2019).
OKRs at Scale
Many large organizations ask their teams to provide team-specific OKRs. If there are 200 teams in your company, the management of 200 team-level goals can become overwhelming. Imagine if you had 500 teams! It can also cause one of the main anti-patterns of OKR implementation: hyperlocal optimization. One team may work hard to hit their key results but the work they’re doing inadvertently hampers another team’s progress toward their own goals (Gothelf, J., OKRs at scale, 2021).
One way to solve this is — instead of asking teams to provide team-level OKRs at first — we identify a set of teams who will be dedicated to the same goal and have them, as a team of teams, set their objectives and key results. As a unit, this team is now on the hook for these goals. Hyperlocal optimization is instantly communicated and dealt with because the measure of success is global for the entire group, not the individual components that make it up (Gothelf, J., OKRs at scale, 2021)
This doesn’t necessarily mean that each team doesn’t have its own team-specific goals to achieve. In fact, one of the first things to ask the component teams for is a set of key results to function as leading indicators of the group’s overall key result goals. In this way, each team is working towards a thing they can influence directly but they’re doing so with (Gothelf, J., Execs care about revenue. How do we get them to care about outcomes? 2017):
- A clear line of sight of the overall goal they’re trying to achieve
- A transparent view into how their work is impacting other teams in the group
- Awareness that if the key results they’ve chosen don’t have the impact they predicted on the group’s goals, they’ll need to adjust their goals
Understanding these relationships between Objectives and Key Results will be critical for what will be discussed next, which is assigning outcomes to product teams.
Assigning Outcomes to Product Teams
Managing by outcomes is only as effective as the outcomes themselves. If we choose the wrong outcomes, we’ll still get the wrong results. When considering outcomes for specific teams, it helps to distinguish between business outcomes, product outcomes, and traction metrics (Torres, T., Continuous Discovery Habits, 2021):
- A business outcome measures how well the business is progressing.
- A product outcome measures how well the product is moving the business forward.
- A traction metric measures usage of a specific feature or workflow in the product.
If you read my last post on Bringing Business Impact and User Needs together with Jobs to be Done, you understand why I think we should add user or customer outcomes to this list and how product outcomes and customer outcomes are interconnected.
Business outcomes start with financial metrics (e.g., grow revenue, reduce costs), but they can also represent strategic initiatives (e.g., grow market share in a specific region, increase sales to a new customer segment). Many business outcomes, however, are lagging indicators. They measure something after it has happened ((Torres, T., Continuous Discovery Habits, 2021).
By the time the team was able to measure the impact of their product changes, customers had already churned. Therefore, we want to identify leading indicators that predict the direction of the lagging indicator. Assigning a team a leading indicator is always better than assigning a lagging indicator (Torres, T., Continuous Discovery Habits, 2021).
User or Customer Outcomes
The challenge of arriving at business definition for human needs starts with language. An agreed-on language is fundamental to success in any discipline, yet confusion has permeated product development because companies continue to define “requirements” as any kind of customer input: customer wants, needs, benefits, solutions, ideas, desires, demands, specifications, and so on. But really, those are all different types of inputs, none of which can be used predictably to ensure success (Ulwick, A. W., What customers want, 2005).
Clayton Christensen credits Ulwick and Richard Pedi of Gage Foods with the way of thinking about market structure used in the chapter “What Products Will Customers Want to Buy?” in his Innovator’s Solution and called Jobs to be Done (JTBD) or “outcomes that customers are seeking”.
A customer job could be the tasks that they are trying to perform and complete, the problems the are trying to solve, or the needs they are trying to satisfy. (Osterwalder, A., Pigneur, Y., Papadakos, P., Bernarda, G., Papadakos, T., & Smith, A. Value proposition design, 2014).
Because they don’t mention solutions or technology, jobs should be as timeless and unchanging as possible. Ask yourself “How would people have gotten the job done 50 years ago?” Strive to frame jobs in a way that makes them stable, even as technology changes. (Kalbach, J. Jobs to be Done Playbook, 2020).
Jobs to be Done (JTBD) is a new way to think about the innovation process. Three key tenets define this approach (Ulwick, A. W., What customers want, 2005):
- Customers buy products and services to help them get jobs done. In our study of new and existing markets we find that customers (both people and companies) have “jobs” with functional dimensions to them that arise regularly and need to get done. When customers become aware of such a job, they look around for a product or service that will help them get the job done. We know, for example, that people buy mowers so they can cut their lawns; and they buy insurance to limit their financial risks; Corn farmers, to take another example, buy corn seed, herbicides, pesticides, and fertilizers to help them grow corn. Carpenters buy circular saws to cut wood. Virtually all products and services are acquired to help get a job done.
- Customers use a set of metrics (performance measures) to judge how well a job is getting done and how a product performs. Just as companies use metrics to measure the output quality of a business process, customers use metrics to measure success in getting a job done. Customers have these metrics in their minds, but they seldom articulate them, and companies rarely understand them. We call these metrics the customers’ desired outcomes. They are the fundamental measures of performance inherent to the execution of a specific job. When cutting wood with a circular saw, carpenters may judge products for their ability to minimize the likelihood of losing sight of the cut line, the time it takes to adjust the depth of the blade, or the frequency of kickbacks. Only when all the metrics for a given job are well satisfied are customers able to execute the job perfectly. Ironically, these metrics are overlooked in the customer-driven world because they are not revealed by listening to the “voice of the customer.
- These customer metrics make possible the systematic and predictable creation of breakthrough products and services. With the proper inputs in hand, companies dramatically improve their ability to execute all other downstream activities in the innovation process, including their ability to identify opportunities for growth, segment markets, conduct competitive analysis, generate and evaluate ideas, communicate value to customers, and measure customer satisfaction.
At its core, the concept of JTBD is straightforward focus on people’s objectives independent of the means used to accomplish them. Through this lens, JTBD offers a structured way of understanding customer needs, helping to predict better how customers might act in the future (Kalbach, J. Jobs to be Done Playbook, 2020).
Bringing Business Impact and User Needs together with Jobs to be Done (JTBD)
Learn how Jobs to be Done (JTBD) work as a great “exchange” currency to facilitate strategy discussions around value between designers, business stakeholders and technology people that allows for managing by outcomes (Photo by Blue Bird on Pexels.com)
If you read my last post on Bringing Business Impact and User Needs together with Jobs to be Done, you understand why I think we should add user or customer outcomes to this list, and how product outcomes and customer outcomes are interconnected through Jobs-to-be-Done (JTBD).
As a general rule, product trios will make more progress on a product outcome rather than a business outcome. Remember, product outcomes measure how well the product moves the business forward. By definition, a product outcome is within the product trio’s span of control. Business outcomes, on the other hand, often require coordination across many business functions. (Torres, T., Continuous Discovery Habits, 2021).
Managing by Outcomes at the Right Level of Altitude
As mentioned earlier, business outcomes often require coordination across many business functions. Coordination isn’t bad. In fact, most of the work that we do will require coordination across teams. However, we can increase the accountability of each team by assigning a metric that is relevant to their own work (Torres, T., Continuous Discovery Habits, 2021).
Coordination for Managing by Outcomes
As an example of coordination, we might ask the product team to increase the number of dogs who like the food (something within the product team’s span of control). In contrast, we might ask the marketing team to increase the pricing transparency after the trial ends, and we might ask the customer support team to decrease their average response times. All three groups contribute to the business outcome of increasing customer retention, but each is doing so in the way that they can best contribute (Torres, T., Continuous Discovery Habits, 2021).
When multiple teams are assigned the same outcome, it’s easy to shift blame for lack of progress (Torres, T., Continuous Discovery Habits, 2021).
Exploration for Managing by Outcomes
When setting product outcomes, we want to make sure that we are giving the product trio enough latitude to explore. This is where the distinction between product outcomes and traction metrics can be helpful. It’s also a key delineation between an outcome mindset and an output mindset (Torres, T., Continuous Discovery Habits, 2021).
Experimentation is at the heart of what software developers call agile development. Rather than planning all activities up-front and then sequentially, agile development emphasises running many experiments and learning from them (Mueller, S., & Dhar, J., The decision maker’s playbook, 2019)
It takes a certain level of maturity to run effective experiments. To avoid shipping experiments for the sake of shipping experiments, teams need to focus on delivering outcomes. They also need to be willing to embrace failure to make progress (Garbugli, É., Solving Product, 2020).
For a learning culture to thrive, your teams must feel safe to experiment. Experiments are how we learn, but experiments — by nature — fail frequently. In a good experiment, you learn as much from failure as from success. If failure is stigmatised, teams will take few risks (Gothelf, J., & Seiden, J., Sense and respond. 2017).
On average, 80% of experiments fail to deliver the expected outcomes but with the right method, 100% of experiments can help you learn and progress (Garbugli, É., Solving Product, 2020).
You should focus on one or two core goals at a time, aligning with your North Star metric or the AARRR steps that you’re focused on. Your goals should be big, your experiments small and nimble (Garbugli, É., Solving Product, 2020).
Teams will be more willing to experiment if they feel they are not being measured by the delivery of hard requirements, but appreciated by achieving great outcomes that create value.
Psychological Safety, Experimentation and Permission to Fail
Most change efforts fail, even when experienced people are involved, and even when the environment is relatively trusting and safe. We should approach improvement like we approach product—using thoughtful experiments and disciplined, intentional learning (Cutler, J. Making things better with enabling constraints, 2022).
For a learning culture to thrive, your teams must feel safe to experiment. Experiments are how we learn, but experiments — by nature — fail frequently. In a good experiment, you learn as much from failure as from success. If failure is stigmatized, teams will take few risks (Gothelf, J., & Seiden, J., Sense and respond. 2017).
Permission to Fail and Sandboxing
Sandboxing is a way to reduce the risk of experimentation. The idea is to create a set of procedures, rules, and constraints that your organization can live with and within which failure is acceptable. You will also need cultural permission to experiment. This means that your progress will not be linear and predictable and that you should not be judged by your delivery rate (the amount of stuff you ship) but by your learning rate, and by your overall progress towards strategic goals — in other words, by the extent to which you achieve the outcomes in question. A sandbox creates positive effects for both leaders and the team (Gothelf, J., & Seiden, J., Sense and respond. 2017):
- For leaders: there is a legitimate fear that their people will get creative in some way that will cause trouble, and for which a leader will be held responsible. Creating clear guidelines within which people can operate can ease that fear.
- For teams: the fear is about crossing some unstated line. If leaders make the lines clear, it creates space for creativity.
When thinking of creating a learning culture, it helps to focus on the collective behaviors of a system, and the “constraints” of that system inform and shape that behavior. Constraints shape a system by modifying its phase space (its range of possible actions) or the probability distribution (the likelihood) of events and movements within that space. Because constraints are both key actors and key indicators of a system, constraint mapping can be a highly productive first step in considering how to intervene (Juarrero, A., Dynamics in Action: Intentional Behavior as a Complex System, 1999)
Designing effective enabling constraints is an art. Many things feel intuitively correct but have potentially harmful consequences. For example (Cutler, J. Making things better with enabling constraints, 2022):
- In an effort to increase certainty about plans and commitments, the team undertakes a comprehensive annual planning effort. This feels good on the surface, but it forces premature convergence, encourages over-utilization of shared resources, and encourages big, inflexible projects.
- In an effort to centralize communication, the team adopts a single tool for documentation (a theoretically enabling constraint). This feels good on the surface—having documentation everywhere is painful—but since a large % of communication with external teams happens outside the central tool, you find a two or three (or more) tiered system of communication (e.g., executive communication happens in slides, not in the tool).
The trick, then, is designing effective enabling constraints. An additional layer to consider is the layer of trust, respect, empowerment, and psychological safety. Example: deadlines. In theory, deadlines can be enabling constraints. However, without empowerment and trust, deadlines become disabling. Teams cut the wrong corners, and optimize for low trust. These two points—the counterintuitive nature of constraints, and the trust/safety element—explain why most change efforts fail. High trust environments can easily pick the wrong constraints. Or they try too much at once and put people in a state of change overload (Cutler, J. Making things better with enabling constraints, 2022).
No enabling constraint is guaranteed to work, but some are better than others. What should someone designing an enabling constraint look out for? (Cutler, J. Making things better with enabling constraints, 2022):
- It is easy to know if you are doing it or not. For example, asking everyone to use a single document repository is a bit vague. People WILL need to use other systems to document things. Do those count? What goes in it? What doesn’t? An alternative might be to run an experiment where the team commits to putting ONE document type in the centralized repository or tool. Put another way, it is within reach and achievable.
- It has an expiration date and is treated as an experiment. The best enabling constraints are treated as an experiment. The team commits to giving it an honest try for a period of time. The team is promised an opportunity to weigh in on the experiment, before agreeing to extend it.
- It helps people go through the motions. If you have a future state in mind, it helps to help people go through the motions a bit and try things out. In a safe way.
- The world doesn’t end if it “fails”. Sometimes things don’t go as planned. That’s normal. The best enabling constraints fail gracefully. They are safe-to-fail probes.
- Fast feedback potential. The best enabling constraints will provide fast feedback. Experiments that last forever, with no sense if they are helping/hurting, are dangerous (or at a minimum draining, and encourage people to just work around them).
Overcoming Fear of Failure with Premortems
The scientist and decision-making expert Gary Klein is a proponent of using ” premortems”: doing a postmortem in advance to envision what a potential failure might look like so that you can then consider the possible reasons for that failure. To put the premortem into question form you might ask: If we were to fail, what might be the reasons for that failure? Decision researchers say using premortems can temper excessive optimism and encourage a more realistic assessment of risk (Berger, W., The book of beautiful questions, 2019).
While you’re envisioning the possibility of failure, be sure to consider the opposite, as well, by asking: What if we succeed — what would that look like? Jonathan Fields points out that this question is important because it can help counter the negativity bias. Fields recommends visualizing, in detail, what would be likely to happen in a best-case scenario (more on that in the Importance of Vision). The reality may not live up to that, but that vision can provide an incentive strong enough to encourage taking a risk (Berger, W., The book of beautiful questions, 2019).
Questions you can use to help overcome the fear of failure (Berger, W., The book of beautiful questions, 2019):
- What would we try if we knew we could not fail? start with this favorite Silicon Valley question to help identify bold possibilities.
- What is the worst that could happen? This may seem negative, but the question forces the team to confront hazy fears and consider them in a more specific way (which usually makes them less scary).
- If we did fail, what would be the likely causes? Try the premortem exercise I’ve mentioned earlier, listing some of the potential causes for failure. This should — at least — create a list of pitfalls for you to avoid.
- … and how would we recover from that failure? Just thinking about how we would pick up the pieces if we fail tends to lessen the fear of that possibility.
- What if we succeed — what would that look like? Now shift from the worst-case to the best-case scenario. Visualizing success breeds confidence — and provides motivation for moving forward.
- How can we take one small step into the breach? Consider whether there are “baby steps” that could lead up to taking a leap.
One culture-building practice that organizations use to create permission to fail is the blameless postmortem. This regularly occurring meeting provides an opportunity for the entire team to go through a recent time period (product release cycle, quarter, etc) or to review a specific incident and honestly examine what went well what could be improved, and what should not be continued. Often these postmortems are facilitated by someone outside the team to avoid any bias or conflict of interest. The motivation for this process is to (Gothelf, J., & Seiden, J., Sense and respond. 2017):
- Treat failures as learning opportunities: Think of this activity as a continuous improvement but applied to the way the team works rather than the product it’s working on. In order to learn from failures, you need to make an accurate assessment of what happened, why it happened, and how it can be prevented next time.
- Protect from blame: It would be simple to treat this inquiry as a hunt for the person responsible so that this person can be disciplined. But if this is the outcome of the inquiry, then the people involved will not be motivated to share the truth of what happened. Instead, they will cover it up to avoid punishment. So, in order for people to learn, the blameless postmortem process must include an ironclad guarantee that they can speak without fear of punishment. And that guarantee must be upheld each and every time.
Outcomes and Metrics
Key performance indicators (KPIs) are metrics that measure your product’s performance. They help understand if the product is meeting its business goals and if the product strategy is working. Without KPIs, you end up guessing how well your product is performing. You may have a hunch or intuition, but how can you tell it’s right? Using KPIs and collecting the right data helps you balance opinions, beliefs, and gut feelings with empirical evidence, which increases the chances of making the right decisions and providing a successful product (Pichler, R., Strategize: Product strategy and product roadmap practices for the digital age, 2016).
When we assign traction metrics to product trios, we run the risk of painting them into a corner by limiting the types of decisions that they can make. Product outcomes, generally, give product trios far more latitude to explore and will enable them to make the decisions they need to ultimately drive business outcomes. However, there are two instances in which it is appropriate to assign traction metrics to your team (Torres, T., Continuous Discovery Habits, 2021):
- Assign traction metrics to more junior product trios. Improving a traction metric is more of an optimization challenge than a wide-open discovery challenge and is a great way for a junior team to get some experience with discovery methods before giving them more responsibility. For your more mature teams, however, stick with product outcomes.
- If you have a mature product and you have a traction metric that you know is critical to your company’s success, it makes sense to assign this traction metric to an optimization team. If the broader discovery questions have already been answered, then it’s perfectly fine to assign a traction metric to a team. The key is to use traction metrics only when you are optimizing a solution and not when the intent is to discover new solutions. In those instances, a product outcome is a better fit.
When you’re deciding which traction metrics the product team needs to track, here are a few things to keep in mind (Pichler, R., Strategize, 2016):
- Avoid vanity metrics, which measure that make your product look good but don’t add value.
- Don’t measure everything that can be measured, and don’t trust blindly an analytics tool to collect the right data. Instead, use the business goals to choose a small number of metrics that truly help you understand how your product performs. Otherwise, you risk wasting time and effort analyzing data that provides little or no value.
- Be aware that some metrics are sensitive to the product life cycle. For example, you usually cannot measure profit before your product enters the growth stage. Tracking adoption rates and referrals are very useful in the introduction and growth stages but are less so in the maturity and decline stages.
What makes a Good Metric?
Here are some rules of thumb for what makes a good metric — a number that will drive the changes you’re looking for (Croll, A., & Yoskovitz, B. Lean Analytics, 2013):
- A good metric is comparative. Being able to compare a metric to other time periods, groups of users, or competitors helps you understand which way things are moving. “Increased conversion from last week” is more meaningful than “2% conversion.”
- A good metric is understandable. If people can’t remember it and discuss it, it’s much harder to turn a change in the data into a change in the culture.
- A good metric is a ratio or a rate. Accountants and financial analysts have several ratios they look at to understand, at a glance, the fundamental health of a company. You need some, too.
- A good metric changes the way you behave. This is by far the most important criterion for a metric: what will you do differently based on changes in the metric?
- “Accounting” metrics like daily sales revenue, when entered into your spreadsheet, need to make your predictions more accurate. These metrics form the basis of Lean Startup’s innovation accounting, showing you how close you are to an ideal model and whether your actual results are converging on your business plan.
- “Experimental” metrics (like the results of a test) help you optimize the product, pricing, or market. Changes in these metrics will significantly change your behavior.
There are several reasons ratios tend to be the best metrics (Croll, A., & Yoskovitz, B. Lean Analytics, 2013):
- Ratios are easier to act on. Think about driving a car. Distance traveled is informational. But speed–distance per hour–is something you can act on, because it tells you about your current state, and whether you need to go faster or slower to get to your destination on time.
- Ratios are inherently comparative. If you compare a daily metric to the same metric over a month, you’ll see whether you’re looking at a sudden spike or a long-term trend. In a car, speed is one metric, but speed right now over average speed this hour shows you a lot about whether you’re accelerating or slowing down.
- Ratios are also good for comparing factors that are somehow opposed, or for which there’s an inherent tension. In a car, this might be the distance covered divided by traffic tickets. The faster you drive, the more distance you cover–but the more tickets you get. This ratio might suggest whether or not you should be breaking the speed limit.
Leading versus Lagging Metrics
Both leading and lagging metrics are useful, but they serve different purposes (Croll, A., & Yoskovitz, B. Lean Analytics, 2013):
- A leading metric (sometimes called a leading indicator) tries to predict the future. For example, the current number of prospects in your sales funnel gives you a sense of how many new customers you’ll acquire in the future. If the current number of prospects is very small, you’re not likely to add many new customers. You can increase the number of prospects and expect an increase in new customers.
- A lagging metric, such as churn (which is the number of customers who leave in a given time period) gives you an indication that there’s a problem–but by the time you’re able to collect the data and identify the problem, it’s too late. The customers who churned out aren’t coming back. That doesn’t mean you can’t act on a lagging metric (i.e., work to improve churn and then measure it again), but it’s akin to closing the barn door after the horses have left. New horses won’t leave, but you’ve already lost a few.
In some cases, a lagging metric for one group within a company is a leading metric for another. For example (Croll, A., & Yoskovitz, B. Lean Analytics, 2013)
- The number of quarterly bookings is a lagging metric for salespeople (the contracts are signed already)
- For the finance department (that’s focused on collecting payment), quarterly bookings is a leading indicator of expected revenue (since the revenue hasn’t yet been realized).
Be aware that indicators only make sense in the context of when they are captured. For example. Retention is a lagging indicator, which is impossible to act on immediately. It will be months before you have solid data to show that people stayed with you. We also need to measure leading indicators like activation, happiness, and engagement. Leading indicators tell us whether we’re on our way to achieving those lagging indicators like retention. To determine the leading indicators for retention, you can qualify what keeps people retained- for example, happiness and usage of the product. The success metrics we set around options are leading indicators of outcomes we expect on our initiatives because options are strategies on a shorter time scale (Perri, M., Escaping the build trap, 2019).
Metrics, Goals, and Moving Targets
When picking a goal early on, you’re drawing a line in the sand–not carving it in stone. You’re chasing a moving target because you really don’t know how to define success (Croll, A., & Yoskovitz, B. Lean Analytics, 2013).
You might assume your product has to be used daily to succeed, only to find out that’s not so. In these situations, it’s reasonable to update your metrics accordingly, provided that you’re able to prove the value created. Here are some best practices when picking metrics (Croll, A., & Yoskovitz, B. Lean Analytics, 2013):
- Know your customer. There’s no substitute for engaging with customers and users directly. All the numbers in the world can’t explain why something is happening. Pick up the phone right now and call a customer, even one who’s disengaged.
- Make early assumptions and set targets for what you think success looks like, but don’t experiment yourself into oblivion. Lower the bar if necessary, but not for the sake of getting over it: that’s just cheating.
- Use qualitative data to understand what value you’re creating and adjust only if the new line in the sand reflects how customers (in specific segments) are using your product.
Jobs to be Done as a Common Strategy Vocabulary for Managing by Outcomes
At its core, the concept of JTBD is straightforward and focus on people’s objectives independent of the means used to accomplish them. Through this lens, JTBD offers a structured way of understanding customer needs, helping to predict better how customers might act in the future (Kalbach, J. Jobs to be Done Playbook, 2020).
From that perspective, there are a few ways that my colleagues and I have been taking full benefit of the advantages of using Jobs to be Done for creating a shared understanding of product outcomes and customer outcomes:
- Discuss problems instead of solutions: because Jobs to be Done are a great way of describing Customer and/or User Needs independent of Solutions, they are — directly or indirectly — a great way to help create the mindset (that designers keep complaining about that lack in product teams) of understanding the problem space before jumping to solutions.
- Provide clarity and scope of product discovery activities: because uncovering Jobs to be Done requires clarity about who are the job performers we need to focus our research on, and what are the things these job performers are trying to get done, it makes it a lot easier for us to prepare for research, especially when it comes to recruiting research participants and what are the problems we need to probe.
- Provide a great way to synthesize discovery findings: if you know me, you know that my design and research philosophy is to get everyone involved during the processing of research data (instead of long reports for stakeholders to read out about research). But even when you do the processing of the research data together with the team, Jobs to be Done are great for synthesizing research outputs (e.g..: user needs, goals, success criteria) in a format that is easy to consume — for example — with Job Stories (coming up next)!
Jobs to be Done, Product Outcomes, and Customer Outcomes
In this process of translating outputs into themes, one could argue that — if you’re trying to innovate by creating features that customers want — it becomes tough to tell where customers’ outcomes should stop, and product outcomes should start.
Translating Customer Outcomes into Job Stories
A key part of Agile is to break down efforts into individual units of work. User stories are short descriptions of features and functionality written from the perspective of the end user. Teams can focus on only a small part of the whole and make progress in a controlled way. User stories are commonly written in a three-part format. The first element indicates a user’s role in the system. The second points to a capability that enables the person to get a task done. The last part often describes a benefit or reason for using the capability (Kalbach, J. Jobs to be Done Playbook, 2020).
The format for user stories is typically (Lombardo, C. T., McCarthy, B., Ryan, E., & Connors, M., Product Roadmaps Relaunched, 2017):
As a [user type]
I want [desire]
So I can [result]
Although user stories are good for breaking down work, they typically fail to connect the solution being built with user needs (Kalbach, J. Jobs to be Done Playbook, 2020).
User Stories lack an indication of why someone would behave in a certain way and what they need to get a job done. In fact, often user stories are derived from the capability being built, not from observing actual behaviour.Kalbach, J. Jobs to be Done Playbook (2020)
Job stories are an alternative to user stories. They follow the tradition of breaking down efforts into smaller pieces, but through the JTBD lens. The technique was first pioneered by the product development team at Intercom, a leading marketing communications solution. They wanted to avoid leading designers with a preconceived solution, as well as tying development to the company vision and strategy (Kalbach, J. Jobs to be Done Playbook, 2020).
The generally accepted format for a job story is (Lombardo, C. T., McCarthy, B., Ryan, E., & Connors, M., Product Roadmaps Relaunched, 2017):
I need [desire]
So I can [result]
Examples of job stories include (Kalbach, J. Jobs to be Done Playbook, 2020):
- When an important new customer signs up, I want to be notified so that I can start a conversation with that person.
- When I visit someone’s profile page, I want to see how many posts they have in each topic so that I have an understanding of where they have the most knowledge.
- When I have used the application multiple times, I get nudged to contribute so that I am encouraged to participate.
If you look closely, these job stories can inform the scrum team — while they are writing user stories — of your customers or user goals and their success criteria: since my colleagues are using Job Stories to synthesize user research findings, scrum teams can have more confidence that their user stories are based on real user input.
In an upcoming post, I’ll be talking about how I’ve been coaching Agile Teams (with the help of the colleagues Ritesh Chopra, Kevin Simmons, and Rebecca Thieme-Reagan and the incredible facilitation skills of Martina William and Anton Fischer) to connect the dots between Product Vision, Product and Customer Outcomes, Roadmap Planning and Backlog Planning through combining Storytelling, Job Maps and User Story Mapping.
Managing by Outcomes as a Two-way Negotiation
Setting a team’s outcome should be a two-way negotiation between the product leader (e.g., Chief Product Officer, Vice President of Product, etc.) and the product trio (Torres, T., Continuous Discovery Habits, 2021):
- The product leader brings the across-the-business view of the organization to the conversation and should communicate what’s most important for the business at this moment in time. But to be clear, the product leader should not be dictating solutions. Instead, the leader should be identifying an appropriate product outcome for the trio to focus on. Outcomes are a good way for the leader to communicate strategic intent. For example, if Sonja’s team is focused on increasing the number of dogs who like the food, her product leader can encourage her to keep focusing on the number of dogs who like the food broadly. Or, based on the strategic needs of the business, the leader might refine this outcome to have the team focus on specific breeds or strategic geographic regions. The key is that the leader should not narrow the scope so much that the team is tasked with a traction metric—engagement with the transition calendar.
- The product trio brings customer and technology knowledge to the conversation and should communicate how much the team can move the metric in the designated period of time (usually one calendar quarter). The trio should not be required to communicate what solutions they will build at this time, as this should emerge from discovery. For example, Sonja’s team, if asked to focus on a specific customer segment, might summarize what they know about that customer segment, share how successful past attempts to get dogs to like the food in that customer segment have been, and estimate how much impact they can have on the metric in the designated time period (e.g., we can increase the number of dogs in that segment who like the food by 10% in the next three months).
On the other hand, the product leader and product trio can negotiate resources (e.g., adding engineers to the team) and/ or remove competing tasks from the team’s backlog, giving them more time to focus on delivering their outcome (Torres, T., Continuous Discovery Habits, 2021).
A particular scenario of note is when teams are assigned an outcome for the first time: the product trio will need some time to learn what might move the metric. This is why a stable product trio focused on the same outcome over time is so critical. Every time we mix up the team or change the outcome, we take a learning tax as the team gets up to speed (Torres, T., Continuous Discovery Habits, 2021).
Commitment to action is built on participation and ownership. Organizations and business scholars have long puzzled over how to incentivize this sense of ownership, which is central to building commitment to action. Stock ownership plans, performance-based pay, and related schemes have all been tried. These have merit, but in the end, monetary rewards matter less to individuals than participating in the decisions that they are asked to implement (Spetzler, C., Winter, H., & Meyer, J., Decision quality: Value creation from better business decisions, 2016).
Simply put, a quality decision requires commitment from two Parties: the people who have the power to decide, allocate resources, and support their choices; and those who will lead the implementation. Both parties must have the opportunity to participate in the decision process. When the implementers are included in the decision process, they (Spetzler, C., Winter, H., & Meyer, J., Decision quality: Value creation from better business decisions, 2016):
- Suggest new alternatives.
- Provide insights and information from their unique perspectives
- Help gather information. Evaluate feasibility and identity potential execution failures.
- Explore and share their perspectives about the decision’s value drivers, thereby preparing to make value-driven decisions during implementation.
Roadmaps and Two-way Negotiations
In today’s fast-paced and constantly changing world, there is a natural desire for certainty and predictability. In the past, roadmaps often read like a Gantt chart, with specific dates and commitments of features. However, because we now live in an agile world, such commitments and dates are often missed, leaving customers and stakeholders disappointed. Symptoms of this particular problem include (Lombardo, C. Todd, Connors, M. S., McCarthy, B., and Ryan, E., Product Roadmaps Relaunched, 2017):
- The roadmap includes a list of features and dates that are frequently unable to be realized.
- A product team is scrambling in the last few weeks/days toward a product release.
A good roadmap is not so much a project plan as a strategic communication tool, a statement of intent and direction. A roadmap is a strategic document that should offer guidance to your teams on what to focus on (Lombardo, C. Todd, Connors, M. S., McCarthy, B., and Ryan, E., Product Roadmaps Relaunched, 2017).
A product roadmap should (Lombardo, C. Todd, Connors, M. S., McCarthy, B., and Ryan, E., Product Roadmaps Relaunched, 2017):
- Put the organization’s plans in a strategic context
- Focus on delivering value to customers and the organization
- Embrace learning as part of a successful product development process
- Rally the organization around a single set of priorities
- Get customers excited about the product’s direction
At the same time, a product roadmap should not (Lombardo, C. Todd, Connors, M. S., McCarthy, B., and Ryan, E., Product Roadmaps Relaunched, 2017):
- Make promises product teams aren’t confident they will deliver on
- Require a wasteful process of up-front design and estimation
- Be conflated with a project plan or a release plan (we cannot stress this enough)
If you’ve done a really great job in your customer discovery, then the roadmap is merely “confirming the mutual understanding” of these needs. It’s also an opportunity to discover where you might be wrong, of course, which is perhaps even more valuable as you still have the opportunity to change direction — not so once the product is built. (Lombardo, C. Todd, Connors, M. S., McCarthy, B., and Ryan, E., Product Roadmaps Relaunched, 2017).
This brings us to what can be a difficult conversation to be had: even if product teams “know better” through discovery and learn what the desired outcomes from their customers’ perspective are, it is going to be difficult to have genuine two-way negotiation around managing by outcomes with product leaders if their roadmaps are fixed.
A fixed roadmap communicates false certainty. It says we know these are the right features to build, even though we know from experience their impact will likely fall short (Torres, T., Continuous Discovery Habits, 2021)
If the product trio finds flaws with their initial solution, they can quickly shift to a new idea, often trying several before they ultimately find what will drive the desired outcome (Torres, T., Continuous Discovery Habits, 2021).
This brings us to a point in the work of design strategist that can make or break the journey toward outcomes instead of outputs: influencing the way stakeholders create their corporate strategy and helping them shift from Outputs to Outcomes is probably one of the best ways to help teams solve the right problems.
In a previous post, I argued that designers hadn’t found the vocabulary and tools to frame users’ problems in a way that aligns with stakeholder business strategies.
From that perspective, the role of designers must change. We must move from focusing only on specific innovation projects and design briefs to involvement in strategic decisions that influence and shape organizational strategy.
If designers are to take a more active role in shaping organizational strategy, how can they do that? Further, what skill set do they need to acquire to think more strategically?
If you’re ready for the challenge, this journey towards outcomes versus outputs will eventually lead to revisiting how roadmaps are created.
Lombardo et al. distinguish between primary and secondary components of the roadmap. The primary components are necessary for an effective roadmap:
- Product vision — This is the overarching vision that guides the product roadmap.
- Business objectives — What kind of product do we need in order to attain those business goals? Having well-defined goals on the roadmap will help you and your organization to measure progress.
- Broad timeframes — Broad timeframes like calendar quarters or Now, Next, and Later offer guidance about timings without committing to very specific deadlines.
- Themes — They answer the question “What would need to be true for our product to realize its vision and attain its business activities?” Themes can be defined as customer needs or problems for the product to address. Themes focus on outcomes rather than outputs.
- Disclaimer—Roadmaps can have a caveat just to make it very clear to any stakeholders, other teams, etc. that anything in the roadmap is subject to change and evolve.
Here is — yet again — another advantage of using Jobs to be Done to frame our value proposition, especially with regards to product vision: because the product vision communicates why you are building something and what the value proposition is for the customer if we focus on the customers’ outcomes can deliver the most significant value we can be confident to push our vision horizon far out, knowing that the outcomes our vision are trying to address are stable over time.
Secondary components (Lombardo, C. Todd, Connors, M. S., McCarthy, B., and Ryan, E., Product Roadmaps Relaunched, 2017):
- Features — as we discussed extensively by now, we should avoid having lots of features on a roadmap, mostly because it will limit the team to come up with solutions, with people expecting whatever is on the roadmap to get delivered. Features and solutions are the specific deliverables that will fulfill the needs and solve the problems identified in the roadmap themes.
- Stage of development — By including labels such as “discovery”, “design”, or “prototyping” on a roadmap, stakeholders and other people not close to day-to-day product development — should be able to see at a glance where the product is at.
- Confidence — Indicating the level of confidence you have in your availability to address each item or theme on the roadmap in the next release is a great way to help offset the sentiment that once it’s on paper, it’s a promise.
- Target customers — Highlighting which customer segment(s) your product is looking to address, really helps with the ‘communication’ aspect of your product roadmap. Instead of just seeing a bunch, you can now tell more of a story about upcoming themes and their impact on specific customers.
- Product areas — A large and complex product — or a new product where basic functionality is still being laid down in many areas — many benefit from a roadmap where themes or features are annotated per specific area of the product.
If you’re paying attention, you’ll notice this is the same solution to a different problem mentioned earlier. We find that inexperienced product pros often jump to the solution. After all, we’ve been acculturated to have “the answer” since we were in kindergarten. It’s a natural bias we all have—we want the answer and we want to be problem solvers. The issue is that we go right to the solution (output) rather than do what experienced product pros do, which is focus on the problem (outcome). To take it a step further, the smartest product pros ask what outcomes they’re seeking and drive toward those (Lombardo, C. Todd, Connors, M. S., McCarthy, B., and Ryan, E., Product Roadmaps Relaunched, 2017).
Outcomes and Roadmaps Objectives
Here are a few guidelines on OKRs as they apply to product road mapping (Lombardo, C. T., McCarthy, B., Ryan, E., & Connors, M., Product Roadmaps Relaunched, 2017):
- Everything on the roadmap must be tied to at least one of your objectives.
- Stick to a manageable number of objectives; from our experience and research, fewer than five seem to be most effective.
- Focus on outcomes, not outputs.
Think “What kind of product (solution) do we need to attain those business objectives?” For example, work on improved usability will increase customer lifetime value (LTV) by decreasing churn. The goal for the customer is a better user experience; the goal for the product is decreased churn and increased LTV (Lombardo, C. T., McCarthy, B., Ryan, E., & Connors, M., Product Roadmaps Relaunched, 2017).
Each of these outcomes is tied to an objective that can then be measured by specific key results, and their roadmap reflects this. This internal objectives-driven approach to road mapping makes direction crystal-clear for internal teams. In this case, rather than being based on customer needs, each objective focuses on a measurable business benefit that the work of the product team is expected to achieve (higher average sale price or higher renewal rate, for example). This “outcome-based” roadmap makes clear why certain work by the product team is important (Lombardo, C. T., McCarthy, B., Ryan, E., & Connors, M., Product Roadmaps Relaunched, 2017).
Outcomes and Roadmaps Themes
Many product people are used to filling roadmaps with features and solutions, and we can attest from experience that old habits die hard. The key question to ask yourself when faced with a list of concrete deliverables is why? Why are these things important? What will result from doing them? How will the fortunes of the company or the happiness of the customer improve? Why would we do this at all? (Lombardo, C. Todd, Connors, M. S., McCarthy, B., and Ryan, E., Product Roadmaps Relaunched, 2017).
|HMTL5 Redesign||Works better on mobile||Make mobile experience as good as desktop|
|Twitter and Facebook integration||Allow customers to promote the product by sharing results||Make it easy and fun for users to promote our product|
|Infrastructure work for scalability||App slows down under heavy traffic||Ensure access and that we can fulfill peak demand|
By asking yourself (or whoever is requesting a particular feature) why, you are attempting to discern the difference between the output requested and the result or outcome desired. In other words, you are attempting to distinguish the end from the means (Lombardo, C. Todd, Connors, M. S., McCarthy, B., and Ryan, E., Product Roadmaps Relaunched, 2017).
Maybe instead of an HTML5 redesign of the whole site, certain core functions mobile users like to perform on the go could be ported into a native iOS or Android app. That might be less work and more effective in achieving the result described in the theme. (Lombardo, C. Todd, Connors, M. S., McCarthy, B., and Ryan, E., Product Roadmaps Relaunched, 2017).
Even though these two terms suggest subtle differences, the important point is that both refer to a gap or pain in the customer’s experience. When identifying the themes and sub-themes for your roadmap, remember to consider both needs and problems from all angles (Lombardo, C. T., McCarthy, B., Ryan, E., & Connors, M., Product Roadmaps Relaunched, 2017).
Here is — again — how Jobs to be Done work as a great way of translating customer pains and gains into roadmap themes: because they describe Customer and User Needs independent of Solutions, Jobs to be Done are a great way to help create the mindset of understanding the problem space before jumping to solutions.
Facilitating Two-way Negotiations through Visual Thinking
Mark Dziersk is convinced that “design” and “strategy” traditionally reflect two very disparate realms within the world of business. He urges designers to communicate with those responsible for strategy by using their talent for visualization and storytelling — “languages” that can powerfully convey content in such areas as the DNA of the consumer experience, innovation options, and approaches to decision making (Dziersk, M., “Visual Thinking: A leadership Strategy” in Building Design Strategy, Lockwood, T., & Walton, T., 2010).
The truth is that very few designers understand strategy, much less leverage in their work. But the design world is trying and is making inroads. Dealing with and converting ambiguity to clearly focused design strategy is key and gives design thinking the leverage for running a come-give business in the post-dot.com, post “distribution dictate direction” business world we live in (Dziersk, M., “Visual Thinking: A leadership Strategy” in Building Design Strategy, Lockwood, T., & Walton, T., 2010).
Visualizing thought processes can help break down complex problems. It empowers teams and staff to build on one another’s ideas, fosters collaboration, jump-starts co-creation, and boosts innovation.
Furthermore, to really have an impact during discussions and decision points so that they’ll be remembered forever, capture what’s been said (at least some of it) visually (Van Der Pijl, et al. Design a better business, 2016).
If designers want to influence the strategic decisions that drive product vision forward, I’ve found that using alignment diagrams can be a great way to get teams to create a shared understanding of the problems we are trying to solve and the solutions that will address those problems, helping teams transition between problem space and solution space at the correct times.
Visual Thinking and Alignment Diagrams
Misalignment impacts the entire enterprise: teams lack a common purpose, solutions are built that are detached from reality, and strategy is short-sighted (Kalbach, J., ”Visualizing Value: Aligning Outside-in” in Mapping Experiences, 2021).
Alignment Diagrams coordinate insights from the outside world with the teams inside an organization who create products and services to meet market needs.
In other words, Alignment diagrams or models serve as a hinge upon which we can pivot from the problem space to the solutions space.
Jim Kalbach uses the term alignment diagram to refer to any map, diagram, or visualization that reveals both sides of value creation in a single overview. They are a category of diagrams that illustrates the interaction between people and organizations.
Here is — yet again — another advantage of using Jobs to be Done to facilitate two-way negotiations that allow for Managing by Outcomes: because the customers buy your “why,” not your “what,” Jobs to be Done provide an objective way for us to visualize how our organization’s core values align with those of our customers/users.
There are a few alignment diagrams that I find particularly helpful for facilitating discussions around managing by outcomes.
Customer Journey Maps
Customer Journey Maps are visual thinking artifacts that help you get insight into, track, and discuss how a customer experiences a problem you are trying to solve. How does this problem or opportunity show up in their lives? How do they experience it? How do they interact with you? (Lewrick, M., Link, P., & Leifer, L., The design thinking playbook. 2018)
Experience Maps look at a broader context of human behavior. They reverse the relationship and show how the organization fits into a person’s life (Kalbach, J., ”Visualizing Value: Aligning Outside-in” in Mapping Experiences, 2021).
Lean UX Canvas
Lean UX Canvas codifies the Lean UX process to help teams frame their work as a business problem to solve (rather than a solution to implement) and then dissect that business problem into its core assumptions. We then weave those assumptions into hypotheses. Finally, we design experiments to test our riskiest hypotheses (Gothelf, J., & Seiden, J., Lean UX: Applying lean principles to improve user experience, 2021).
Many teams generate a lot of ideas when they go through a journey-mapping or experience-mapping exercise. There are so many opportunities for improving things for the customer that they quickly become overwhelmed by a mass of problems, solutions, needs, and ideas without much structure or priority (“Opportunity-Solution Tree” in Product Roadmaps Relaunched, Lombardo, C. T., McCarthy, B., Ryan, E., & Connors, M., 2017).
Opportunity-Solution Trees (OST) are a simple way of visually representing the paths you might take to reach a desired outcome (Torres, T., Continuous Discovery Habits, 2021):
- The root of the tree is your desired outcome—the business need that reflects how your team can create business value.
- Below the opportunity space is the solution space. This is where we’ll visually depict the solutions we are exploring.
- Below the solution space are assumption tests. This is how we’ll evaluate which solutions will help us best create customer value in a way that drives business value.
Opportunity solution trees have a number of benefits. They help product trios (Torres, T., Continuous Discovery Habits, 2021):
- Resolve the tension between business needs and customer needs
- Build and maintain a shared understanding of how they might reach their desired outcome
- Adopt a continuous mindset
- Unlock better decision-making
- Unlock faster learning cycles
- Build confidence in knowing what to do next
- Unlock simpler stakeholder management
Here are the four steps to creating an Opportunity Solution Tree (ProductPlan, “Opportunity Solution Tree”, 2022):
Step 1: Identify the desired outcome: Narrow your goal to a single metric you want to improve (e.g., revenue, customer satisfaction, retention, etc.).
Step 2: Recognize opportunities that emerge from generative research: Dig in deep to understand the needs and pain points of your customers. Keep in mind that pain points are opportunities!
Step 3: Be open to solutions from everywhere: The caveat, however, is that a potential solution must directly link to an opportunity—otherwise, it’s just a distraction from the primary goal of your OST.
Step 4: Experiment to evaluate and evolve your solutions: Now, it’s time to test a single solution with sets of experiments.
Like highway maps that show towns and cities and the roads, connecting them, Impact Maps layout out what we will build and how these connect to ways we will assist the people who will use the solution. An impact map is a visualisation of the scope and underlying assumptions, created collaboratively by senior technical people and business people. It’s a mind-map grown during a discussion facilitated by answering four questions: WHY, WHO, HOW and WHAT of the problem the team is confronting (Adzic, G., Impact Mapping, 2012)
User Story Maps
User story mapping is a visual exercise that helps product managers and their development teams define the work that will create the most delightful user experience. User Story Mapping allows teams to create a dynamic outline of a set of representative user’s interactions with the product, evaluate which steps have the most benefit for the user, and prioritise what should be built next (Patton, J., User Story Mapping: Discover the whole story, build the right product, 2014).
Jeff Patton is one of the few who has been able to translate Agile into a User Centric practice. User Story Mapping is probably my favorite visualisation tool to create shared understanding around product, users, and context, and it helps with prioritization discussions.
Mental models are affinity diagrams of behaviors made from ethnographic data gathered from audience representatives. They give you a deep understanding of people’s motivations and thought processes, along with the emotional and philosophical landscape in which they operate (Young, I., Mental Models, 2008).
Service Blueprints are visual thinking artifacts that help to capture the big picture and interconnections. They are a way to plan projects and relate service design decisions to the original research insights. The blueprint is different from the service ecology in that it includes specific details about the elements, experiences, and delivery within the service (Polaine, A., Løvlie, L., & Reason, B., Service design: From insight to implementation, 2013).
Value Stream Mapping is a practical and highly effective way to lean to see and resolve disconnects, redundancies, and gaps in how work gets done (Martin, K., & Osterling, M., Value stream mapping, 2014)
Strategy Canvas help you compare how well competitors meet costumer buying criteria or desired outcomes. To create your own strategy canvas, list the 10-12 most important functional desired outcomes — or buying criteria — on the x-axis. On the y-ais, list the 3-5 most common competitors (direct, indirect, alternative solutions and multi-tools solutions) for the job. (Garbugli, É., Solving Product, 2020).
Anti-patterns for Managing by Outcomes
When setting product outcomes, avoid these common anti-patterns for managing by outcomes discussions (Torres, T., Continuous Discovery Habits, 2021):
- Pursuing too many outcomes at once. Most of us are overly optimistic about what we can achieve in a short period of time. No matter how hard we work, our companies will always ask more of us. Put these two together, and we often see product trios pursuing multiple outcomes at once. What happens when we do this is that we spread ourselves too thin. We make incremental progress (at best) on some of our outcomes but rarely have a big impact on any of our outcomes. Most teams will have more of an impact by focusing on one outcome at a time.
- Ping-ponging from one outcome to another. Because many businesses have developed fire-fighting cultures—where every customer complaint is treated like a crisis—it’s common for product trios to ping-pong from one outcome to the next, quarter to quarter. However, you’ve already learned that it takes time to learn how to impact a new outcome. When we ping-pong from outcome to outcome, we never reap the benefits of this learning curve. Instead, set an outcome for your team, and focus on it for a few quarters. You’ll be amazed at how much impact you have in the second and third quarters after you’ve had some time to learn and explore.
- Setting individual outcomes instead of product-trio outcomes. Because product managers, designers, and software engineers typically report up, to their respective departments, it’s not uncommon for a product trio to get pulled in three different directions, with each member tasked with a different goal. Perhaps the product manager is tasked with a business outcome, the designer is tasked with a usability outcome, and the engineer is tasked with a technical-performance outcome. This is most common in companies that tie outcomes to compensation. However, it has a detrimental effect. The goal is for the product trio to collaborate to achieve product outcomes that drive business outcomes. This isn’t possible if each member is focused on their own goal. Instead of setting individual outcomes, set team outcomes.
- Choosing an output as an outcome. Shifting to an outcome mindset is harder than it looks. We spend most of our time talking about outputs. So, it’s not surprising that we tend to confuse the two. Even when teams intend to choose an outcome, they often fall into the trap of selecting an output. I see teams set their outcome as “Launch an Android app” instead of “Increase mobile engagement” or “Get to feature parity on the new tech stack” instead of “Transition customer to the new tech stack.” A good place to start is to make sure your outcome represents a number even if you aren’t sure yet how to measure it. But even then, outputs can creep in. I worked with a team that helped students choose university courses who set their outcome as “Increase the number of course reviews on our platform.” When I asked them what the impact of more reviews was, they answered, “More students would see courses with reviews.” That’s not necessarily true. The team could have increased the number of reviews on their platform, but if they all clustered around a small number of courses, or if they were all on courses that students didn’t view, they wouldn’t have an impact. A better outcome is “Increase the number of course views that include reviews.” To shift your outcome from less of an output to more of an outcome, question the impact it will have.
- Focusing on one outcome to the detriment of all else. As we saw in the Wells Fargo story, focusing on one metric at the cost of all else can quickly derail a team and company. In addition to your primary outcome, a team needs to monitor health metrics to ensure they aren’t causing detrimental effects elsewhere. For example, customer-acquisition goals are often paired with customer-satisfaction metrics to ensure that we aren’t acquiring unhappy customers. To be clear, this doesn’t mean one team is focused on both acquisition and satisfaction at the same time. It means their goal is to increase acquisition without negatively impacting satisfaction.
The Right Time for Managing by Outcomes Discussions
You might be asking yourself “These are all great, but when should I be doing what?”. Without knowing what kind of team set up you have, and what kinds of processes you run in your organization, the best I can do is to map all of the techniques above the the Double Diamond framework.
The Double Diamond Framework
Design Council’s Double Diamond conveys a design process to designers and non-designers. The two diamonds represent a process of exploring an issue more widely or deeply (divergent thinking) and then taking focused action (convergent thinking).
- Discover. The first diamond helps people understand, rather than simply assume, what the problem is. It involves speaking to and spending time with people who are affected by the issues.
- Define. The insights gathered from the discovery phase can help you to define the challenge in a different way.
- Develop. The second diamond encourages people to give different answers to the clearly defined problem, seeking inspiration from elsewhere and co-designing with a range of different people.
- Deliver. Delivery involves testing out different solutions at small-scale, rejecting those that will not work and improving the ones that will.
Map of Managing by Outcomes Activities and Methods
Process Awareness characterizes the degree to which the participants understand the process procedures, rules, requirements, workflow, and other details. The higher the process awareness, the more profoundly they will engage in the process, so the better results they deliver.
In my experience, the most significant disconnect between the work designers need to do, and the mindset of every other team member in a team is usually about how quickly we tend — when not facilitated — to jump to solutions instead of contemplating and exploring the problem space a little longer.
Knowing when the team should be diverging, when they should be exploring, and when they should closing will help ensure they get the best out of their collective brainstorming and multiple perspectives’ power and keep the team engaged.
My colleagues Edmund Azigi and Patrick Ashamalla have created a great set of questions and a cheat sheet that maps which questions are more appropriate for different phases of the product development lifecycle. So the following set of activities is inspired by their cheat sheet.
Managing by Outcomes Discussions during “Discover”
This phase has the highest level of ambiguity, so creating shared understanding is critical. While a degree of back and forth is expected and Managing by Outcomes discussions might be too early, you can still move to clarity faster by having a strong shared vision, good problem framing, and clear priorities defined through outcomes upfront.
Here are my recommendations for frameworks, methods, and activities to ensure you are solving the right problems and provide the insights that help you with managing by outcomes:
- User Research
- Hypothesis Writing
- Problem Framing
- Challenge Briefs
- Value Proposition Design
- Jobs to be Done (JTBD)
- Testing Business Ideas
- A Value Opportunity Analysis (VOA)
- Desirability Testing
Managing by Outcomes Discussions during “Define”
In this phase, we should see the level of ambiguity diminishing, and facilitating investment discussions have the highest payoff in mitigating back-and-forth. Helping the team make good decisions by creating great choices is critical.
Here are my recommendations for frameworks, methods, and activities to answer the questions that help you create great choices and provide the insights that help you with managing by outcomes:
- User Story Mapping
- Design Sprints / Studio
- Concept Validation
- Outcome-Driven Innovation / JTBD
- Importance vs. Satisfaction Framework
- Kano Model
- Objectives, Goals, Strategy & Measures (OGSM)
- Product Backlog & Sprint Planning
Managing by Outcomes Discussions during “Develop”
In this phase, we are going to a point where the cost of changing your mind increases rapidly as time passes. The team should focus on learning as cheap as possible (by capturing signals from the market), and discussions around investment should answer whether we should pivot, persevere, or stop.
Here are my recommendations for frameworks, methods and activities to decide if you should pivot, persevere, or stop, and provide the insights that help you with managing by outcomes:
- User Story Mapping
- Design Studio
- Collaborative Prototyping
- UXI Matrix (Pugh Matrix)
- Usability Testing
- Usefulness, Satisfaction, and Ease of Use (USE)
- American Customer Satisfaction Index (ACSI)
- System Usability Scale (SUS)
- Usability Metric for User Experience (UMUX)
Managing by Outcomes Discussions during “Deliver”
In this phase, the visibility and traceability systems should collect data from actual customer usage and make hard choices about pivoting, persevering or stopping on the product’s next iteration.
On the other hand — since the product is now in the hands of customers and users — we should be able to collect the richest data from live usage that can inform decisions about our viability hypothesis, enabling you to adjust strategic choices accordingly.
Here are my recommendations for frameworks, methods, and activities to help you trace back the outputs to outcomes and provide the insights that help you with managing by outcomes:
- Designer – Developer Pairing
- Pirate Metrics (a.k.a. AARRR!)
- UXI Matrix (Pugh Matrix)
- Objectives, Goals, Strategy & Measures (OGSM)
Facilitating discussions for Managing by Outcomes
I think designers should facilitate the discussions and help others raise awareness around the creative and problem-solving processes instead of complaining that everyone else is jumping into solutions too quickly.
I’ll argue for the need for facilitation in the sense that — if designers want to influence the decisions that shape strategy — they must step up to the plate and become skilled facilitators that respond, prod, encourage, guide, coach, and teach as they guide individuals and groups to make decisions that are critical in the business world through effective processes.
That said, I’d argue that facilitation here does not only mean “facilitating workshops,” but facilitating the decisions regardless of the required activities.
Adzic, G. (2012). Impact Mapping: Making a big impact with software products and projects (M. Bisset, Ed.). Woking, England: Provoking Thoughts.
Berger, W. (2019). The book of beautiful questions: The powerful questions that will help you decide, create, connect, and lead. New York, NY: Bloomsbury Publishing.
Bland, D. J., & Osterwalder, A. (2020). Testing business ideas: A field guide for rapid experimentation. Standards Information Network.
Brand, W. (2017). Visual thinking: Empowering people & organizations through visual collaboration. Amsterdam, Netherlands: BIS Publishers B.V.
Brown, T., & Katz, B. (2009). Change by design: how design thinking transforms organizations and inspires innovation. [New York]: Harper Business
Cagan, M. (2017). Inspired: How to create tech products customers love (2nd ed.). Nashville, TN: John Wiley & Sons.
Cagan, M. (2020). The origin of product discovery. Retrieved February 7, 2022, from Silicon Valley Product Group website: https://svpg.com/the-origin-of-product-discovery/
Calabretta, G., Gemser G., Karpen, I., (2016) “Strategic Design: 8 Essential Practices Every Strategic Designer Must Master“, 240 pages, BIS Publishers; 1st edition (22 Nov. 2016)
Cutler, J. (2022). Making things better (with enabling constraints and POPCORN). Retrieved March 27, 2022, from The Beautiful Mess website: https://cutlefish.substack.com/p/making-things-better-with-enabling?s=w
Design Council. (2015, March 17). What is the framework for innovation? Design Council’s evolved Double Diamond. Retrieved August 5, 2021, from designcouncil.ork.uk website: https://www.designcouncil.org.uk/news-opinion/what-framework-innovation-design-councils-evolved-double-diamond
Dziersk, M., (2010), “Visual Thinking: A leadership Strategy” in Building Design Strategy. Lockwood, T., & Walton, T., Allworth Press.
Doerr, J. (2018). Measure what matters: How Google, Bono, and the gates foundation rock the world with okrs. Portfolio.
Garbugli, É. (2020). Solving Product: Reveal Gaps, Ignite Growth, and Accelerate Any Tech Product with Customer Research. Wroclaw, Poland: Amazon.
Gothelf, J., & Seiden, J. (2017). Sense and respond: How successful organizations listen to customers and create new products continuously. Boston, MA: Harvard Business Review Press.
Gothelf, J. (2017). Execs care about revenue. How do we get them to care about outcomes? Retrieved July 23, 2022, from Jeff Gothelf website: https://jeffgothelf.com/blog/execs-care-about-revenue-how-do-we-get-them-to-care-about-outcomes/
Gothelf, J., & Seiden, J. (2021). Lean UX: Applying lean principles to improve user experience. Sebastopol, CA: O’Reilly Media.
Gothelf, J. (2021). OKRs at scale. Retrieved July 23, 2022, from Jeff Gothelf website: https://jeffgothelf.com/blog/okrs-at-scale/
Groen, B., Wilderom, C., & Wouters, M. (2017). High Job Performance Through Co-Developing Performance Measures With Employees. Human Resource Management, 56(1), 111–132.
Juarrero, Alicia. 1999. Dynamics in Action: Intentional Behavior as a Complex System. MIT Press
Kalbach, J. (2020). Jobs to be Done Playbook (1st Edition). Two Waves Books.
Kalbach, J. (2021). Mapping Experiences (2nd Edition). Sebastopol, CA: O’Reilly Media.
Lewrick, M., Link, P., & Leifer, L. (2018). The design thinking playbook: Mindful digital transformation of teams, products, services, businesses and ecosystems. Nashville, TN: John Wiley & Sons.
Lombardo, C. T., McCarthy, B., Ryan, E., & Connors, M. (2017). Product Roadmaps Relaunched. Sebastopol, CA: O’Reilly Media.
Martin, K., & Osterling, M. (2014). Value stream mapping: How to visualize work and align leadership for organizational transformation. New York, NY: McGraw-Hill Professional.
Matts, C. (2018). Constraints that enable. Retrieved March 27, 2022, from The IT Risk Manager website: https://theitriskmanager.com/2018/12/09/constraints-that-enable/
McCarthy, B. (2019). How should product teams use OKRs? Retrieved July 23, 2022, from Product Culture website: https://www.productculture.org/articles/2019/6/1/how-should-product-teams-use-okrs
Mueller, S., & Dhar, J. (2019). The decision maker’s playbook: 12 Mental tactics for thinking more clearly, navigating uncertainty, and making smarter choices. Harlow, England: FT Publishing International.
Oberholzer-Gee, F. (2021). Eliminate Strategic Overload. Harvard Business Review, (May-June 2021), 11.
Osterwalder, A., Pigneur, Y., Papadakos, P., Bernarda, G., Papadakos, T., & Smith, A. (2014). Value proposition design: How to create products and services customers want. John Wiley & Sons.
Patton, J. (2014). User Story Mapping: Discover the whole story, build the right product (1st ed.). Sebastopol, CA: O’Reilly Media.
Perri, M. (2019). Escaping the build trap. Sebastopol, CA: O’Reilly Media.
Pichler, R. (2016). “Choose the Right Key Performance Indicators” in Strategize: Product strategy and product roadmap practices for the digital age. Pichler Consulting.
Podeswa, H. (2021). The Agile Guide to Business Analysis and Planning: From Strategic Plan to Continuous Value Delivery. Boston, MA: Addison Wesley.
Polaine, A., Løvlie, L., & Reason, B. (2013). Service design: From insight to implementation. Rosenfeld Media.
ProductPlan. (2022). Opportunity Solution Tree. Retrieved May 19, 2022, from Productplan.com website: https://www.productplan.com/glossary/opportunity-solution-tree/
Seiden, J. (2019). Outcomes Over Output: Why customer behavior is the key metric for business success. Independently published (April 8, 2019).
Spiek, C., & Moesta, B. (2014). The Jobs-to-be-Done Handbook: Practical techniques for improving your application of Jobs-to-be-Done. Createspace Independent Publishing Platform.
Torres, T. (2021). Continuous Discovery Habits: Discover Products that Create Customer Value and Business Value. Product Talk LLC.
Tullis, T., & Albert, W. (2013). Measuring the user experience: Collecting, analyzing, and presenting usability metrics (2nd edition). Morgan Kaufmann.
Ulwick, A. (2005). What customers want: Using outcome-driven innovation to create breakthrough products and services. Montigny-le-Bretonneux, France: McGraw-Hill.
Van Der Pijl, P., Lokitz, J., & Solomon, L. K. (2016). Design a better business: New tools, skills, and mindset for strategy and innovation. Nashville, TN: John Wiley & Sons.
Wodtke, C. R. (2021). Radical focus: Achieving your goals with objectives and key results (2nd ed.). Cucina Media.
Young, I. (2008). Mental models: Aligning design strategy with human behavior. Brookly, New York: Rosenfeld Media.