Você está na página 1de 30

Analyzing Opportunities

In 2004, two Serbian men and a Hungarian woman won $2 million from the Ritz Casino in London. Luck? Robbery? Or simply smart betting? How about a combination of all three. The team had a small camera that scanned the roulette wheel and the position of the ball when it was released by the croupier and sent this information to a remote location. Based on historical data of that croupier and a simulation model of the mechanics underlying the movement of the ball, a workstation recommended bets. Was the system able to perfectly forecast the outcome of the roulette? No, but it was allegedly able to improve the gambling odds from the traditional 37 to 1 to 6 to 1. In other words, by making use of data analysis and a clever process, a losers gamble was turned around into a profitable businessat least until the casino caught on. Financial evaluations of innovation opportunities at times can feel like betting in a casino. After all, many innovations fail, and most will not pay back their investment. But by following the methods that we outline below, you can emulate that trio of London gamblers and begin to turn the odds in your favor. By incorporating into your forecasts information about past innovations and the innovation process, you can increase your likelihood of success.
Horizon 1 Horizon 2 Horizon 3 Horizon 2 Nature of Uncertainty Type of Uncertainty Example of a Market Uncertainty
Uncertainty about a number, such as a sales forecast We think that sales volume for the new product will be between 120k and 150k per year.

Horizon 1

Uncertainty about a scenario, such as an approval or a cancellation We think there exists a 40% probability that the results of our pilot market warrant a full launch of the new product. Otherwise, the product will be cancelled. We think that the drugs toxicity might require the termination of all clinical trials with a probability of 20%.

Many types; not even clear which type matters The market for transportation devices smaller than cars seems to have big potential. However, we neither understand the exact user needs to peoples willingness to pay We might be able to improve the efficacy of drug x by customizing the treatment to the patient. However, we dont know how to go about this customization. version 080907

Example of a Technology Uncertainty

We think that the drug will have a 20%-30% higher efficacy than the drug currently on the market

Exhibit ESTIMATION FRAMEWORK: The three different types of opportunities deal with different types of uncertainty requiring different analytical tools in their evaluation.

As we discussed in the previous chapter, innovations can be classified into three groups, based on their level of uncertainty and time horizon. For Horizon 1 opportunities, your main concern is estimating the likely sales. For Horizon 2 opportunities, sales are also uncertain, but youre less concerned with that than with a bigger riskwhether opportunity will reach the market. Early cancellation is a real risk, and you have to factor that into your analysis. For Horizon 3 opportunities, the uncertainty is so great that a formal financial analysis is almost impossible. You should instead seek basic insights about the quality of the opportunity. Horizon 1 and Horizon 2 opportunities lend themselves to sophisticated analyses, such as stochastic calculations that may seem intimidating at first sight. We call this rocket science innovation management. Just as the trio at the Ritz benefited from modeling and analyzing thousands of previous spins of the roulette wheel, so too can you benefit by applying rigorous analysis to data from past innovations.

Evaluating Horizon 1 Opportunities When evaluating a Horizon 1 opportunity you want to create an expected discounted cash flow (DCF) model. You will also want to estimate the uncertainty of your DCF and can view the variance of the DCF as a measure of risk. You create a DCF model in five steps, which Exhibit HORIZON1 EVALUATION summarizes: (1) forecast the expected sales of the opportunity; (2) adjust the forecast based on the experience from prior opportunities; (3) compute the variance of the forecast; (4) use a typical product lifecycle pattern to estimate monthly (yearly) sales; (5) build an economic model of costs and revenues.

AO-2

From Screening and Scoring


Concept test with input from potential customers Compute variance based on old AF ratios

Define opportunity including target segment and price

Compute expected sales and adjust for past forecast errors

Aggregate opinions of multiple experts

Compute variance based on disagreement among experts.

Spread sales over the product lifecycle. Build a DCF model

To Portfolio Planning / Final Selection version 081407

Exhibit HORIZON1 EVALUATION: For each Horizon 1 opportunity coming from screening, five steps need to be taken to create an expected discounted cash flow.

Forming a sales forecast Entire books have been written about forecasting sales for of new products, so we will focus on the process more than the technical details. You can obtain a forecast using either of two basic methods: surveying customers or relying on experts. When doing a survey, identify customers in the market you are targeting with the opportunity. Then try to discover their purchase intent with questions such as, If a web service reminding you of important dates like your anniversary were available, how likely would you be to use it: Very, Somewhat, Maybe/maybe not, somewhat unlikely, very unlikely. From this, you can compute an average purchase probability and expected sales as shown in Exhibit CONCEPT TEST. Note that in the exhibit, the customers expressed purchase intent is discounted (a 0.4 weight is applied to definite buy, and a 0.2 weight is applied to probably buy). Research has shown that customers are much less likely to adopt an innovation compared with what they say in surveys. These sorts of parameters as well as adjustments for product awarenessyou can only purchase products you are aware ofdiffer widely by industry and should be obtained via market research.

AO-3

Survey to Customers

SpoilMy Spouse is an internet based service that will send an email or SMS to subscribers on a weekly basis offering ideas on how to make their spouse happy (e.g. bring her flowers, come home early, etc). These ideas are user-rated and recommendations are made based on the experience of like-minded subscribers. How likely are you to purchase subscription to this service?

Definitely would buy Results 10

Probably would buy 22 22/200

Might or might Definitely Probably not buy would not buy would not buy 48 48/200 59 59/200 61 61/200

Internal Analysis of Results

Proportion 10/200 Purchase Probability


10

0.4 x 200 + 0.2 x 200 = 0.042 Expected sales = Population size x Awareness x Purchase Probability = 4M x 0.2 x 0.042 = 33,600
version 080907

22

Exhibit CONCEPT TEST: To form a sales forecast, you estimate the average purchase probability for the population by learning the expressed purchase intent and then adjusting based on historical data. You then multiply with the number of potential adopters aware of the product1.

Alternatively, you can rely on experts to forecast sales. This method works best if your innovation resembles products or services that have been launched in the past. The method is similar to what you saw in the previous chapter on scoring and screening. The experts make estimates independently. You then convene them a consensus-building discussion or simply average their forecasts. Dealing with forecast errors Sales forecasts rarely are exactly right. If you forecast that 13 million people would see a movie and 14 million did, you would pop open the champagne. But if you forecast 20 million people would see it and only 5 million do, you would weep. The problem, of course, is that you only know the accuracy of your forecast after the fact.

AO-4

Project
7 12 5 14 1 4 2 8 18 10 9 11 6 20 3 16 19 17

Forecast
5600 2500 6700 2100 11100 6700 9800 3879 1146 3300 3800 2500 6300 1100 9000 1400 1100 1200

Actual
1000 500 4000 1500 8000 5000 7500 3000 900 3300 4000 3000 8000 1400 11500 2000 2000 2400 Average St-Dev

AF ratio
0.178571 0.2 0.597015 0.714286 0.720721 0.746269 0.765306 0.773395 0.78534 1 1.052632 1.2 1.269841 1.272727 1.277778 1.428571 1.818182 2 0.988924 0.483351

Product
ZEN-ZIP 2MM FULL ZEN 3/2 ZEN 4/3 WMS ELITE 3/2 WMS HAMMER 3/2 FULL JR HAMMER 3/2 HEATWAVE 4/3 ZEN 2MM S/S FULL

Forecast
470 3190 430 650 6490 1220 430 680

Actual
116 1195 239 364 3673 721 274 453

AF Ratio
0.246809 0.374608 0.555814 0.56 0.565948 0.590984 0.637209 0.666176

Wine
MONTAGNE STEMILION MONTAGNE STEMILION CARTON PANACHE 1 BORDEAUX SUP IROULEGUY CARTON PANACHE 2 BORDEAUX APREMONT

Year
02 02 n.a. 01 01 n.a. 01 02

Color
rouge rouge mixed Rouge Rouge mixed Rouge Blanc

Forecast
4000 6000 1800 1200 3000 900 1800 1200

Actual
468 1236 372 252 726 297 612 567

AF Ratio
0.117 0.206 0.206667 0.21 0.242 0.33 0.34 0.4725

EVO 4/3 WMS EPIC 4/3 JR EPIC 4/3 EVO 3/2 JR ZEN FL 3/2 EPIC 3/2

440 1060 380 380 90 2190

623 1552 571 587 140 3504 Average St-Dev

1.415909 1.464151 1.502632 1.544737 1.555556 1.6 0.9966 0.369666

ROSE DE LOIRE CTES DU RHNE (6) CTES DU RHNE (6) BORDEAUX CHTEAUNEUF DU PAPE CARTON PANACHE 6

03 01 01 02 00 na

Ros Rouge Rouge Rouge Rouge mixed

2300 4000 3000 2500 300 2160

2934 5370 4146 4057 703 5064 Average St-Dev

1.275652 1.3425 1.382 1.6228 2.343333 2.344444 0.858337 0.472292

Exhibit AF-RATIOS: Three cases of comparing forecasted numbers with the actual outcomes in diverse settings including power and automation, surf apparel, and wines.

AO-5

Even though forecasts are faulty, you should take time to examine your past forecasts and contrast them with your actual performance. For each opportunity, compute the ratio between the actual number and the forecasted number, providing an AF ratio for each opportunity. This works for sales and financial forecasts. AF ratios can vary from figures that are significantly less than onea ratio of 0.1 indicates that you over-forecasted by a factor of 10to figures of 3 or highera ratio of 3 indicates that the real number was three times as high as the forecast. Exhibit AF ratios shows examples from very different industries, including power and automation, surf apparel, and French wines. At first sight, the analysis does not say much. Sometimes, the forecasts are too high; sometimes, too low. But with a little more analysis and graphical support similar to what is shown in Exhibit AF-GRAPH, you can obtain a couple of powerful insights2. By looking at the average AF ratio, you can see if a firm is forecasting correctly on average. Most companies consistently forecast more than they actually sell; humans are optimistic animals. As you can see in Exhibit AF-RATIO, the power and automation and surf apparel forecasts are very accurate on average, with an average ratio close to 1. At least for the company that provided the data underlying the exhibit, wines suffer from over-forecasting; on average, sales are only 85 percent of the forecasts. (Maybe the forecasters spent too much time sampling the product.) In the future, if you need to forecast the sales for a new wine, you should adjust the forecast by multiplying it by 0.85 (and keep the forecasters away from the bottles). The AF ratios also help you estimate your risk. A company that consistently produces forecasts with AF ratios between 0.9 and 1.1 faces less risk (a lower forecast variance) than an outfit with AF ratios ranging between 0.5 and 2. (Exhibit AF-RATIO (right) shows how you can use the distribution of past AF ratios to estimate the standard deviation for the new product sales. For every dollar forecasted, you really face a distribution of potential outcomes as is shown in the distribution in Exhibit AF-GRAPH.) Risk follows a common statistical pattern. To see this, compare the three distributions of AF ratios in AF-Graph. The left and the right histograms resemble the normal distribution while the

AO-6

middle one seems closer to a uniform distribution. Once you understand the statistical behavior of your old forecasts, you can build a forecast model that incorporates your tendency to err. Risk can be modeled, in other words, whether the opportunity is a robot, surf shorts, or a Saint Emilion wine.
Financial Returns for Power and Automation Products Product Sales of Surf Apparel and Wet-suits
5 4 3 2 2 1 0 0.0 0 1 .0 0 2 .0 0 1 0 0.2 0 4 2 1 .0 0 1 .6 0 0 0 .0 0 1 .0 0 2 .5 0 version 082007 12 10 8 6

Product Sales of French Wines

Frequency with which AF Ratio was Observed

6 5 4 3

AF Ratio

AF Ratio

AF Ratio

Exhibit AF-GRAPH: Each of the three graphs shows the empirical distribution of the AF ratios shown in Exhibit AF-RATIOS.

Building a financial model Once you have a sales forecast, you can create your overall financial model. For this, you first distribute the total sales over the lifecycle of the product using a lifecycle similar to that of other innovations of this type. Considering your development and launch costs as well as the major fixed and variable costs, you can compute the discounted cash flow (DCF) for this innovation. Exhibit ECONOMICS-EXAMPLE provides an example of this by plotting the cumulative cashflows (inflows and outflows) with the innovation. This graphic can easily be created in Excel and provides a basis for the most common financial evaluations of Horizon 1 opportunities. (In Chapter FI, we discuss several aspects related to how to appropriately choose discount factors a topic that can lead to heated debates when evaluating innovation opportunities.) Given your focus on value creation, your most important measure is the financial returns relative to your investment. Yet with the discounted cash flow analysis, you can also compute other commonly used metrics, including break-even time, time to your first commercial sale, and the return on investment from the innovation.

AO-7

+$
Sales Revenues

Cumulative Inflow or Outflow ($)

Development Costs Ramp-up Costs Marketing and Support Costs Production Costs

Time

-$

version 081507

Exhibit ECONOMICS EXAMPLE: Most opportunities begin with a cash outflow, typically covering the costs of development and launch. Successful opportunities are those in which the cumulative cash inflows (revenues net of production costs) justify the initial investment3.

Now that you have computed the expected sales and the corresponding standard deviation, you can also obtain the standard deviation of your financial measures. The easiest way to do this is to perform a Monte-Carlo simulation in the Excel model, varying the sales number (or other numbers that experience suggests are uncertain such as development time) according to the histogram computed in Exhibit AF RATIOS. This can be simply done by randomly creating an AF ratio between zero and two and then multiplying that AF ratio with the sales forecast. As the opportunity moves through development, all forecasted numbers should be compared with actual performance and updated. Because Horizon 1 opportunities are seldom terminated, the development process should be executed with a focus on efficiency and timeliness.

AO-8

Evaluating Horizon 2 opportunities Earlier in this chapter we compared financial evaluations of innovations with bets at a casino. Before you take your money to the roulette table, you want to make sure that you understand the rules of the games and the odds. Imagine that you place a $100 bet on #14 when playing roulette. You know that (unless you have secret camera) the odds of winning are 1 in 37. If you win, you will receive $3,600. More likely, though, you will lose, and the casino will take your $100. From simple statistics, you can value this bet using the expected payoff, which is computed as: Expected payoff of putting $100 on #14 = 36/37 (-$100) + 1/37 ($3600-$100) = -$2.70 Thus your expected payoff is negative, and the casino makes more money than you do. (Surprise, surprise.) Exhibit CASINO-EXAMPLE explains this bet. The concept of the expected payoff works in cases where you make a lot of bets and what matters is the average payoff. If you bet only once, you might also consider other measures, such as the probability of losing money or the payoff in the worst (best) case scenario. On the left of the exhibit, you can see how the uncertainty of the innovation can play out (also called an event tree). On the right, you can see the various possible outcomes and associated probabilities.
Probability Outcome is 0 (Probability 1/37) Pay-off is 0 36/37 Outcome is 1 (Probability 1/37) Pay-off is 0 Outcome is 2 (Probability 1/37) Pay-off is 0

Outcome is 14 (Probability 1/37) Pay-off is 3600 Outcome is 36 (Probability 1/37) Pay-off is 0 -100 1/37 +3500
version 081507

Exhibit CASINO-EXAMPLE: An event tree describes the odds of winning at the roulette table (left) and a histogram shows the potential payoffs (right).

Expected payoff Enough roulette. Now lets consider a real innovation project. In Exhibit PHARMA-EXAMPLE, you have the event tree for a new chemical compound considered for investment. Based on AO-9

Phase 1 clinical trials, you will learn how toxic the drug is for humans. If people cannot tolerate the drug at all (10 percent probability), all trials stop. Otherwise, the drug moves into Phase 2 trials. A home run would be a drug with medium toxicity but high efficacy, in which case you would make $500 million. If the toxicity is medium, you have a 20 percent chance of obtaining a high efficacy score. For a more toxic drug (strong toxic activity, 40 percent probability), you have a better chance of obtaining a high efficacy (80 percent). But because of the higher toxicity, fewer patients could use the drug, and the payoff would be only $100 million.
Phase II shows medium efficacy (p=0.8) Pay-off: 100 Probability Phase II shows high efficacy Pay-off: 500 0.82

Phase I shows medium toxic activity (p=0.5)

Phase I shows strong toxic activity (p=0.4)

Phase II shows medium efficacy (p=0.2) Pay-off: 0 0.08 Phase II shows high efficacy (p=0.8) Pay-off: 100 -130 -30 Pay-Off 0.1 +370

Phase I shows substance cannot be tolerated by humans (p=0.1)

Investment Phase I: $30M Investment Phase II: $100M

version 081507

Exhibit PHARMA-EXAMPLE: An event tree and histogram for a pharmaceutical compound.

To calculate the expected payoff for this drug, apply the same logic as before. The expected payoff (not yet including the necessary investment) can be calculated as: Expected payoff = 0.5 Payoff if medium toxicity + 0.4 Payoff if strong toxicity + 0.1 Payoff if too toxic = 0.5 (0.8 $100M + 0.2 $500M) + 0.4 (0.2 $0M + 0.8 $100M) + 0.1 $0M = 0.5 $180M + 0.4 $80M = $90M + $32M = $122M

AO-10

If you factor in a development cost of $30 million for the Phase 1 trials and $100 million for the Phase 2 trials, you end up with a net gain (loss) of: $122 million - $130 million = -$8 million. And the drug seems as risky as a bet in roulette. But this analysis misses something critical.

Understanding option value Despite the similarities with gambling, evaluating an innovation requires a more detailed analysis of how the innovation will be created and how it will proceed through selection. In other words, you can make an intermediate decision after obtaining information from Phase 1 but before making the additional (large) investment in Phase 2. To see why a more detailed analysis is needed, consider the three possible outcomes of the Phase 1 trials: Case 1 (Medium toxicity): When you learn about the outcome of Phase 1, you have already spent the associated $30 million. These costs are what economists call sunkthe money is gone and you cant recoup it, so its no longer relevant to your analysis. Knowing a medium level of toxicity, you can expect to earn a payoff of 0.8 $100 million + 0.2 $500 million = $180 million. For this, you would have to spend another $100 million. Would you do this? The answer is an enthusiastic yes! This investment would create an expected $80 million in value. Case 2 (Strong toxicity): When you learn about strong toxicity, you again should look forward, not backward. What matters is that you have an expected payoff of $80 million (0.2 $0 + 0.8 $100M), not that you have spent $30 million. This is especially important when deciding on the next $100 million investment. Investing $100 million to obtain an expected $80 million destroys value. So, as painful as it might be having spent $30 million for nothing, you kill the project (more on the organizational aspects of this later). Case 3 (Too toxic to tolerate): Again, the $30 million is sunk and, because you have no hope of making any money going forward, you should terminate the project after Phase 1. Now, lets return to the initial decision concerning the initiation of Phase 1 trials. Is this project a good investment or not? You have a 50 percent shot at $80 million and a 50 percent (40 percent +10 percent) shot at nothing. For this, you have to spend $30 million for the Phase 1 trials. Suddenly, what looked like a bad investment a moment ago looks much more attractivea $40 AO-11

million payoff for a $30 million investment. Note also that this investment is now less risky, as you have eliminated the scenario in which you suffer a $130 million loss. What can you learn from this example? First, most investments in innovation happen in phases, and the early phases are usually less expensive than the later ones. While the first phases by themselves typically do not lead to financial rewards, they provide something almost as valuable: information. At the beginning of Phase 1, you had a compound about which you knew little. Once you have spent the initial $30 million, you knew a lot more. The additional information had the same effect as the techniques the gamblers used at Ritz: it improved your odds. You could invest in promising compounds (those with medium toxicity) and abandon unpromising ones. Second, when investing into a new innovation, keep in mind that you have the right, even the responsibility, to walk away if the information you gather points to failure. Failing is part of innovation, and failing early makes more sense than failing late. Walking away from an innovation is often referred to as an exit option, as it resembles the financial option of buying (or not buying) a stock at a predefined price.

Estimating the probabilities of success Once you have outlined all the ingredients for the calculations in Exhibit PHARMA-EXAMPLE, valuing the compound (and making the necessary decisions) is a matter of calculus. The calculation turns out to be simple, once you write them down. In contrast, the much harder problem is to get the numbers required for the inputs. So how do you estimate success probabilities? What does it mean to have a 50 percent probability of success? To see the importance of these questions, consider again the payoffs described in Exhibit PHARMAEXAMPLE. By re-doing the above calculations with slightly lower (or higher) estimates for the probability of obtaining a medium toxicity, you observe the following: A 1 percentage point increase in probability (for example, a move the probability from 50 percent to 51 percent) is worth $0.8 million in value, which corresponds to roughly 10 percent of the overall value of the innovation.

AO-12

Moving the probability from 45 percent to 55 percent increases project value by 100 percent.

Thus not only are the odds of innovation success important, their exact magnitude can have a dramatic effect on your decision-making. For this reason, we recommend that you do the computations outlined above with a range of plausible success probabilities, not just one. If the opportunity is profitable only for a small range of probabilities, you probably should not pursue it. In general, you can choose among three estimation methods that yield probabilities of success (POS). Given that even small changes in POS have large impact on the NPV of an opportunity, consider using two or ideally all three of them. Historical outcome data. You can use historical data about the proportion of similar opportunities that succeeded to estimate the POS for opportunities currently under evaluation. Exhibit PHARMA-POS shows a set of probabilities for product approval in the drug development process. Given that the POS data shows significant differences across illnesses, you would want to group the data points of prior opportunities according to illness. The main strength of forecasting POS using historical data is that it provides a rigorous statistical approach. But it also assumes that the future will resemble the past.

AO-13

Indication/Disease
Alzheimer's disease Anxiety disorders Asthma Attention-deficit hyperactivity disorder Bacterial infections Cancer Diabetes mellitus Diarrhea Erectile dysfunction HIV-1 infections Head and neck cancer Irritable bowel syndrome Ischemic heart disorders Leukemia Malignant hypertension Migraine Multiple myeloma Multiple sclerosis Muscle wasting Parkinson's disease Unstable angina pectoris

P1
0.926 0.992 0.809 0.617 0.616 0.862 0.826 0.742 0.52 0.833 0.842 0.904 0.988 0.62 0.552 0.737 0.651 0.877 0.552 0.721 0.818

P2
0.76 0.935 0.709 0.464 0.385 0.777 0.657 0.502 0.4 0.688 0.697 0.748 0.924 0.492 0.4 0.558 0.484 0.706 0.4 0.543 0.658

P3
0.636 0.833 0.226 0.286 0.109 0.176 0.4 0.333 0.2 0.375 0.444 0.571 0.75 0.111 0.197 0.286 0.25 0.5 0.197 0.3 0.385

Total POS
0.448 0.773 0.130 0.082 0.026 0.118 0.217 0.124 0.042 0.215 0.261 0.386 0.685 0.034 0.043 0.118 0.079 0.310 0.043 0.117 0.207

Exhibit PHARMA-POS: Historical odds of passing various clinical milestones for a large sample of compounds submitted to a regulatory body for approval4. P1-P3 refer to Phases 1 to 3 in clinical development.

Aggregating expert opinions. Just as you have seen with scoring and voting methods of selection, various techniques can be used to aggregate expert opinions into a probability score. In a first step, the experts should formulate their opinions independently. In a second step, they discuss their results, typically moving towards a consensus POS forecast. Or a final forecast can be obtained mathematically by, for example, averaging their POS estimates. Alternatively, you might designate an experienced expert to play the role of oracleMerck uses this approach. This person listens to all arguments and then translates them into a single number. Unlike historical data, expert opinions can look forward, factoring in the idiosyncrasies of

AO-14

the opportunity under consideration. But they also can become mired in office or professional politics. Criteria. Many firms have clear criteria that define what POS level any given opportunity deserves. Proctor & Gamble has found that by using explicit criteria of what corresponds to, say, a 50 percent POS, expert discussion can focus more on facts and less on opinions5. Experts, for example, are likely to agree on whether theres a 90 percent probability that a new product can be made on existing machines at high volume. This approach is objective, but defining the criteria consumes a lot of time. And even more time gets spent translating the criteria to numeric probabilities. Again, there is no one right way to forecast POS. You should use multiple approaches and understand that you face an extra level of risk if these approaches arrive at very different estimates. Any POS forecast should always be pressure tested: if a 5 percent reduction in POS turns your return into a loss, the opportunity is probably not worth funding. As we discussed in the previous chapter, accidentally killing a good opportunity might be painful, but is typically far less expensive than further developing a poor one. All three approaches benefit from feeding back the actual outcome into your forecasting for future opportunities. We will elaborate on this point after discussing Horizon 3 opportunities. Exhibit HORIZON2-EVALUATION summarizes the overall process.
From Screening and Scoring Estimate the probabilities that the milestone is completed (POS) based on: Historical data Opinion aggregation Criteria

Define all costs required to get the opportunity launched. Identify milestone(s) at which the opportunity could be killed (and no further cost incurred)

Forecast the payoff collected upon launch (expected value is enough)

Build an integrated model: - Exit option - Probability for each pay-off

To Portfolio Planning / Final Selection


version 081507

Exhibit HORIZON2-EVALUATION: Summary of steps that need to be taken when analyzing a Horizon 2 opportunity.

AO-15

Evaluating Horizon 3 opportunities Horizon 3 opportunities are all about uncertainty. But, in contrast to the tidy risks of Horizon 1 opportunities, the uncertainty for Horizon 3 opportunities is, simply put, one big mess. You dont know many critical facts and you dont even know that you dont know even more things. So how do you evaluate an opportunity like this?

Exhibit SEGWAY: The Segway scooter is a two-wheeled mobility device. Consider the example of the Segway scooter, a personal-transportation device launched in 2001. Given the radical nature of the innovation, the Segway team faced a number of uncertainties, including: Can you produce a self-balancing scooterthe Segway has two wheels on the same axleand sell it for less than $5,000? Will local governments allow Segways on sidewalks? Are you able to produce more 100,000 units a year? Can you create a sufficient product awareness to sell more than 100,000 units a year? How will consumers respond to the Segway?

AO-16

Focus on hypothesis testing With the Segway, youre not dealing with a Horizon 1 or 2 uncertainty. At inception, the Segway represented a new radically technology is search of a market. The innovators did not know who might adopt it. Commuters? People with disabilities and the elderly? Cops? Tour operators? When facing this much uncertainty, you focus on learning. You have only hypotheses about how it might meet peoples needs but no solid evidence. For this reason, Horizon 3 evaluations focus on the creation of knowledge, not potential sales.

Issue
Technical Feasibility

Current Understanding
Technology has been used in a previous product. Detailed engineering not clear yet, but uncertainty primarily relates to component costs. Costs will be between$3,000 and $5,000. Not clear if Segways will be allowed on sidewalks or if they will require drivers license. No prior experience with mass production of a product of this type. The product appears to be similar in its production process to golf carts and other simple electric vehicles. Are you able to create sufficient product awareness? Currently, several newspapers and online chat rooms are hotly debating what product you will launch. You are hoping to fundamentally change the way consumers think about transportation. You have no interest in niche markets such as golf carts. Given the risk that somebody could steal your idea, youve done no market research.

Type of Uncertainty
Horizon 1: Parameter uncertainty

Regulatory Approval Ability to Mass Produce

Horizon 2: Scenario uncertainty Horizon 1: Parameter uncertainty

Marketing Campaign

Horizon 1: Parameter uncertainty

Consumer Interest

Horizon 3: Unknown unknowns

Exhibit HYPOTHESES1: To analyze a Horizon 3 opportunity, the various sources of uncertainty need to be identified and categorized into three horizons defined in Chapter SO (Screening Opportunities).

When evaluating a Horizon 3 opportunity, you start by creating a table like the one shown in Exhibit HYPOTHESES1. This table lists all aspects of this opportunity, such as its technical feasibility, its fit with current regulations, and the customer response. For each one, you then assess your current level of understanding: Level 1 understanding (parameter uncertainty). You exactly understand the problem but have estimates for only some of its parameters (for example, the unit cost of the Segway will between $3,000 and $5,000).

AO-17

Level 2 understanding (scenario uncertainty). You can envision all possible outcomes (and have some probability estimates) for your hypotheses (for example, whether each state or major city will either approve the Segway for sidewalk use).

Level 3 understanding (unknown unknowns): You dont even understand all the relevant variables related to the hypothesis (for example, to support your sales hypothesis, you need to first set a price point, identify market segments, learn the purchase probabilities and appropriately adjust these probabilities)6.

Uncertainty (phrased as hypothesis)


H1: We can produce a self-balancing scooter for a unit cost below $5k H2: Governments will allow people to use our scooters on side-walk H3: We can produce at volumes >100k per year H4: We can create product awareness without having an established brand H5: More than 100k consumers per year will want to purchase a scooter

Full Understanding (no uncertainty) Build a protype ($2M) Lobby for changes in regulation (>$5M) Build a plant (>$10M) Public relations ($1M) Purchase intent test (<$0.5M)
version 081607

Exhibit HYPOTHESES2: Each uncertainty can be resolved at a cost. Your goal is to find the uncertainty for which you can create a maximum of learning per dollar of investment.

Once you have mapped out the uncertainties, you can answer two questions. What are the biggest (and scariest) uncertainties? And where do you get the biggest amount of learning per dollar of investment? As is illustrated by the leftward-pointing arrows in Exhibit HYPOTHESES2, it is relatively expensive to reduce the uncertainty about regulatory approval. True, regulatory approval is crucial, but before spending millions lobbying for Segways to be allowed on sidewalks, you might want to find out if consumers want to ride the machine at all. Maybe, they want to use them only on golf courses or at airports. Creating a table such as Exhibit HYPOTHESES is very helpful in pinpointing where small investments might create new knowledge. In this case, a simple purchase-intent survey such as outlined in the section on Horizon 1 evaluations would have revealed that mainstream consumers were not interested in the AO-18

Segway as means of transportation at a price of $5,000. With this knowledge, the management team probably would have waited to build a mass-production factory. Learning vs. opportunity costs of time Learning and testing hypotheses seem sensible steps. But they suffer from the motherhood and apple-pie syndromethat is, who could disagree with them? Theyre too easy to pursue mindlessly. Thus, before you plunge in, you need to consider what youre sacrificing when you spend time on them. Your willingness to move ahead should be driven by two considerations. The first is the opportunity cost of time, or, put differently, the urgency of a launch. This variable is typically determined by the competitive landscape. The second is the chance of inexpensive uncertainty resolution. This depends on how Exhibit HYPOTHESES looks for the opportunity at hand, specifically if you have a key hypothesis that you can test for a small investment7. Based on these two variables, you can create a matrix as described in Exhibit UNCERTAINTY RESOLUTION. In some situations, you simply must place a big bet: if the opportunity cost of time is high and you cant reduce uncertainty, there are no intermediate steps. You either launch the rocket now or never. Granted, this scenario is rare. In many situations, you can simply wait (and potentially become the second mover) or follow a staged investment approach as discussed above.

AO-19

Rate at which Uncertainty Can be Reduced with Incremental Investment.

Uncertainty is high for Uncertainty can be some time regardless reduced substantially of our investment. with modest investment.

Opportunity Cost of Time (e.g., immediacy of competitive threat)

High Opportunity Cost of Delay

Go for it and translate into Horizon 2 development

Adopt a dynamic, contingent-action approachstages with decision points.

Low Opportunity Cost of Delay

Wait and see possibly as second mover.

Take slow, deliberate, and careful steps to reduce uncertainty.

version 080907

Exhibit UNCERTAINTY RESOLUTION: Depending on the possibility of uncertainty resolution (that is, of learning via small investments) and on the opportunity cost of time, different strategies become optimal.

So how to you make a final decision? Again, recall that Horizon 3 opportunities need to be evaluated based on the learning that they create. The investment in the opportunity needs to be staged with the milestones corresponding to the hypotheses that you want to test. As you saw with Segway, you generally want to start with the biggest uncertainty reduction per dollar invested. Whether you make the initial investment, kill the opportunity or postpone requires a group evaluation similar to what you have seen elsewhere in this chapter. If you cannot resolve any of the uncertainty resolution quickly and you face time pressures (upper left box in Exhibit UNCERTAINTY RESOLUTION), the potential payoff better be big and likely to happen under a wide range of possible scenarios. One venture capital firm that we studied uses the following approach to make the initial funding decision for Horizon 3 opportunities. After hearing the creators pitch, each partner takes a turn

AO-20

and discusses (a) the key hypotheses that she would like to see tested and then (b) gives a recommendation with respect to funding or not funding the opportunity. These discussions start with the most junior partner and are wrapped up by the most senior partner. That way, opinions of less experienced partners arent influenced by those of their more senior colleagues, who should be more comfortable expressing opinions that dont line up with the groups consensus. The final vote is typically pro forma. In most cases, the partners have already reached an agreement.

Which Opportunities to Move Forward? Our discussion so far has been on analyzing the three types of opportunities yet, we still have not addressed the question of which opportunities to now move forward. Recall from our earlier discussion of opportunity classification (see Chapter SO (Screening Opportunities)) that we should not compare one type with another. The amount of resources that should be allocated to each type of opportunity will be discussed at length in the following chapter (see Chapter SP (Innovation Strategy and Opportunity Portfolio)). So, how do we compare within each type? For Horizon 1 and Horizon 2, you select the set of opportunities that creates the biggest value. In other words, you maximize the value of the innovation portfolio. Too often, companies just rank opportunities according to expected returns and then pursue the ones at the top of the list. But this approach ignores interdependencies among the opportunities. The value of the portfolio is not just the sum of the individual values. (See Chapter SP (Innovation Strategy and Opportunity Portfolio) for more details.) For Horizon 3 opportunities, you should compare the required investments and the associated learnings across opportunities and determine which ones would create knowledge most relevant to your company as a whole. (Chapter SP (Innovation Strategy and Opportunity Portfolio) also helps more with this question.)

AO-21

Human and Organizational Aspects of Opportunity Evaluation All of the techniques we have discussed for evaluating opportunities are probabilistic in nature. This reflects the uncertain reality of innovation. But in trying to implement these techniques, you have to keep in mind that probabilistic thinking does not come naturally to many people.

38

Proportion of Days with Rain

0.8

159

82

0.6 203 0.4 257 0.2 589 0 0 282 0.1 575 0.2 0.3 0.4 0.5 0.6 172

147

0.7

0.8

0.9

Forecast Probability of Rain


version 081607

Exhibit POS-FORECASTING1: A comparison of the forecasted probability of rain (x-axis) plotted against with the proportion of days on which rain came (y-axis). Each number along the plotted line corresponds to the number of times a specific forecast was made. For example, a 0.3 probability of rain was forecasted 257 times8.

To get more comfortable with forecasting the probability of success of an innovation, consider the example shown in Exhibit POS-FORECASTS-1. It shows data from another area in which people often speak in terms of probabilities, namely weather forecasting. You might have asked yourself in the past what it really means if you have a 70 percent probability of rain. After all, it will either rain tomorrow or it will not. The graphic will help you ponder that question. On the x-axis, you have different levels of rain probability that the National Weather Service has announced for a particular region and period. Pick the point of 0.7, that is, a 70 percent probability. By moving upwards in the graphic, you hit the point labeled 159, which means that, for this region and period, the National Weather Service had announced a 0.7 rain probability for 159 days. But what you want to know is how many times it actually rained the AO-22

following day. To get to that number, move from this point to the left, towards the y-axis. If the forecast had been perfect, you would expect that rain would have fallen in 70 percent of the cases in which the National Weather Service forecast a 70 percent probability. You see that the actual number was only slightly higher (about 72 percent). Overall, thats an impressive performance.

1.0 0.9 0.8


Proportion with Pneumonia, as Determined by X-ray

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Forecast Probability of Pneumonia
version 081007

Exhibit POS-FORECASTING2: The graphic compares the forecasted probability of pneumonia as expressed by the doctors participating in the study (x-axis) with the proportion of times the patient indeed suffered from pneumonia. For example, only 5 percent of the patients for whom doctors estimated a 25 percent probability of pneumonia suffered from it9.

Now consider a second example, created in the exact same way. Instead of weather forecasters, our subjects are doctors who need to estimate the probability that patients have pneumonia. Exhibit POS-FORECASTS2 shows the proportion of patients who turned out to have pneumonia as a function of the probability forecast of the treating physician. For cases in which the doctor forecast a 50 percent probability of pneumonia, x-rays indicated that only 10 percent of the patients had the disease.

AO-23

Unlike the weather forecasts, in which all data points tidily matched the predictions, you see that their data points stray far from the good forecasting line Of course, just because doctors fail at forecasting does not mean they mistreat their patients. From a forecasting perspective, you might complain about their performance. From a medical perspective, the picture changes. What the doctor really might have been saying when predicting 50 percent was, This patient is at risk for pneumonia, and we should take a chest x-ray. Put differently, the doctor did not distinguish clearly between the forecast (50 percent) and ordering an action (the chest x-ray). Why does this distinction matter? Because the same problem exists in most innovation processes. Both those proposing innovations to be fundedinventors, scientists, entrepreneurs and those who control the requested resourcesmanagers and venture capitalistsoften confuse the forecast (this innovation will or will not work) with the action (should we fund this innovation?). Of course, advocates will argue that the odds are good and will attempt to shift the 45 percent odds to 55 percent odds, while funders will argue the opposite. But to make a prudent financial valuation, you must draw a clear line between your risk assessment and the associated payoffs. To improve your forecasting capability, the following three steps are useful: (a) ensure that the evaluation of opportunities is done by independent reviewers (as opposed to people who have a stake in the success of the opportunities); (b) increase the independence of the review and improve its predictive power by involving a larger population inside or outside your company; (c) give regular feedback to all units involved in forecasting. The need for an independent review board The adage that You dont let the turkeys vote on Thanksgiving! has a direct relevance to the evaluation of opportunities. You must separate those who select opportunities from those who create and develop them, and this is where independent review boards come in. To understand their usefulness, consider Exhibit INLICENSE-POS. The data summarizes a McKinsey study on the innovation odds of compounds that drug companies created internally versus those that they licensed from outsiders.

AO-24

Clinical Success Probabilities Among the Leading Pharmaceutical Companies 80 70 54 48 39 27 14


Internally developed compounds Licensed compounds

68

Phase I

Phase II Clinical Trials

Phase III Cumulative Success


version 081407

Exhibit INLICENSE-POS: Success probabilities for internally developed chemical compounds compared with licensed compounds. Internally developed compounds are more likely to pass the Phase 1 review, yet their overall success rate is only half of that of licensed compounds10.

Looking at the overall success ratesthat is, the probability of the innovation surviving all three phasesyou notice that licensed compounds are twice as likely to succeed. Why is this? When licensing a compound, an organization does exactly what we described above. It has a set of criteria and applies them to the licensing candidates. In this setting, the people involved in the valuation have no personal stake in the innovation and can be dispassionate. But when you are evaluating an internally developed innovation, people involved with its creation probably have some influence on the review. This often leads to a company being reluctant to terminate a project early on. Just as parents would not harm their own children, many innovators do not want to kill their projects. For this reason, as you can see in INLICENSE-POS, internally developed projects are more likely pass through the first phase but also more likely to fail later. This tendency can cost you real money. As you have seen in the context of valuing individual projects, the value of an innovation depends crucially on your ability to walk away from it. A review board can be truly independent in making these tough decisions. An effective board is not just independent. Its also diverse, including the perspectives of disinterested innovators as well as from such corporate divisions as finance, marketing, and operations. The link between innovation and finance is often weak in big corporations. As a

AO-25

consequence, finance people will attempt to develop models similar to the one described in Exhibit PHARMA-EXAMPLE but fail to incorporate insights from staffers most familiar with the innovation. Using large populations to support the evaluation process In addition to this traditional approach towards forecasting success probabilities, one additional tool deserves discussion: prediction markets. Prediction markets aggregate peoples opinions using a basic principle of economics. If I believe that an event will occur with a 70 percent probability, I should be willing to pay up to 70 cents for a lottery ticket that pays $1 if the innovation succeeds and nothing otherwise. If you feel more confident about the success of the innovation, you should be willing to pay more. Prediction markets let you (and many others) trade your bets in ways similar to how stocks are traded at Wall Street11. Just as stock prices mirror the markets aggregate opinion on a companys future cash flow, a prediction market reveals the participants aggregate opinion on the odds of an innovations success. Several successful implementations of prediction markets for innovation opportunities have been reported (links available at www.MasteringInnovation.com). You can trade prediction shares in nearly anything, ranging from the next pope to the outcomes of elections. But there is little empirical evidence that these markets really work when it comes to forecasting innovation success. Prediction markets certainly represent a step forward for most firms, relative to the typical status quo of having no system of forecasting. Whether they can replace three-step probability forecasting discussed above will depend on the specific application. Chapter OI (Open Innovation) will elaborate more on how one can benefit from the wisdom of the crowds.

Feedback and calibration One of the reasons that weather men do so well in forecasting the probability of rain (see Exhibit POS FORECAST1) is that they have many opportunities to practice. But there is more to this than practice alone. Practice only leads to better forecasting if there exists some feed-back.

AO-26

Doing many weather forecasts alone will not help, doing the forecasts and then contrasting them with the actual weather, however, can be very powerful. Why cant we do the same exercise that every new weather-person needs to go through for those in charge of forecasting the odds that an innovation turns out successful? The idea of calibrating the innovation forecasting process is compelling. However, it requires that the organization overcomes two obstacles: Forecasts will never be perfect, and this reality has to be accepted by forecasters and the managers who evaluate them. Nobody likes to be reminded of yesterdays errors, so most innovation forecasters prefer to not go on record when making a prediction. Many companies fail to keep track of their old forecasts. Old forecasts get overwritten by new forecasts if they are stored electronically, or they are filed way, never to be retrieved. Overcome these obstacles, and your firm can create a graph like the one shown in Exhibit POSFORECASTING3, which tracks the forecasting performance of Ely Lilly. Notice that the probability forecasts by the companys review board are remarkably close to the actual innovation success rates. With enough discipline and the right inputs, forecasting innovation odds is possible.

AO-27

IRB Performance, All Raw Data 21 76 73

0.90 0.80

Most of the high sample size observations lie within a 0.10 band about the target line

Actual Success Rate

0.70 0.60 0.50 9 0.40 0.30 0.20 0.10 6 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90
version 080907

22

67

29

15 15 8

12

IRB Probability Assessment

Exhibit POS-FORECASTING3: The graph compares the forecasted probability of success as expressed by the independent review board (x-axis) with the proportion of actual innovation successes at Eli Lilly. Each number along the plotted line corresponds to the number of times a specific forecast was made. For example, a 0.3 probability of success was forecasted 8 times12.

Chapter Summary The objective of opportunity evaluation is to assign financial values to opportunities and to identify the types and sources of risk. Where possible, you should aim to create rocket-science evaluation models. Horizon 1 opportunities can be evaluated based on discounted cash flows. The biggest threats to a good evaluation are overly optimistic forecasts of sales and cost overruns. Their development process should be highly structured and data driven. Horizon 2 opportunities require an options-based evaluation, as they typically allow the organization to abandon the opportunity after some bad news has surfaced. The biggest threats to a good evaluation are overly optimistic POS forecasts and the unwillingness to kill an opportunity.

AO-28

Horizon 3 opportunities should not be valued financially. Their focus should be learning, and the company should try to maximize the knowledge per dollar of investment. If possible, investments should be staged. Staging investments enables you to cash in on the full upside of a good opportunity, yet limits the downside of a bad one. Evaluation of opportunities should be done by an independent review board. Explicit data-based feedback to a board helps it overcome decision biases. Diagnostics What measures do you use to evaluate the financial attractiveness of opportunities? Do you distinguish between different horizons? Do you use measures for the probability of success? How are these probabilities computed? Does your company keep data on old forecasts with respect to financial returns, sales volume, and probability of success? When analyzing the financials of a new opportunity, do you factor in how previous forecasts fared? Are you able to spot weak opportunities early on or do your opportunities tend to linger, regardless of their prospects, once they have started development? Do you evaluate Horizon 3 opportunities with a focus on financial returns or learning? Are your review boards independent or might members have conflicts of interests when selecting opportunities?

Chapter Notes
See the chapter of concept testing in Ulrich and Eppinger. K.T. Ulrich and S. Eppinger, Product Design and Development, 2nd ed. (New York: McGraw-Hill, 2000).
1

AO-29

See Cachon and Terwiesch for a detailed discussion of how to interpret AF ratios and how to use this information to make decisions under uncertainty. G. Cachon and C. Terwiesch, Matching Supply with Demand: An Introduction to Operations Management (New York: McGraw-Hill, 2006).

See Ulrich and Eppinger concerning details on how to build a discounted cash flow model for Horizon 1 opportunities. K.T. Ulrich and S. Eppinger, Product Design and Development, 2nd ed. (New York: McGraw-Hill, 2000).

Source: ADIS International, see Girotra et al for further details. Karan Girotra, Karl Ulrich, and Christian Terwiesch, Risk Management in New Product Portfolios: A Study of Late Stage Drug Failures, forthcoming in Management Science.
5

Panel discussion of the POMS college on Product Innovation and Technology Management, Boston 2006.

See Loch et al. for more details on managing unknown unknowns as well as on how to create a table similar to Figure HYPOTHESES 1. Christian Terwiesch and Christoph H. Loch, Measuring the Effectiveness of Overlapping Development Activities, Management Science Vol. 45, Number 4 (1999): 455-465. Terwiesch and Loch discuss the concept of uncertainty resolution. To measure uncertainty resolution, we have to map out the amount of residual uncertainty over time. A steep initial decline in the resulting graph corresponds to a fast uncertainty resolution. Christian Terwiesch and Christoph H. Loch, Measuring the Effectiveness of Overlapping Development Activities, Management Science Vol. 45, Number 4 (1999): 455-465. Example and data is taken from Murphy and Winkler 1977. A.H. Murphy and R.L. Winklder, Can weather forecasters formulate reliable probability forecasts of precipitation and temperature?, National Weather Digest 2 (1977): 2-9. Example and data is taken from Christensen-Szalanski and Busyhead 1981. Christensen-Szalanski and Bushyhead, Human Perception and Performance, Journal of Experimental Psychology Vol. 4 (August 7, 1981): 928-35.
10 9 8 7

Example and data is taken from Booth et al 2004. B.L. Booth, D.J. Lennon, and E. J. McCafferty, Improving the pharma research pipeline, McKinsey Quarterly (Web Exclusive 2004). Wolfers and Zitzowitz discuss prediction markets including applications to new product sales forecasting. J. Wolfers and E. Zitzewitz, Prediction Markets, Journal of Economic Perspectives Vol. 18 Issue 2 (Spring 2004).

11

12

John S. Andersen, Assessing Technical Feasibility of R&D Projects in Portfolio Management, presented at INFORMS, Seattle, 1998, TA01.3.

AO-30

Você também pode gostar