Paul Saffo: The Secret to Effective Forecasting

Good evening, I am Steward Brand from The Long Now foundation. This is the first of in a sense, two talks. All you have to do is thinking about the future. Paul in a way is lucky to going first, because the next one is Nassim Nicholas Taleb doing the Black Swan. And Taleb is about – he will be here Monday, February 4th. It is about the huge events that no matter how hard to try, you can’t predict and that change every thing. And the way he comes to engaging that is by looking – he is a – he is a financial analyst, mathematical type, as well as Lebanese as well as great story teller. And what he is doing is looking at the past in a certain way and trying to identify all the illusions that we used the past to think about the future with. Saffo; whose background is in studying the history of technology also looks at the past, looking for all kinds of illusions which we delude ourselves about the future, at a more micro scale than Taleb does. But there are interesting consistencies and apparent contradictions but they are probably just paradoxical, between the two. So let’s start with Paul Saffo. Thank you Stewart and it is wonderful to be here. And it is interesting to be on the stage ahead of Taleb, because it’s a funny thing. These things go in cycles. But it is fashionable at the moment to make fun of futurists and forecasts and make great sport about how people’s predictions and forecasts are terrible. Which I have no objection to, I have done it myself at the better part of 25 years as a professional in the field and I have even taken pain, taken great pains not to call myself a futurist, which I will probably get around to explain you in a little bit but. But let me start with a forecast and it – it was made a couple of years ago, and just analyze it a little bit. Hunt of Bin Laden, experts agree, Al Qaeda leader is dead or alive. Now on the surface this would look like a terrible forecast. In fact I am not even sure if it’s a real forecast. This was courtesy of a jokester friend of mine, Yocy Vardi(ph) who sent it my way and I have never gotten CNN to admit if they actually did this. But let’s assume it was a real forecast. You know its evacuative, the complaint of one of our former president once said, please give me a one handed economist, you know none of this on the one hand on the other hand thing. I would actually argue that this is not just a good forecast; this is a great forecast because it accurately and completely captured the uncertainty of that moment. The answer is he is dead or alive beats me, that’s what it does. And the biggest mistake a forecaster can make is to pretend to be more certain than in fact this fact suggest. And it goes even more deeply than that, I think especially important today, this moment in time, is very uncertain moment in time where uncertainty just seems to be everywhere and the indicators head off in opposite directions. Take the financial markets. A few weeks ago watching golden Google both go upwards, you know, and the question was which one would cross 850 first; 850 being the 30 year record high for gold – well the spot on gold made it today that hit $900. But you know it’s a funny thing, once up on a time in – in more simple and innocent times, if stock prices were high like Google’s price that meant times were good. And if gold prices were high like gold is at the moment that meant times are bad. Both are high; go figure. And then there is oil. Headed over a $100 now, it’s under what happens next. The mortgage crisis, and any body who has been in Europe lately knows what the Euro is doing and why you think twice before ordering a latte in Paris These indicators are headed in opposite directions. And in my business as a forecaster when change clusters at the extremes it’s a certain bet that much more fundamental change lies ahead. And I will talk a little bit about what I think lies ahead tonight, but mostly what I want to do is share with you some tricks of the trade, some rules of thumb that I have learned over the last 25 years as a forecaster and show you a little bit of how I approached dealing with – looking at this uncertainty. And uncertainty really is the key element. The way I think about it is you can take an event at a given moment in time and extending outwards from that event is a cone of uncertainty. And the act of forecasting amounts to the act of mapping that cone of uncertainty. And it is an act of embracing uncertainty, and above all by the way, it is an act that everybody should do. In these uncertain times forecasting is more important than ever and the biggest mistake you can make is to lead it – leave it to the experts. Everybody should forecast for themselves, and if you do that regardless of the quality of your forecast you will a more sophisticated consumer of the forecasts from the people who do it on a full time basis The trick with the cone of uncertainty and – by the way the reason why it is a cone is pure common sense. In fact everything I am going to talk about tonight is common sense and because its common sense very little of it is original. Think about the weather in San Francisco. Now our weather is not the best example but you know the cone 15 minutes out is fairly narrow, it is a safe between, unless you live in Marin what the weather will be, but go out a day the cone gets broader. And then after a couple of days – about three days, well the curtain of good information falls and it’s like any one’s guess. So the essence of this – if you are thinking about that spread of uncertainty that increases overtime, and it begins with an event and it forces you to think about just what the uncertainty really is. By the way just a footnote, I forecast, I don’t predict. You know at a cocktail party some one will come up to me and say, so – usually their question is how – how are your predictions doing? Actually the first question is what stocks should I buy? That’s the more boring question, but the second one that I – start, it is great conversation killer because I say, I actually don’t predict, I forecast, and that's about mapping the cone of uncertainty and they find someone else to talk to. It works – works every time But uncertainty is not just an artifact of imperfect foresight. Uncertainty is intrinsic in the process and in my opinion it’s very good news. Think about it, a world of no uncertainty, a world in which you really – or somebody could accurately predict is a world that I don’t think any of us would really want to live in. That's a world of a fundamentalist religious believer; god has a plan, it’s all written out, but why bother because you can’t change it. That’s the world of the appointment in Samara, if you remember that old tale, of the servant who goes to this nook in Samara and bumps into death in the market place, comes rushing home agitated and says to his master, “Master, master, lend me your horse, I must flee to Baghdad.†The master says, “Why?†He said, “Well death is menacing me in the market place.†So the servant rides off. The master goes into market place, finds death, he is really – he is seen in black hook – you know, and says “Dude what you are doing, messing with my servant?†He said “I wasn’t threatening your servant; I was just startled to see him, because we have an appointment in Samara tonight.†So – that’s not the world we live in. This is a world of uncertainty and that’s good news because uncertainty is opportunity. Any body who invests in the stock market, you know the worst market to invest in is a market where there is no changes. Just painful if you call it wrong, but there – uncertainty and opportunity go together and in a perverse sort of way, that means this is the best of all possible times. Now admittedly – you know for some people, especially if you invested in sub prime mortgages, saying that today is the best of all possible times is evocative of the immortal words of John Jacob Astor who was sitting in the bar of the S. S. Titanic said, “You know I asked for ice, but this is ridiculous." Nonetheless the art of forecasting is understanding uncertainty and also balancing between a stance where – if you look at things too narrowly, if you draw that cone too narrowly, you are going to miss things that happen. If you broad – draw too broadly you are going to spend your whole time navel gazing around events that may never come to pass. And the art – what makes forecasting hard isn’t predicting the outcome. What makes forecasting art is predicting the edges of the cone, and nobody – I have been saying this for 25 years and no body seems to get it. May be I just don’t know how to talk. Here is a way to think about it. Wild cards are an especially effective method to do this. And this is a first of a couple of rules I – I did an article in Harvard Business Review and I pulled a little bit out of this – but out of respect for you all I have thrown out the bunch of stuff – I didn’t do – put into HBR. Think about wild cards, you know Peter Schwartz, a brilliant futurist is fond of remarking that the difference between a good forecast and reality is that a good forecast has top be believable and internally consistent. And of course realty labors under no such constraints. And – and one way to think about that, the unlikely, the outlandish and the like, is to turn them in to wild cards. Wild cards are a permissible way to think crazy ideas without getting ridiculed. And there are all sorts of ways to think about wild cards, but the bottom line is that wild cards test the edge of the cone; they define the ragged edge of plausibility of any good forecast. So think about wild cards and – and think about people who do it well. John Petersen at the Arlington Institute on the East Coast is a brilliant thinker about wild cards. But of course the daily newspaper is a great source of wild cards and – and just being a smarty pants is not a bad idea. I remember when we started the recall campaign for governor – long before Schwarzenegger appeared on scene, and I was talking to a journalist friend on the East Coast and I called him up and I said, you know we are having a recall campaign again. And we have had one of these ever since Hiram Johnson ran for Governor, but I just have this feeling this one is going to work. And you know with our damn luck we are going to get another damn movie actor as governor. He thinks I am a genius. I was just being a smart ass. And of course the amazing part of that wild card is not only did we got a governor – an ex-movie actor as a governor, he is a great governor, go figure. So – you know the difference – at least he is a pretty good governor. So the bottom line is think about wild cards and look around in all sorts of places. I find they – they pop out in – in the odd sorts of spots but – but also bad forecasts, really bad forecasts can be really great wildcards. This is a magazine – it is one my favorite magazine, in fact it was my most favorite magazine until the advent of Wired, which was kind of an active unrequited love because it lasted exactly one issue. As I said there hasn’t been a magazine like this one until now and there wasn’t one after. It was around 1970, they didn’t put a date on it. So I can’t – but you can judge that's the age by the picture, that's a very young Al Toffler before he wrote Future Shock. So – so bad magazines with lurid forecasts are a great place to look for good wild cards. And in this they had a special – because it was like 1969, 1970; they had a newspaper from 1984 and there was a wanted section in it. For 1984 genetic engineer, need MS in bio chemistry and lab experience will be – and composing genetic, IEDNA, because people didn’t know about codes for babies – Skinner Community Hospital, Watson Avenue. This was – you know this was a good seven years before the first issue of bio technology appeared on screen. The other ones are pretty good too. Attractive girls with good personalities and basic office skills needed for film about sexes in the 70s. Its wonderful stuff and I must admit; I look at this old magazine more often than you might imagine. Wild cards sensitize us to surprise. They help us kind of get in underneath the current of the zeitgeist and get a sense of what's going on. And by the way there is one other source I should mention, not just for wild cards but for forecast, and that's science Science fiction people will have real arguments about whether it’s a good place for forecasting. It’s actually one of the few things that will actually give you predictive power, because it turns out science – science fiction ends up being this idea bomb, it’s a mean bomb that goes in to the head of teenagers and they carry that idea through school ignoring their professors and ignoring their bosses until they reach middle age and they get the opportunity to finally carry out their dreams which they got from reading science fiction. And that’s how we got cyber space. Will Gibson you know famously wrote about it in a novel and the people who read it – when they got into the business world, that’s what they may. And the atomic scientists who worked on the Manhattan Project were all influenced by H G Wells and many of the astronauts in the Mercury Program grew up on Space Opera. So science fiction is actually a really interesting special case of how to look ahead, because you can see what people’s dreams are and they will eventually get a chance to do it. But go back to this notion of surprise. Things surprise us for a very simple reason and that is change is never linear. At least interesting change is never linear. Even in here, in Silicon Valley, what surprises me is that the future constantly arrives late and in unexpected ways. In fact the secret in Silicon Valley is most ideas take 20 years to become an overnight success. And they do so because change looks like this, it’s a S curve, it’s the curve of Moore’s Law. We all know the doubling period of chips and it has lead to all the marvels we have, all sorts of consumer devices, television followed an S curve pattern from invention in the 1930s to take off in the 1950s. The internet itself followed an S curve. I mean the internet was almost exactly 20 years old in 1987, the year ALGOR invented it, when it started to take off. So S curves are everywhere. They are all around us. And as a forecaster in general a safe posture is things are going to arrive late. And – the other half of that statement – I will spoil aside a few points ahead, if everybody thinks something is around the corner, take the posture of no, no, no – its further out. And then if they all say it will never arrive, you can say it’s about to come. I guarantee that's a safe default position. But – but let me give you an example, because inflection points are tip toeing past us all the time. But the problem is this is how we generally look at the future. We – even in Silicon Valley people draw the future as a straight line. And mean while things go – it’s because we don’t like change and – and a few other details about why that flat spot is so flat. But let me give you an example of an inflection point tip toeing past us right now. This is a picture I shot in May 2004 the first DARPA Grant Challenge. This is the robotic race across the desert, a 150 miles, 21 teams qualified and – and I was there at the start and this is sort of what happened at the start. It was like Monty Python. 21 teams – all but four vehicles died in the shoots or at the gate and – and this one was my favorite, this little R2D2 kind of guy comes out the jersey barriers, it gets to the entrance, it pauses, it look left, it looks right, it thinks about it a bit and it hangs a hard left and dies in the coyote bush 20 feet. The robot that got farthest was this one, Sandstorm from Carnegie Mellon; here it is, yeah you have lost dudes, get over it. But it’s a nice vehicle and – and you won’t whistle when you see the next picture. So it’s – and by the way for those you aren’t familiar with the race, so this is a Humvee with a light armed unit on the – on the roof and behind it is a truck with three dudes in it. And one dude is driving and one dude has got his thumb on a kill switch. At the moment the robot misbehaves, he like – you are dead. There was one – a huge military vehicle that was the scariest thing I have ever seen that they really had to shutdown, but I don’t want to punch on. So the next picture is Sandstorm, I kept trying to think what's the punch line to, why did the robot cross the road, but I can’t recall. This is a picture of a crossing the road, moments before it drove through a fence. If people - if those people had known, they would have been stepping about 50 feet farther back and that little INSAT picture is Sandstorm stuck on the side of the road where it died. 7.4 miles into the race, now I was a part there as part of a medical support team and we were halfway on the race out in the desert feeling pretty darned foolish because there was nothing near us. But now jump and – and only four – four robots got out of the gate. Jump ahead 18 months later and here is the finish of the second race. The grand challenge number two and that is Stanly from Stanford and he is a part time consulting professor at Stanford, I am glad to say the Stanford team one, but I will in the interest of the Carnegie Mellon folks in back, admit it’s not so much a Stanford one but Carnegie Mellon lost. And only by a small amount, it’s off topic but let me explain it for software developers in the room. Sandstorm Carnegie Mellon team leader is Red Whittaker, Red is an ex-marine, let me correct myself, Red is a marine, there is no such thing as an ex-marine. He believes in hot hardware things, good hardware is more important than good software, Sebastian Thrun who led the Stanford team and by the way we had Stanford stole Sebastian from the Carnegie Mellon team and moved him out to Stanford where he is the head of the AI lab, believes in really good software and will make up for bad software, Sandstorm was kicking the Stanford vehicle all the way until a bolt came loose in that, laid our unit on the top of it, started wobbling and it got really bad as stigmatism and that was crawling along and Stanley went roaring past. there - the difference on this race so is there were 23 finalists entering the race, 22 the robots got farther than Sandstorm got in the first race and five of them finished. Does that look like an S curve to you? That’s an inflection point, that kind of 18 month jump. I will go then back into that, but here is the lesson of all of this. Our tendency is to over estimate short term change because of our inflated hopes and expectations and when cold reality fails to confirm to our inflated expectations, we do the opposite, we underestimate the long term implications. This is why - this is the only slide that we would argue against being a forecaster. Ordinary people get to be wrong ones, the internet arrives then it go - where that come from. Forecasters and visionaries and software developers and the like get to be wrong twice because they stand at the start and they say oh I see the future, it’s just around the corner and then they stand around, of course in silicon valley I should add, we don’t draw this as an S curve in the valley it is a J curve. Things you know just keep going upwards forever, Gary Gruner is in the audience, he will appreciate that this is why venture capitalists sleep like babies, they sleep for two hours, wake up and cry - so now one way and – and – and I should add the – the most elegant statement of this I ever heard when I was a young child living here in California kind of an - a royalist part of the state and there was a rancher who said to me son, “never mistake a clear view for a short distance†and that's the best forecast seeing advice anybody has ever given me and I pass it on in this period of sharing, never mistake a clear view for a short distance, things will always arrive late and in completely unexpected ways. Now one way to avoid being blind sided by this non linear pattern, the S curve is to look for indicators. Bill Gibson, the novelist put it very nicely at a conference here that Kevin Kelly organized many years ago, he said you know the future is already arrived, it’s just not evenly distributed yet and he is absolutely right and if you think about that you know the way – the way good forecasters do it is we look for things happening in the present and we extrapolate that and you get tuned to those weird little things that don’t fit, it seem out of place and kind of stick in your mind, of course some of them are hugely obvious, the keeling curve I mean we all deserve a collective dope slap for missing this one, I mean the numbers were there. The first report I saw that – that talk about global climate change was a Nova report done as an expert study in - about 1977 we still ignored it, but there are other things that are really good indicators and doctors have a great term for this. Generally I don’t use Latin in my speeches ever since I – I - a friend – a historian pointed out to be a – after Churchill died, they were looking through his papers and in the margins of one of his speeches, it was written, he had written to himself, speech [0:25:40.5] ____ use Latin and shout. But a fact come to thinking that if I am not using Latin, I am using Greek, the word Prodrome means roughly running ahead, it’s an indicator, now if a doctor tells you that he sees a Prodrome, that's not good news. You don’t want your doctor to talk to you about Prodromes, but think about prodromes, of indicators - things running ahead of the actual event. One of my favorites because I was lucky enough to notice it at the time was on the - in January 1985 on the Usenet which then was in an extraordinarily obscured thing, this – this guy named Spencer Bolts said I just found an interesting software problem, of course we are going to solve it well the interesting software problem is what we went on to call y2k. We knew about it then, it was written up in computer world that spring, you know it was exploring around, but nobody took it seriously until about you know October 1999. So these things are all around if you just look for them. Good indicators, the best indicators are things that don’t seem to fit, they don’t make sense or they are just really weird and – and if you look at it that way, you see them every where I was flipping through the examiner two days ago and there was a little thing buried on page 14, San Mateo police department puzzled by homemade bombs and I thought oh I also hope that it is not a new indicator, but I can’t get it out of my mind – but let me give you a happier example. This is the Roomba, how many people here have a Roomba? Okay what's the name of your Roomba sir? He does have – who have – okay you know the next time I am going to do a plant in the audience, forget you let me tell you what I founded in 2003 the Roomba came out and something very strange started happening. I have a lot of geek friends you know and they got a 128K max, ok and I did too, and – and all of a sudden I noticed some of my engineer geek friends were totally stoked about their new robotic vacuum cleaner and I thought well this is odd, these are people who I don’t even recall, owning a vacuum cleaner and they are excited about it and I started checking more and you talked to the company and they said yeah, you know we are seeing this really weird thing too two thirds of Roomba, we discovered giving their Roomba’s names and I thought I never remembered anybody giving a vacuum cleaner a name and then more amazingly one third of Roomba owners confessed to having taken their rumba vacuum cleaners on vacation with them or over to a friend’s house, oh folks that's a screaming indicator that someone is going on in the robotic space, you know build on that in a second, but then there are just those indicators that are really weird coincidences. The third DARPA grand challenge on third November 2007 and the Carnegie Mellon team It was started at 8AM on - it was November third and it was about 250 or I guess about a hundred mile won it, they finally caught up to Stanford and they won the race but - and this was the urban grand challenge. It was started at 8AM on - it was November third and it was about 250 or I guess about a hundred mile race and they had to drive around town and they had to know the vehicle codes and all this stuff and – it was a good race, you know the – the robots actually performed. But here is the indicator at almost the exact same moment that this race began this is what happened on high way 99, 250 miles to the north. A whole bunch of people in cars over a 100 cars in the end I believe it was 108 cars and 18 big rigs smashed into each other like salmon going up stream in the fog, the fire men working the front of the accident said they could hear the cars pounding into the back at the end. At the very same time that the robots were taking off, people were doing this, okay. What's the indicator, people shouldn’t drive. People shouldn’t drive, it’s crazy that people drive and if you ask Sebastian Thrun he aggress and in fact his forecast is by the year 2030 half of all vehicle miles in this country will be driven by autonomous robots, which sounds really unlikely and tell you look at this and you think at the very moment we demonstrate it 250 miles to the south, that we already have robots that would understand the California vehicle code better than anybody in this room So these indicators are absolutely everywhere. But one another detail, if you miss an indicator don’t worry, you got time. If you miss that Usenet announcement in 1985, you had until 1991, 92’, 93’ to get on to the bandwagon and write your book about the Y2K price because – because nobody else was watching it. Chang is slow, it’s uneven. It’s slow at the start and then it becomes very fast. So often times you can go back and mine history for the indicators so long as you do this. Look back; look back twice as far as you are looking forward. Mark Twain is alleged to observe, you know history doesn’t repeat itself but sometimes it rhymes. And he is absolutely right, even though he didn’t really say it. There are also people who say, god darn it, you forecasters – you know, you use a rear view mirror to look into the future. Well I am here to tell you, a rear view mirror is a damn good forecasting tool if you use it in the right way because besides it’s really the only tool we have. So if you really do look back at history in the right way it is enormously informative. There are also you know the latest fashionable term in forecasting is, “forecasters don’t get any better but they keep changing the term and hoping people remember – you know forget their bad history. So it was a brief horrible period where they were called futurologists, we are glad that died. And then there are futurists doesn’t quite die and then there are – you know people like me who use with word forecaster. The latest term is foresight, as you know some how it makes it more respectable. So you have all these little foresight institute things coming up and back. Well, if you want to do good foresight you better do good back sight. Back sight is the secret to foresight. Look back – and what you can see is that random little indicators will suddenly line up into a very powerful beacon looking at the future. You know I have been hinting about a robot trend. Let me give you some context. It turns out about every decade or so an enabling technology arrives that sets the entrepreneurial landscape. And the technology that shaped the 80’s that arrived in the late 70’s was the microprocessor. But people didn’t think about buying microprocessors, they bought personal computers. It was a revolution because now we could have computers on our desks. So the 1980’s was a processing decade enabled by the arrival of cheap microprocessors and the posture child was the personal computer. Then the 90’s however were shaped by a fundamentally different technology, by the way in my defense this was a forecast I actually made back then and so I – in the documentations on my web page, the 90’s were shaped by the communications laser, which built on top of the processing, power that already existed. And cheap lasers gave us optical storage, CD-Rom and most importantly they gave us fiber optic band width. And that was the access decade. Our devices were no longer defined by what they processed for us. They were defined by what they connected us to. And by the way these are little tiny S curve events that – that happened and got established and brought some real surprises in. This decade is being shaped by fundamentally different technologies, not the microprocessors, its not the lasers, it’s the arrival – oh and by the way indicators of the laser decade, I shot this on a street in San Francisco in 1992, by the way if you are looking for indicators in traffic watch both – look both ways first. I almost got hit when I took this. Oh wow cool you know, I got a good picture of it. But these things are what cause the surprise. And you know I mentioned earlier the future tends to arrive in unexpected ways. You know think about how the internet came in as a total surprise to so many people. In fact here in 1987 there were people who talked about – about the fiber optics and giving us optical storage on disk and books were dead. So now just for a moment, place yourself in 1987 and the wise people – Bill Gates was saying, we have a CD-Rom revolution coming and – and we are going to have hypertext and entire books on – on disk and get over bookstores there, you know they are going to sell you electronic books, paper books are dead. There were a few maniacs running around like Ted Nelson and – and some of Doug Engelbart saying, no, no we need a global – you got a cyber document system where everything is connected to everything and you don’t have to go to stores and they said; that’s very nice, may be in 50 years. The training wheels for that are CD-Rom and hypertext on disk. But imagine saying to someone in 1987; little kids – long before the year 2000, little kids will come home from school, turn on their computers, log on to a server in Tokyo, download the latest images of their favorite Japanese anime cartoon characters, over what – that thing you thought was 50 years away, the global hyper document system arrived in 1994, it was the world wide web, oh and by the way triggered the largest explosion in consumer spending in 40 years and what was the hottest thing being sold? Books – they went up like as a dangerous crazy. This decade however is being shaped by a very unnoticed technology and that’s cheap sensors. We are in a middle of a vast sensor revolution of things that are – you know like RFID, this is a wonderful slide from a company that went out of business. You know people talk about RFID as a substitute for barcode. It may be but for god sake that is the least important implication here. We think about this way. In the 80’s we created our computers. In the 90’s we networked them together. And now in this decade we are giving them eyes, ears and sensory organs and what’s the only thing missing from that, put wheels on the darn things and they are robots. I am here to tell you that I will – and I am not – well I am not phrase to different way, because I know what Stewart will do to me. The nest big thing is robots. The next big things that come out of no where and everybody is going to, “oh my god where did that come from? And I couldn’t live without that, I didn’t know it existed,†is robots. I don’t know how far off it is, but it is not too far into the future. And when you look at this context from history its – its crushingly, inexorably obvious. By the way robots may be the next big thing; they are not the only big thing. Sensors pop up in other places. You are putting sensors in your running shoes, I am sure people here have this. This is the new Nike shoe, it has got a wireless link, your shoes talk to your – your iPod and if you slow down the shoes say, “hey the dude is slowing down, switch to the inspirational music.†It plays Rocky and – and this is an indicator too of another shift by the way. I mean this is not an evening where I really wanted to talk about forecast but I can’t resist. Today the web is synonymous with people accessing information – you know just get on there and you surf around. Once upon a time the telecommunications network in this country was about people talking to other people and today that’s such a small fraction of what the telecommunications network does. They can give you phone service for free over the internet. The same thing is about to happen to the web. The future in the web is not people accessing information. It’s machines accessing the information, its machines generating the information and in the same way that a fraction of one percent of telecommunications traffic is attributed to the voice today, in a decade or so you will see this same thing happen to the web, where most of the web viewing is done by machines and most of the data is being generated by machines. But pushing onwards; another good way to see indicators is to cherish failure, preferably other people’s failure. Remember that flat spot on the S curve. Really and truly most ideas take 20 years to become an overnight success in Silicon Valley because we fail our way into the future. And by the way the valley is damn good at failing. The valley is built on the rubble of failure not the spires of earlier success. Think about why did the web take off in Silicon Valley. After all it was invented on the Swiss French border by an English boffin who thought he was designing group ware for physicists. Well it took off here because the whole industry was collapsing just before the web arrived, the interactive TV industry, and it left a whole much of programmers out of a job, but they had been sensitized to rich media and video and the like, and they are literally standing on a street corner going, gee what do I do next. They see the web and they go, cool. The first – you know the first really big web development company US Web – its first office of any size was a building they had rented from Apple at below market rates, and Apple said you can have the contents too. It was the Apple Interactive TV Studio, but let me give you another recent example of an interesting failure. This is from 1985, how many people here are on second life. Show off your hands, this is where it started this is the first graphical multiple user dimension Habitat from Lucus film in 1985 wasn’t around for a long but basically did every thing that you do in – in second life he said very slowly and it was followed by failure after failure after failure and then sure enough you know in the almost - almost 20 years we get second life and the take off. The lesson here, anybody who is doing investment or – or you want to start a company, you want a short term win, you don’t want to wait for 20 years, look for something that’s been failing for 20 years and everybody has said it is never going to happen and chances are that's the thing it will take off. This rule is the hardest. Be indifferent as a fore - I am a forecaster not a futurist because I am a professional bystander, futurists are advocates about the future, inevitably you scratch a futurist and beneath their policy talk, they are passionate about something personal – personal, helicopters or rocket ships or preserving their – their brain in a computer or something, that's fine to be excited about the future, fine to be an advocate but engage in cognitive dissonance. When you are making the forecast, detach what you wish would happen from what you think is likely to happen because you get excited, you arbitrarily narrow the cone and you are in a deep – a deep trouble. It results in diminished under appreciation of uncertainty and often times in just playing old fashion irrational behavior, like Christian fundamentalists who believe in the end of the world. I don’t know for sure that the world won’t end but I do know there are some people who believe that the world will end and they are trying to do everything they can to make it happen. These people have been wrong for 2,000 years and it doesn’t seem to bother them that they have the worst forecasting record of anyone around, but being wrong I must add can still be very profitable, Hal Lindsey’s is "The Late Great Planet Earth" out sold every book in the 1980’s except the Holy Bible. So - but it is also useful because if you think about indicators being everywhere the folly of other fools can be a very powerful indicator that something was going on and you can take these and stick them together, take multiple events when I – once I thought about y2k for a while around 1989 - 1990 after the harmonic convergence of August 1987 which was the happy end of the world, the Branch Davidians in 1993, the unhappy end of the world and Heaven’s gate you know they were way off in 1997. The unhappy end, there was – it was cruel but there was a bumper sticker that read so many – so few comments - so many stupid people. And – and the whole industry has emerged, people doing these crazy books, but what was obvious to me in the early 90s is forget your computer, forget y2k the bug in our computer was nothing compared to the software bug in our culture of all of these people who absolutely believe this stuff and built an industry around it and also if you look at the history millennial movements the biggest impact of the prophecy happens after the prophecy is failed to come to pass and if we had thought hard about that, we would have understood what was going on. An indicator that I didn’t figure out, but set off alarm bells for me was the bombing of the – of the Buddha by the Taliban in Afghanistan, when that happened in spring before 9/11, I was calling friends at the state department and else were saying I don’t get it but this is a big deal. Something is going on, there is a – there is something happening you know disguised, I dint figure it out, but by the way Tom Clancy did, you know he wrote a novel five years before the fact that had a 747 flown in to the white house by an angry Japanese pilot not by those others. This is also an indicator in my opinion speaks to the circumstances today. We are still - we are surrounded by millennial crisis cults whether it’s the Taliban, whether it is fundamentalist Christians trying to use recombinant DNA to create a red heifer in Jerusalem to accelerate the second coming, in my opinion the single largest facture affecting global politics today is the cultural effect of the millennium and nobody is talking about it, but let me push on to another rule, assume you are wrong whenever you make a forecast and forecast often. If you must forecast, go back look at your forecast and be ruthless about knowing where you are wrong. Good forecasting is the inverse of a traditional good research. A good researcher will spend a lot of time you know carefully looking at the data, holding off, coming to a theory and then finally reach a point where he can come to a conclusion and then ignore everything else that doesn’t match the conclusion. As a forecaster I can never have 100 percent information. So I do the opposite, I come to a conclusion as quickly as I can and then I set out to demonstrate why I am wrong because I know if I don’t do it, somebody else will explain it. The consequences of the failure to forecast often were sadly but dramatically demonstrated. On September 8th 1923 on the coast of California south of here a couple hundred miles. This boat - this destroyer the US's Chansey did not make at San Diego. It was on an engineering run just after fleet week in San Francisco as part of Squadron eleven destroyer force battle fleet - this is back in the old days where destroyers huddle together. There were 14 force stackers going in a line down the coast and they ended up on the rocks because of a particular kind of forecasting error. Actually only nine ships made at ground, the others made it offshore and here is story of why. They were going down the coast cruising at about 20 knots it is a lee shore sailors know that is a dangerous thing. There was foggy weather there was wind this turn but they were using what's known as dead reckoning, what you kind of guess at your position in the water, navigators know what this is. For lay people, dead reckoning as one navy skipper explained to me, he says that's – that's navigation by gosh and by golly and by god and their calculated position is exactly – is where the dotted line starts there. This is the California coast just beyond Santa Barbara where the channel turns it towards Los Angeles and so they thought they were in just about the right spot where there should be turning in. This area by the way is known as the graveyard of the pacific at least California sailors and the trick here is here is you better make the turn right because if you go in a straight line, you are going to hit the San Miguel islands which they were very worried about that day because just that morning a passenger ship, the Cuba grounded on San Miguel island and one of the other destroyers the, US Reno, sailed towards rescue and say said for gods sake let’s not miss the turn. They were - aware of the risk, they were also knew was high uncertainty in their position that were guys and they were reluctant to slow down because they were on an engineering run trying to get a record. Well they had some other technology, they had a good forecasting tool, it was a radio station at Point Arguello Light station NPK that would you would radio them and they would radio the bearing that you were on. The skipper of the lead boat, Lieutenant commander Hunter requested bearings and they said you are at 333 degrees that was consistent with the earlier bearings they have got and his navigator Lieutenant Blojit trusted the radio direction finder but commander Hunter was a little old fashioned and he was skeptical of this new fangled radio thing and more importantly the bearing that they gave him did not match his fix that he had made by dead reckoning and he thought well if it doesn’t match my position, it should be wrong. These are kind of like heavens gate folks, heavens gate went down to ocean side telescope and bought a Celestron C11 to look at Comet Hale-Bopp so they could see this space ship flying in behind the comet that was going to carry them up this planet, they took it home, they set it up and they didn’t see the spaceship. So what did they do? They returned the telescope because was defective. Now it’s not quite that bad, but the immortal words of lieutenant Commander Hunter, his instructions dispels his radioman quote impossible bearing, tell them we are at south of Arguello, damn it, ask them for the reciprocal bearing. The reciprocal bearing you know it’s a – it’s a radio beam kind of - you can either say well you are 333 degrees or the other way and he said – a 168 degrees. Two miles east of my assumed position but that didn’t insult that coming down the California coast and he issued the faithful order at 21:00 hrs change course to 095 degrees That was their actual position. For those if you don’t know what 095 degrees is, that’s a hard left turn into the coast line. The die was cast. In fact the original bearing was right. Meanwhile what happened was the Delphy’s navigator wasn’t the only one harboring doubts. The 11th Boat back – oh by the way, one of those wonderful ironies of forecasting, the lead boat was called the Delphy. You know as a forecaster I just pray for coincidences like this. So the 11th Boat back, the Kennedy, the navigator there had doubts and he violated Navy Doctrine. There were two violations in Navy Doctrine that night. The first one by the navigator, he listened to the radio bearings being send to the Delphy. Under destroyer doctrine at the time only the navigator in the lead boat was allowed to navigate and it was against doctrine for the other guys to even listen it. He listened and he was plotting it, he was saying we are not what they think we are. And he kept tugging the arm of his skipper, Commander Roper who said we are not there, Roper agrees – and Roper makes a second violation of the evening, which almost ruined his career; destroy your doctrine at the time as the boats followed each other. And you will always follow the boat in front of you. And you never varied off that because they had really thin skins and the only safety for destroyers was to stick close together and shoot all at once. He – as the Delphy executed its hard left turn on to the coast, Roper says, I am not so sure we are going to delay the turn and kind of scotch a little out. And sure enough they then saw the other ships piling up. He turned out to see – he led the remaining ships to safety. He was court marshaled The lesson – part of the lesson is – if you want a piece of California geography named after you, drive your destroyer on to the beach, the mistake is immortalized as destroyer rock just off the California coast. But the real lesson here is about uncertainty. When the Delphy skipper hit the rocks along with those nine other destroyers, it happened because he narrowed his cone of uncertainty at the very moment that that data was screaming to widen it, to hedge its bets and to hold off and just see what the future would bring. And his mistake is memorialized on this chart for all the rest of us. So unless you want to get your name on a rock somewhere, and if you really want to understand uncertainty, the lesson above all is this; embrace uncertainty. In all of its complexity and gut wrenching port hand of change, uncertainty is our friend, uncertainty is opportunity. And the last piece of advice is to really understand uncertainty you must face change head on. And what I love, being tuned to indicators elsewhere, I didn’t even have to do a power point because I found the indicator the best advice I have found for forecasting, as long as I can remember on the counter of one of my favorite espresso bars barely 20 blocks from here and it read this, “Hurray for tips ups, If you fear change leave it in here.†That is my advice; embrace uncertainty, if you fear change leave it in here. Thank you.