MIKE COHN: So let’s get started We’re going to talk about how to plan and track on agile projects I don’t want to bore you with a bunch of my background, but I want to tell you one thing because it’s going to be interesting and relevant to what we talk about The late ’90s was a great time for me I worked at a good company with a really good CEO– kind of the one time I had a really good CEO And I took advantage of the situation to get good at estimating and planning– not necessarily me, but the teams I was working with I was VP of engineering there and my CEO understood that not every project needed to go be a big blockbuster and make a lot of money Not every single project needed to come in on exactly the right date, that we would be late sometimes, some projects would make a lot of money, some might lose money But overall we had a good portfolio of ideas and that was more important to this company So I took this opportunity to help my teams get good at estimating and planning our projects And what I told them to do was that every project had to be estimated in a different way And I put in two simple rules This first one was really easy You had to tell me what you were doing I wanted to know what they were doing I was not going to approve it, or anything like that, I just wanted to know what people were doing, because I wanted to get good at this I wasn’t good at it And the second rule was, you had to do something different You couldn’t do exactly what the other teams did This worked out to be the most important rule because it led to a very quick, kind of, Darwinian survival of the fittest idea approach Because, let’s say, you’re starting a project in November, you would look at all the projects that had gone before Maybe they weren’t totally done but they were halfway through and they were on schedule and that looked promising And you’d look at a project that had gone before and you’d say, wow I want to do what they did Well, maybe they only got lucky Maybe they didn’t really have the right estimating process They got lucky Well, you’d come to me and you’d say I want to do what they did and I’d say you can’t do it You got to do something slightly different So they would find some slight variation If that worked well, the team that maybe started in February would look at what you’ve done It was a minor improvement, you’d go with that So this led to a very quick evolution of ideas and I’m a talk about some of the things that involved through that period because some of them were very minor I did not make people change it in a big way Some of the variations were very small So we went through ideas very quickly as a good way to become good estimator More importantly, good planners of our projects So I want to talk about how to estimate and plan software projects But what I’d really like to talk about is, suppose you’ve been doing this a long time We’re here at Google and I appreciate Google having me out here, but a lot of us have been doing software for a long time and we’re fed up Right? This is a tough job You know, we’ve got to think hard and we get deadlines and sometimes were here late It’s stressful What if we decided enough is enough I’m tired of software development Let’s all go back to our one true love Let’s go into the landscaping business Some high growth industry Let’s go into landscaping Sounds more fun Well the first job you’ve got I’m going to hire you I’ve got a big pile of rocks, as you can see, in my front yard and I need to move this pile of rocks into my backyard I’m from Boulder, Colorado area We got snow on December 20th You might have seen our blizzard in the news It’s the one that shut down our airport for days and days Well up until about a week ago we had snow still on the ground This never happens I used to live out this way I never would have moved there, if I had known the snow stayed on the ground And it normally doesn’t It stays on for a day or two But we’ve had snow on the ground for three straight months almost now So now that the snow is finally gone I do have a big pile of rocks that’s uncovered I now need to move it into the backyard I’ve been able to get out of it for the last few months by saying, it’s covered with snow I don’t want to move it Now I need to So suppose we’ve got this big rock How are we going to estimate this Well one way is going to be to look at it and estimate how many wheelbarrow loads are represented by that pile And then after an hour let’s just see how many wheelbarrow loads we’ve moved Based on how many we moved in an hour we can extrapolate the total duration Not a big surprise that this technique might work this way I think there’s 80 wheelbarrow loads there I work for an hour and I’ve moved 20 I don’t know what I was thinking when I made that slide up It gives a very optimistic estimate of how many wheelbarrow loads I can move in an hour But, you know, maybe I’ve moved four instead of 20 So if I assume I can do 20, I can do the math and say, well there’s 80 I’ve done 20 Therefore, I’m done in four hours Very simple estimating approach And I guarantee when I am doing this work this weekend, or maybe I can put off till next weekend, this is how I’ll do it I’ll think about it this way What I won’t do is this I’m not this anal But I could if I wanted to, draw a graph of the progress Right? It might give me a good two minute break to go draw this

Fire up a charting program Now here I’m showing that I started out with 81 hours, later I’m down to 60 So I’ve got this little graph of my progress Well OK, we’re not going to make as much money if we go into the landscaping business, so let’s stick with software development I just wanted to tell you about my landscaping problem So let’s get some terms out I want to make sure we’re using some terms consistently So I’m going to use the term iteration to represent a short constrained period of time It’s a time box Typically going to be one to four weeks Extreme programming teams tend to be on the shorter end of this They tend to be one or two weeks Scrum teams, another agile process, typically tend to be on the two to four week A little bit more in the middle to the end side Note that they overlap There’s a lot of overlap between scrum and extreme programming, of course So anywhere between one to four weeks, maybe a month, a calendar month, if you want So it’s an iteration We’re going to have a couple of those iterations on this diagram I want to think about the term release A release typically multiple iterations You may not actually release it I do a fair amount of work for some video game development companies Video game development companies can’t do periodic releases If they’re doing kind of like a Triple-A game that goes in a box, they have to release it one time Then they get the next version out next year if they’re doing like a football game or something But they don’t get to put out a release every six weeks So those types of environments might break it down and say well, we’re not actually going to release this out to the world, but they’re still going to think about it in terms of a release, maybe a quarterly milestone Where are we going to be in a quarter? So a release is going to be multiple iterations Within these iteration I want to put in some work, alright? I’m going to put in– I like to think of these as user stories– the requirements, the features that we’re going to do within iterations Let’s assume that we’ve put in these units of work So what I’ve done here is I’ve put in some units to work, the smaller boxes, with some numbers on them, some labels This one here is a four, It’s about twice the size That’s a bad one That’s mislabeled This four is about twice– let’s call it mis-estimated Right? I did that on purpose This one’s a four It’s about twice the size of the two It’s about four times the size of the one We’re going to talk a lot this evening about how to put these numbers on here For now let’s just assume they’ve somehow got on there So I have seen some estimates on these units of work, typically user stories for me We’re going to talk about a term called velocity Velocity is the amount of work planned or completed within an iteration So in this case, let’s see if I can add one of these up On the left there I’ve got fifteen, this one over here This iteration we’re planning to get 15 units of work done I use the term units of work Agile teams typically estimate in one of two units They typically estimate in what’s called ideal time or they estimate in what’s called story points We’ll talk mostly about story points this evening but both units work So velocity is going to be a measure of the amount planned are completed an iteration So we may hear the terms like planned velocity Our planned velocity is 15 That’s what were planning to get done in the next five iteration 15 points per iteration Or we may talk about an observed velocity We ran the last iteration and had an observed velocity of 16 One better than our planned velocity So you may use the terms that way Just some terms I just want to get out We think about planning on a agile project Planning occurs at multiple levels I like to think of this as an onion A planning onion I like that Planning kind of stinks on most of our projects Onions stink, so I like to think of them going together that way So we’ve got the daily plan here at the bottom Where do teams make their daily plan? Yeah, at the stand-ups The daily standups, the daily scrum, whatever you call it We have the iteration plan Teams get together at the start of an iteration and say, what can we commit to in the next two weeks, four weeks, whatever it is They make an iteration plan We have released plans Where are we going with this thing in the next three to six months? Again even if it’s a two year game or a two year big, huge project We’re going to break it down into a little bit smaller things Product planning Where’s this product going over its life Well in ’07 we’d like to do this In ’08 we think we’ll do this and we’re not really sure what we’re going to do in ’09 So some level of product planning Portfolio planning Does this product even fit in our portfolio? Where are we going? Google’s an interesting example We’re here at Google tonight Always when I do this I’m trying to think of an example of like well, I bet your company wouldn’t do this, right? Being at Google its hard to come up with an example of what might not be coming next, right? One of the jokes was, somebody, before we got started was joking about, well we might be working on a cloning system, right? Who knows, right? So normally with a joke I can make, is I bet you’re sitting back at your desk at work tomorrow you come up with a great idea for an operating system You go to your boss and say, hey we’d like to do an OS

It doesn’t fit in the product portfolio We’re not going there I don’t know if that works here, right? We might be coming up with some sort of an OS over time So do things fit in our portfolio? What’s our company’s strategy? Where are we going? What are we trying to do? Maybe we decide to come up with a new printer That probably doesn’t fit in the portfolio or the company’s strategy here We’re not likely to switch to being a hardware business So agile teams plan at least three bottom levels Somebody in the company is planning above here Product owners typically Scrum calls and product owners are responsible for a level or two above this Product owners typically aren’t very involved in the daily plan, but they will be involved in the iteration release product plan– maybe the portfolio plan Further up the organization we’re going to have product management, executive staff, those types figuring out the portfolio and strategy But agile teams are going to plan at these lower levels These three lower levels And these are critical for the execution success of projects Now I want to show how these three levels relate in my mind This is an interesting picture You’re going to see this slide later I’m going to bring this one back in a half hour or so and you’ll see this in a slightly different context So agile teams plan at these three levels On the left here in the blue we show agile teams planning with their, what’s called, their product backlog This is a scrum term for a requirements document essentially Product backlog, a feature list, what we plan to do in here When I first started doing scrum for the very beginning back, first project I ran, we started doing scrum in January ’95 We didn’t use the term product backlog Ken Schwaber hadn’t started writing about scrums. I hadn’t come across that term We just called this a prioritized feature list It was just a prioritized list of the things that were desired I do use the term product backlog these days of course but prioritized feature list is a little bit more descriptive of what it Is So I mention that as a term right now So we have this list of things that we’d like in there and I want to have some sort of size estimate on those Associated with this, we also have an iteration backlog over here in the green, to the right The iteration backlog is the list of tasks and the hour estimates for those tasks Now there’s a lot of debate about what’s the right unit for the product backlog We’ve got these things over here I’ve written these in the form of user stories, short simple statement told from a user’s perspective, and we’re going to talk mostly about story points over here But there’s a lot of debate Maybe you want to do ideal days There’s not a lot of debate on the iteration backlog over here This is tasks in hours I have seen teams doing other things They’ll put some sort of point over there I’ve never heard a good argument for doing so You’re about to do the work You’re planning your iteration, your two week iteration, four week iteration, whatever it is You’re in a meeting planning the work you’re about to do in the next couple of weeks You should be able to take a guess in terms of the number of hours Teams that want to do it in something other than hours are really just saying, well we can’t estimate this and we want to put something on here because we have to but we can’t estimate it Normally they mean they’re afraid of being wrong They’re in a company where there’s too much pressure to be right on these things and they’re feeling too much pressure on this That needs to be removed from the team because, yeah, some of these estimates are going to be high, some are going to be low, it’s just a tool to figure out what we can commit to during iteration So no debate in my mind on the right side here This should be tasks and hours On the left down here in the red this is the iteration– of the daily standup The daily standup, the level at which we plan, is promises or commitments to our peers One of the nice things about doing this as a standup meetings is we don’t stand up very often at work And so we stand up during the daily scrum, the daily standup meeting, and say I will work on this That’s a nice commitment to your peers All you’re committing to doing is not getting pulled into 32 other meetings Not as though he’s saying, I will finish this or I will do this In a lot of companies saying I will work on this, that’s a big commitment That means I won’t get distracted into 14 other things So it’s a big deal So three different levels we’re going to plan at I just want to show this picture about how they relate We’ve got product backlog on the left Individual items on there get broken down into tasks in hours During the project, during the iteration, day by day, we make promises or commitments to our peers This here is a diagram– whoops– diagram of the overall picture of what agile planning is going to look like, in my opinion I start out up here with the release planning at the top Release planning starts when somebody brings to the team a set of conditions that they hope they can have satisfied They come to the team and they say, I’d like this much, I’d like this scope I’d like it by– let’s see, were here on March 20th– I’d like it by June 20th I’ll give you three months to do this And you can use the team you have and two new hires Or, you can move two people from that other team

So they bring us a set of features, a set of resources in mind, and a deadline in mind Scope, schedule, and resources And the team– they’re job is not to say yes, we’ll go do this, if they can they should say yes, we’ll go do this– but their job most likely is to explore the solution space, figure out how we can best satisfy this person requesting the software So I’ve shown this is an iterative process That’s the red arrows up there We do release planning by trying to figure out how can we best satisfy this someone who is after something more that we can probably deliver That’s typically the situation And I don’t mean that in a negative way that our product managers are after more than we can deliver I mean that in the same sense that when I go to Best Buy are Good Guys there’s more in there that I can afford to buy, right? I go in there looking for a nice TV. Well I’d like the bigger TV, and I’d like one for the kitchen too And new speakers would be nice We always want more than we can have So I don’t mean this in some dysfunctional product manager’s trying to put us under the screws way I just mean we want a lot of stuff and how could we get it done in time So we do release planning We iterate over that We explore the solution space to figure out how best to meet their needs, to satisfy their conditions of satisfaction, their expectations for a project That’s release planning We also have a level of iteration planning that occurs Notice that iteration planning is going to start out with the same pattern here I’ve got a set of conditions of satisfaction that come down But I’ve only brought down the scope for the iteration planning Up above I have the scope, schedule, and resources Why do you think I didn’t bring down the schedule and resources into iteration planning? What do you think? Yeah, I already have the schedule These are essentially fixed Agile teams have learned that it’s good to have a fixed rhythm– a cadence to their projects I tend to like two week iterations What scrum calls a sprint So I run two week sprints, two week iterations Every two weeks we do iterations I don’t start them out by saying, how long should we do this one? And we do a four week and then a two week and then a three, and then an eight, and then a two Teams have learned it’s nice to have a consistent rhythm So the schedule isn’t really negotiated during iteration planning Neither are the resources In most environments it takes more than one iteration to begin to blow the budget We want to spend more money We need new hires Well we may get that approved, but it’s certainly not part of iteration planning It takes longer than one iteration to even bring somebody on in most environments So iteration planning is iterative as well We look at the iteration We try to figure out how best to start to satisfy our expectations from our product owner or customer We iterate over that trying to figure out what should be built in the iteration We develop some software We end the iteration with potentially shippable product Something that’s truly done We may not choose to ship it, but I want it done enough that I can look at it I can poke it and prod it, see how it works I can show it to people and say, what do you think? What should I do next? How do you like this? I can get feedback from it Some of the things we might be feeding back? Maybe feeding back information on the quality of the release of the potentially shippable increment How are we doing on a quality level? How are we doing in terms of the features that we’ve added in? What do you think? Do you like these features? I may get feedback by showing this to people they want new features we hadn’t thought of I may get feedback that features that we have thought of our higher or lower in priority Let’s move our product backlog around I’m going to get feedback based on how fast the team is moving Wow, this team’s going really great They’re going very quickly Let’s accelerate the release date Or, well the team’s going at a pretty good pace, but we’re adding new ideas Maybe we need a second team on here So all these types of feedback are going to be filtering in to what we’re doing So this is an overview of the high level process Here’s the three things that we’re going to talk about now We’re going to talk about how to estimate This is the hard part of what we’re talking about tonight Estimating and planning Estimating’s hard Planning, when we get to it, is actually going to be really simple Then we’re going to talk about something called a burn down chart So I wanted to make sure we talked about estimating, and planning, and tracking tonight So those are the three things that we’re going to talk about By the way, I don’t have the last block a time reserved for questions so if you have questions please just ask them in context So if I’m talking about estimating, ask me an estimating questions If you ask me about burn down charts I’ll say, wait a few minute, we’ll get to that But generally, in general, ask me questions I like to take them in context Because if you have a question probably lots of other people do Let’s just talk about story points Oops I just said I want questions then I tried to move on

without getting one AUDIENCE: Can we get copies of the presentation? MIKE COHN: Oh, yes Can you get copies of the presentation? Yes, I’ll have them available for a price So, no I’ll have them on my website hopefully tonight So assuming the connection in the hotel is fast enough to upload things I’ll have them up on my site So you can go to mountaingoatsoftware.com and then there’s a presentations tab, and it’ll be up there So, I meant to try and get it up there this morning, but yeah, this will all be up there I put everything up there So even things that I did at the software development conference is up there That’ll be up there tonight as well I put everything up there I’m not real big on treating this stuff secretly Story points Probably the most common technique for estimating today I think teams have started to realize that this is a good approach, and I’ve been pushing this pretty hard over the last couple years as the best way to estimate So a lot of teams have switched and begin to estimate in story points rather than other techniques like ideal time It’s a very common approach The name story points comes from the product back log or requirements technique of user stories So we estimate user stories– requirement statements– in story points The idea of a story point is it’s essentially a combination of the size and complexity of the work Now if you think about the size and complexity, if you add those two things together, what you really get is the duration– the amount of time that something is going to take So story points are about estimating the time that something is going to take, but we’re going to estimate them in unit lists, but numerically relevant numbers What I mean by that is we’re going to call something a five The other thing we’re going to call a 10 What that means is the 10 will take about twice as long to do as the five It’s about twice as big as a five Now unit lists, in the sense, that I don’t care if you use 50 and a 100, five and 10, I actually encourage you to use fairly big numbers The reason I want to use fairly big numbers, let’s say we estimate in the billions This is going to be nice because at some point your boss is going to be hanging out at a cocktail party with another agile boss They do this You may not know this, but they go to cocktail parties and talk about their teams Now if you estimate in the billions, your boss is going to say something like, oh, my team has a velocity of 13 billion What about the other boss who’s going to say, well wow, my team only has 46 You’re getting 13 billion, we get 46? Your boss is going to come back, really proud of you, the bonuses will be flowing The point of this silliness is that they’re not comparable that way There are things we can do within an organization to standardize what a story point means, right? So we can go into your company and we can get a story point to equal a story point across all teams within one company We can’t do a worldwide We’re not going to do it across the whole software industry But within a company we can If you’re interested remind me and I’ll tell you how you can do that We still got that landscaping stuff in my front yard I told you about the rock I didn’t tell you about the big pile of walk-on bark that I have to I’ve got another part of my backyard that we want to put these tree shavings down and we had some kind of drainage issues Were going to fix that by mostly just covering it up with a big pile of bark So I’ve got these two big piles of landscaping material in the front of my yard There in about the same place in the front yard, close enough that it’s not going to affect our estimate They happen to be about the same size I know that within a very small margin of area they are the same size because I paid the landscaping company to deliver piles of the same size I ordered the same amount of each And they’re essentially going to the same place in my backyard– one over here one over here They’re essentially going to and from the same distances What estimates might we put on those? Let’s say we look at that drain rock The great big rock I’m going to put, let’s say, 100 on there If we’ve called the rock 100, what do you want to put on the same size pile of a lighter material going about the same distance? AUDIENCE: 30 MIKE COHN: What? 30 I heard a 30 Let’s go with 30 So with 30 we’re saying I can do that about a third of the time So moving the rock will take me about three times as long as moving the bark will be I can buy that When I go to shovel the rock, that’s kind of hard I have to shovel the rock I’ve done this before I know that sometimes I’m shoveling it and I tip the shovel and the rock falls out Every now and then I get the wheelbarrow half full and it tips over because I haven’t loaded it evenly and I have to start over, right? The bark is really easy Just shovel it in It’s actually kind of fun– just don’t tell my wife that It’s actually kind of fun to shovel the bark in, right?

That doesn’t bother me The rock, yeah, you have to take break That stuff’s hard to do I don’t want to do that all day The bark is going to be fun The rock, not so much So we’ve put numbers on these We put 130 Or, we put 100 on one and 30 on the other OK Again that has nothing to do with what we’re talking about, we’re talking about software So let’s skip the rocks for a few minutes again I just like coming back to that because you guys are now helping me figure out how long it’s going to take to do this work I’ve got to do at home I’ve been talking long enough We’ve got all this great food here Google’s renowned for it’s food I’d like to get some So what I’d like to do– while I’m doing that– I’d like to have you do something for me I’d like you to estimate in zoo points A few animals I’ve got up here Now were going to define a zoo point as a combination of the mass and volume of the animal So it’s not just as simple as taking its weight I want to make it a litter harder than that So the mass and the volume The giraffe’s got a lot of volume, real tall So a combination of mass and volume Estimate in zoo points these eight zoo animals Do these in little small groups One of the fundamentals of agile, by the way, is self-organizing teams. So I’m going to let you self-organize So self-organize into groups of probably four or five where you’re going to do this Were going to do one more fun exercise later where you’re going to be in groups So be in a group of four or five This, it really doesn’t matter how big you are but later we’re going to want to be in four’s and fives, probably, so I’d encourage you to start out that way So go ahead and do this for a few minutes while I get some of the good looking food OK let’s come back together and see how we’ve done with our eight zoo animals Which group has it right? AUDIENCE: We all do MIKE COHN: You do? AUDIENCE: I don’t know MIKE COHN: You guys in the back have it right? AUDIENCE: What is the definition of right? MIKE COHN: I just need a group that we can use as an example That’s all I want OK Let’s go with yours What did you come up with for lion? AUDIENCE: 10 MIKE COHN: 10 Everybody OK with 10 on the lion? It’s hard to be wrong on the first one OK What about kangaroo? AUDIENCE: Eight MIKE COHN: Eight Kangaroo about 80% of the lion? Everybody? No? Kangaroo’s smaller Smaller than that? They’re kind of tall Aren’t they about human size? OK AUDIENCE: [INAUDIBLE] MIKE COHN: What about rhino? 15? So we’ve got a 15 on a rhino By the way, I’m sure that you mean these numbers in the billions, [LAUGHTER] right? A common agile convention is to just drop the billion, right? So you were already ahead of me just dropping the billion So a rhino’s a 15 It’s about 50% bigger than a lion Everybody OK with that? AUDIENCE: [UNINTELLIGIBLE] a lot bigger than a lion MIKE COHN: It’s a lot bigger than a lion? OK What about a bear AUDIENCE: 10 MIKE COHN: 10 So a bear’s about the same as a lion Well I meant those– well I meant those little guys that live in Australia– the cute ones I see at the zoo AUDIENCE: The Koalas? MIKE COHN: Yeah, koala bears These guys AUDIENCE: They’re vicious MIKE COHN: They’re vicious? Well they’re not as big as a lion I mean, we’re not rating viciousness, right? That’s not a 10 How’d you put a 10 on this guy? AUDIENCE: You were thinking of a black one MIKE COHN: So you were thinking of black bears, or grizzly bears Well you didn’t ask me I’m your product owner Nobody asked me You guys did You guys did AUDIENCE: You were off chowing down your own food MIKE COHN: OK I was off enjoying the food I wasn’t that far away, though Yes, one group did ask me and I said I didn’t want to answer I said that you were on to me and I wanted to set up some other group for being wrong Don’t worry If you had said, oh we think they’re a two Were thinking of panda bears I would’ve said, panda bears? I’m thinking of the great big grizzlies I would have gotten you either way Did anybody estimate bear in a range? Did anybody say, well they’re kind of small, they’re kind of big? Did you guys have them as a range? No AUDIENCE: We did with the gorilla MIKE COHN: You did with a gorilla OK So let’s keep going with these What about giraffe? AUDIENCE: Uh, 13 MIKE COHN: Giraffe’s are 13 So a giraffe is in between a lion and a rhino So kind of halfway in-between a lion and a rhino That’s a pretty small rhino [LAUGHTER] I’ve seen them They’re their bigger AUDIENCE: A baby rhino MIKE COHN: Oh, a baby rhino OK [LAUGHTER] So we can keep going through these Now we’ve got a group here, more than just that the group

that estimated this And with the collective wisdom of all of us here, we can probably refine their numbers a little bit Maybe we’d up rhino a little bit, right? The rest of us can talk them into a bigger number on rhino So we could refine their numbers and among the group here we could come up with slight improvements Now this is key in the sense that, “the wisdom of the crowds” type of a concept, right? Remember the, Who Wants To Be a Millionaire show? And you had two ways, or a couple of ways, to get advice One was to have your smartest friend available on the phone A lot of us have some pretty smart friends that know all sorts of weird trivia Yet, our smartest friends, based on what they experienced from that show, our smartest friends were not as smart as stray people who happened to be in the studio answering the question Right? There’s a lot of wisdom when we put a bunch of people together to figure things out So as a group we might have been able to come up with slightly better numbers On the other hand, what if I’d thrown out some different items at you? I said, well as your product owner I meant to include a couple other items on here What about Indian elephant? AUDIENCE: 30 MIKE COHN: 30 So an Indian elephant is about twice the size of a rhino? They seem a lot bigger What about blue whale? AUDIENCE: 1,000 MIKE COHN: 1,000 100 It turns out we’re pretty good estimators when we stay within one order of magnitude One to 10, 10 to 100 Or, one million to 10 million, whatever it is Within one order of magnitude were pretty good When we start to go outside of an order of magnitude that’s when things start to go wrong Now often on our projects we do have to estimate some things like this plus a few blue whales Just be aware that when we’re estimating those blue whales there’s a lot more uncertainty in those If you’ve got a lot of those you may need to express your overall project in terms of a range We’ll be done in third quarter instead of saying October or instead of saying August You might have to say third quarter You may have some uncertainty in there when we have a lot of these things Let’s go with a different group How did you guys get started? The other team in the front here, what did you do to get started with this? AUDIENCE: [INAUDIBLE PHRASE] MIKE COHN: OK, interesting So you just gave them all a 10 and then said what looks wrong In the back, what did you do? AUDIENCE: [INAUDIBLE PHRASE] MIKE COHN: OK OK So you ranked them by size first. Kind of did a bubble sort of the zoo animals and then it was way easier to put them on their, right? Well that works until we’re doing a couple 100 items, right? I always teach these classes and I give people eight or nine things to do, right? It’s a little different, then we go to our real world and we got a 100 items on our product backlog So we might have to do more course grained sort if we have 100 items. Any other techniques people use to get started? What’d you guys use? AUDIENCE: [INAUDIBLE] MIKE COHN: OK, so some loose bucketing of them OK, the rhino and the tiger are about the same size This and that are about the same size AUDIENCE: [INAUDIBLE] MIKE COHN: Bin It’s the same type of thing We’ve got to make sure the SPCA isn’t listening, in case we’re saying like, oh we put the animals into some loose bins and then estimated them, right? That’s not going to sound real good Different approach? AUDIENCE: [INAUDIBLE] MIKE COHN: OK, so you established a range Let me drill into that before I answer the question So you established a range Did you have a one and 100 in here? AUDIENCE: [INAUDIBLE PHRASE] MIKE COHN: OK, you picked 100 for the big animal Which was your biggest? Hippo So the hippo and rhino both got 100 What was the lowest number you used? AUDIENCE: [INAUDIBLE PHRASE] MIKE COHN: OK, let me see if I can paraphrase what you really did here You said we’d like to put things in a one to 100 range So we’re going to think about them that way And then you thought about your biggest– you put 100 and then you estimated and all the rest– and you didn’t end up with a one to 100 range, you ended up with a 50 to 100 range Yes, yes, yes let me get there So you’ve left yourself room below There’s always room above, right? We can always go bigger We can go smaller but doing things in decimals starts to be real silly, right? So you left yourself lots of room for rabbits, and mice, and little ants, little tiny things down at the bottom And that’s good I like that What I was concerned with is when you said we set up a range, because I will see this People will say, hey let’s estimate one to 100 They’ll say, what’s the smallest? Oh, kangaroo Let’s give the kangaroo a one What’s the largest? A hippo Let’s give that a 100 And that would be false because they’re not a 100 to

one size difference So it’s fine to say we’d like to use a range of one to 100 but then do exactly what you did– put them into there I’d like the basic math properties to hold when we estimate this way That sounds complicated All I really mean is four two’s should be about the same as an eight The basic math properties should hold The other thing that it turns out that helps us be good estimators is relative estimating, which is what we just did, estimating the relative size of things There is a professor at NYU, New York University He asked his students to estimate the height of the Empire State Building He got back estimates that ranged, on the low end, from 200 feet Imagine that for a second Imagine tipping the Empire State Building over and having it be smaller than a soccer field 200 feet to two miles Three kilometers Two miles high Imagine looking up and seeing that thing in the sky We are bad at absolute estimating We are not so bad at relative estimating Think about the fear If I told you that your career is dependent on you estimating how many pounds a rhino weighs, that would be really hard I would be nervous about that What if I said instead that well, your career rests on you estimating how much bigger a hippo is then a, lets make it, a gorilla I’m going to give you a day You can’t go get on the internet and figure it out I’m going to give you a day to figure this out I would much rather estimate the relative size of things It’s much easier for us to do A couple of reasons why this type of estimating works Focuses is on the relative estimate– what I just talked about There’s studies that show we’re better at this I’ve got these references down there Doing estimating this way focuses on estimating the size of things not the duration We then derive the duration empirically by seeing how fast we move through the project We’ve all heard the old jokes about weigh the requirements document Or, analysts who want to weigh the requirements document Well imagine you actually could weigh the requirements document and then something So imagine I weigh a requirements document and it weighs 300 pounds Well then in the first iteration we weigh the part we finished and we finished 20 pounds of requirements Now I know that I’m done in 15 iterations We’re trying to come up with a way of doing that type of thing Figure the size of the project Imagine I ask you to estimate how long it would take to read the new Harry Potter book that’s coming out next month, or something like that I ask you to estimate how long it’s going to take to read that book You look at the book, picture of the book that was in the news, you look at it and you say, oh it looks about 700 pages Then you think about how fast you read and you say, well I better read about a page a minute 60 pages an hour 700 pages divided by 60– that’s about 12 hours And you say, I’ll bet I’m done in 12 hours That’s estimating the size, deriving the duration It’s the best practice way to estimate It is how we estimate in the real world I can give you another example Suppose I ask you to estimate how long it’ll take to drive from here to Austin, Texas You might think about well, Austin, Texas, how far away is that? You’d come up with how many miles it is based on some guess You’d think about how fast you drive Factor in the traffic when you’re driving, all those types of things You’d do the math You’d figure out the duration So estimate the size, derive the duration Put estimates in units that we can add together Time based estimates are not additive This is the big fallacy in time based estimating This is one of the biggest reasons I prefer story points I’m a really crappy runner but I still like to run 5K’s and 10K’s, things like that I’m too big to be a good runner but I still do it I enjoy doing it I’ll probably go for a run later tonight I also got a foot injury It slows me down even more Anybody here a good runner? Anybody like to go do 5K’s or 10 K’s No? I’m not going to pick on you I just needed an example Well let’s suppose somebody is We got Laura over here in our control room We’ll pretend Laura’s a great runner She’s probably giving us dirty looks now I needed a volunteer somewhere So Laura and I are standing at the start of a trail And she says to me, Mike, that’s a five minute trail I say, no it’s not It’s a 10 minute trail I ran it this morning In fact it’s still on my stopwatch 10 minutes and two seconds She goes, well I don’t care that you ran it this morning, I live here I run it every day It is a five minute trail She and I argue back and forth– five- 10 Five- 10 You’ve been in these arguments, right? Or you’ve seen your developers have these arguments– five- 10 Five- 10 We continue the argument We eventually decide to call it seven and a half Now we’ve just done something really stupid It’s not good for her, it’s not good for me

She’s done two and a half minutes early I’m still out panting on the trail past the deadline That’s not a good compromise to make We continue the argument– five- 10 Five- 10 Laura says to me, Mike it’s a one kilometer trail It takes me five minutes I say, exactly it’s a one kilometer trail That takes me 10 minutes to run When we put it into a size we could agree Yes, it’s one kilometer So we can agree on size even though we do it at different rates She runs it twice as fast as I do So time based estimates are not additive because we’re always thinking my time This would take me two days to code It might take you two hours to code You’re a lot better than I am So time based estimates can’t be added but we can add size estimates This would take me five minutes to run It would take you two minutes to run We can add those types of things Yes? AUDIENCE: [INAUDIBLE PHRASE]? MIKE COHN: So the question is what do we do if we have people with different skill levels, right? How can we possibly estimate if we have people with different skill levels? That’s a hard question so I prefer to not answer it I’m not getting paid to be here I’m not going to answer the hard questions That’s not satisfactory? [LAUGHTER] What I’d like to do instead is I’d like to tell you about my daughter I have a seven- year- old daughter, Delaney, at home and she’s going to help me with moving the landscape stuff She’ll help me She’ll help me with the landscaping So I got those two big piles in the front yard Now my seven- year- old daughter does not know cubic volume So she and I can’t go look at that and say, oh, it’s five cubic yards Which I know it’s five cubic yards because that’s what we paid the landscaping company to deliver She doesn’t know cubic yards So lets say she and I go look at that pile of rock– the rock first. We look at the pile of rock and I want to estimate in something that she understands A seven- year- old daughter Seven- year- old kids drink milk She knows what a milk jug looks like A one gallon milk jug So we decide hey, how many milk jugs does that look like? We think about that and we say, well it looks like 100 milk jugs stacked up It actually would be a whole lot more but let’s just use 100 because it’s easy So she and I agree that that looks like 100 milk jugs stacked up Now in my head I’m thinking about how I’m going to do that work I’m going to do that work by shoveling rock into a wheelbarrow and walking it into the backyard Delaney, my seven- year- old, is thinking about, let’s see, I’ll put my little sand pail on the ground and I’ll put rock into the sand pail and I can probably fill it full and carry one sand pail at a time into the backyard So she’s thinking about doing the work at a completely different rate than I am She has a different set of skills She can’t lift a heavy wheelbarrow of rock So I am answering the question On the other side of my yard I have that bark– the white tree shavings So I look at that and I’m going to go with the thirty that we had yelled out earlier I look at that and go, hey, If this one was 100 milk jugs, this piles actually about the same size but I can do this a lot faster And I tell Delaney I say, Delaney, I can do this a lot faster It’s easier to move I think I can do this in about– she doesn’t understand percentages real well, so let’s pretend she does– so I say, Delaney, I think I can do this in about 30% of the time I’m going to call that 30 milk jugs 30 milk jug points She might agree with me Now I’m thinking hey, I’m going to put that in the wheelbarrow I could run into the backyard I can load that heaping full Delaney’s thinking about going, wow, I’ve got two sand pails I bet I can fill them both full and walk into the backyard and I won’t have to struggle I can run into the back yard carrying two pails So I’ve just shown an example where she and I have completely different skill sets yet we can agree because we’re not focused on the actual time to do it, we’re focused on the relative size Now there’s one fallacy in my argument I’ll point that out in a second Because what I talked about is a situation where we can both relate to the work and we could both conceivably do it I have a third thing in my front yard I have three 200 pound boulders They’re about 200 pounds is my estimate I picked one up so I’d know I can actually move that I can pick it up I can struggle with it and I can carry it into the backyard Actually I can get onto the grass and from there I can roll it to where it needs to go

Delaney, my little seven- year- old, who weighs 40 something pounds, could not pick up the 200 pound boulder, right? This is equivalent to some of the programming that we have going on I used to be a pretty good C plus plus programmer I’m still semi decent in Java I could go do some stuff at one of your companies if you want to hire me I would like to do some programming, by the way I love doing it So if anybody needs a pretty lousy programmer let me know I could go do some of the simple stuff you might need But the hard stuff I can’t do that anymore So this is an example where we have a person who can’t even do the job So in that case Delaney may not be able to give me a real good estimate about what that should be So she may not be able to contribute to the estimate because she can’t even do that work and can’t even relate to it On the other hand, my wife, who can’t do the work– she can’t lift up the 200 pound boulder– she could probably contribute to a discussion of the estimate Because she could relate to that She can carry a 50 pound sack of cement so she could imagine how hard it would be to carry a 200 pound thing So she could contribute to a discussion of that estimate Does that make sense as an answer? Good, because it’s a fundamental question It normally does come up Let’s go back to our product backlog here for a second I want to make sure that when we estimate things that we’re comparing apples to apples as we estimate Here’s a common problem that can occur when we’re planning projects We’ve got a product backlog shown over here on the left in blue I’ve got some story point estimates on there Again on the right, Iteration or Sprint Backlog Tasks in Hours Let me take the product backlog Watch closely because I’m going to change those from story points to hours I just want to make them bigger I just want to put some bigger units on there Here’s what might happen if you estimate your product backlog in hours– ideal hours, even ideal days Ideal days are just a little harder because you have to multiply by eight But let’s assume that these are ours now because it’s convenient So I’ve got a 30 hour task, or 30 hour product backlog item up here at the top When we go into the iteration or sprint we plan that We break it down to tasks, we estimate the hours, we come up with 35 hours At the end of the iteration we finish that unit of work That one product backlog has been delivered We completed the 35 hours What will happen next is someone will look at the remainder of the product backlog– 20 hours, 20 hours, 50 hours, 50 hours They will say we have a 140 hours left in this project Somebody will then say, well how many do we finish per iteration? Well we finish 35 What will they do with the numbers 140 and the number 35? They will divide them They will take 140 divided by 35 and say, Ah ha, we will be done in four iterations Here’s what I guarantee will happen You will not be done in four iterations, you’ll be done in five iteration The reason for this is these are not hours These are hours we thought a lot about The team on average spends three to four hours doing iteration planning So let’s not call those hours Let’s call them hours we thought a lot about These over here, the average team spends two to three minutes per item coming up with these These are not hours we thought a lot about These are hours we pulled out of the air or hours we pulled out of somewhere else I can’t take hours we pulled out of the air, divide them by hours we thought a lot about, and get anything meaningful Question? Yes? AUDIENCE: [INAUDIBLE PHRASE]? MIKE COHN: So the question is if these were story points would somebody do the same math? Let me hold off on the question for about one minute and I’ll show you why that won’t be true So I can’t take hours I pulled out of the air, divide by hours I thought a lot about and get anything meaningful All right? The way I like to think about this is we have to be very careful about mixing before the fact and after the fact knowledge What I mean by that is before the fact, we thought things were this big Before we did it, we thought they were this big After the fact, we thought it was this big We can’t mix before the fact and after the fact knowledge I like the way philosophers talk about knowledge Philosophers talk about two different types of knowledge They talk about A-prior knowledge, knowledge before we did it And A-posteriori knowledge, knowledge after the fact I don’t want to mix before and after the fact knowledge That’s what we’re doing in this situation See here’s the real math that should have happened and this will get to the answer why this won’t happen with story points The real math that should have happened is we should have said hey, before the fact we thought there were 30 hours, this number up here My pointer is not bright enough but the 30 at the very top Before we did it, we thought it was 30 Well before we do all these we think there’s 140

So the real math should have been 140 divided by 30 and we would come to the conclusion, well this will take essentially five iterations Which is why I contend that– I wish I could confirm this, but I think it’s every time I’ve seen this, but I can’t say that for sure, so I’m going to say almost every time, although I think it is every– every time I’ve seen this the team ends up being one to two iteration late Any time I see them estimating in this type of way they end up one or two iterations late because they’re mixing the before and after the fact knowledge Question? AUDIENCE: [INAUDIBLE PHRASE]? MIKE COHN: So the question was management came back and said you had to do detailed estimates on the whole thing and she had to take everything to this task in hour level, correct? Yeah The problem with that is it just takes a lot of time to do And I don’t know that it’s any more accurate If I thought it was much more accurate I’d be OK with advising teams to do that It makes people feel better at times, but here’s a way to get around that with the next person that wants to feel better with that Because normally the person who’s asked me that is somebody pretty high powered in the organization– some exec What you do is you say, yeah, yeah, yeah, I’ll estimate for you but you know, I’m really curious I know you’re a golfer When you go out and golf this weekend what are you going to shoot? They’ll give you one number They will not give you 18 numbers They’ll give you one number Ask them to then break that down hole by hole Right? And what they’re going to come up with will not be the same number they came up with the 18, when they gave you the aggregate number AUDIENCE: [INAUDIBLE] [LAUGHTER] MIKE COHN: Well that’s what consultants are good for because we don’t care if we get fired, right? So we can ask that The point of that is when we break things down and then we add them up, we often introduce more error into the estimating process Right? AUDIENCE: [INAUDIBLE PHRASE] MIKE COHN: Absolutely Absolutely Watch the drawbacks to doing that, which is why I don’t want to do that, which is why I want a process where we can put high level, easy to come at, story point numbers– let me back up and put them up there Notice I put the smaller numbers back I want to do that Now here’s why this won’t happen with story points No one’s going to take this and say, well we’ve got 10, 14 story points left Let’s divide them by 35 hours 14 story points divided by 35 hours I don’t even know how to do that They’re obviously different units Let’s not divide them And somebody will say, well how do I get the answer I want? And they’ll say, well you take the 14 story points remaining divided by the three we did and they’ll do the right math So it won’t happen when we have clearly different units Which is why hours are the most troublesome Days are almost as bad because we’re just forcing somebody to multiply by eight before they can make a mistake Yes? AUDIENCE: [INAUDIBLE PHRASE] MIKE COHN: Feedback loops in what sense? Getting better at estimating? AUDIENCE: [INAUDIBLE PHRASE] MIKE COHN: I’m missing the feedback They have new features? AUDIENCE: [INAUDIBLE PHRASE] MIKE COHN: Changes Changes They’d just be new product backlog items, right? We would have to estimate the new product backlog items. And I would encourage that We always want new ideas, right? So, I’m kind of opposed to the idea of change control boards, right? If you can come up with a new idea, let’s put it in All we have to do is have a prioritization process to make sure we’re working on the highest priority things AUDIENCE: [INAUDIBLE PHRASE] MIKE COHN: Correct AUDIENCE: [INAUDIBLE PHRASE] MIKE COHN: Yeah, I won’t talk about it So the question is can we go in and do sensitivity analysis and things like that with this What I’ve done to get around those type of things is I’ve done two point estimating So I’ll say what’s the most likely number for this I’ll say, you know, we could be done in five points or five days, whatever you’re thinking in And as a worst case, it might take 15 days and put some ranges around our estimates Then there’s ways to aggregate those and reduce that down AUDIENCE: [INAUDIBLE] MIKE COHN: Well and sometimes that’s relevant Sometimes that’s relevant So I want to talk about how to combine these into an estimating approach We’re going to talk about a tool technique called, Planning Poker Planning Poker is very loosely based on a technique called Delphi or Wideband Delphi out of the Rand Corporation in the 40’s The idea with Planning Poker is that each estimator is given a deck of cards and the cards have the numbers on them

that we’ve agreed to use as an estimating sequence The estimators talk about the item to be estimated The product owner typically is there, reads it to the team, the team talks about it, and once you think you’ve got an idea of what the answer is you pick the number in your deck OK, I think it’s a 10 I’ve got a 10 selected You think it’s a three Now we all turn our cards over the same time You’re holding up a three I’m holding up a 10 Everybody else on our team is holding up a five Well they want to hear from us Why do I think it’s a 10? So I give my argument You give yours for why it’s a three We talk about it as much as needed Anybody else can talk and argue but we especially want to hear from the outliers Then we pick up our cards We do it again You can bluff if you want Pick the same number Pick a number you want Everybody’s got a number, 1, 2, 3 We turn them over We see if we agree Once we agree, we’re done If we don’t agree, we keep repeating this process until we do agree Here’s an example Suppose you’re holding this set of numbers I want to explain this set of numbers first. This is obviously the Fibonacci sequence– take two numbers, add them together to get the third One plus two equals three Two plus two equals five Eight plus 13 equals 21 But you’ll notice I’ve got a twenty up there Let me explain the 20 but let me back up and start with a different explanation first So I mentioned that I wanted to get good at estimating And I told my teams you have to estimate different ways I had a team come to me and they showed me their estimates They had an Excel spreadsheet They had column a was the estimate Column b was the feature to be delivered And I was just in a bad mood that day I was the VP in this group and the team was coming to me, and I barely even remember what they were working on I had way too many projects going on And I barely remember this, so picture the context You come to this jerk VP, who barely remembers your project, and one of their items was a 17 And I said, that’s not at 17, that’s an 18 I said, in all my years of doing software if I’ve ever seen an 18 in my life that is an 18 And they looked at it and they said, oh you’re right We told Bill– who was conveniently out sick that day– we told Bill to change that We’re not sure why he didn’t but you’re absolutely right, that’s an 18 They left and I said, wow I’ve got a really stupid team It was kind of fun It was kind of fun doing this to the team I just wanted to see how they’d respond Another week or so another team comes to show they’re estimates They had the exact same thing I said I’m going to have fun again I say that’s not a 17, that’s an 18 They said, we don’t use 18 I said, what do you mean you don’t use 18 I said it’s a number It’s between 17 and 19 You have to use it They said no We found out that we shouldn’t use every number We can’t tell the difference between a 17 and an 18 So why let ourselves get in arguments about it’s a 17 No it’s an 18? So we’ve decided to get rid of certain numbers They left and I said, wow I’ve got a really smart team I learned from them This is great That project went on It wasn’t quite done yet It was on schedule, but it wasn’t quite done yet But everybody liked what they were doing The next team decided to start estimating And they said we’d like to do what that team has done I said, you can’t You’ve got to do something different They grumbled and left and came back the next day and said OK, we’ve got it We figured out what we’re going to do differently And what they decided they were going to do differently, was instead of rounding to the nearest number, which is what the first team did, they said we’re going to think of these things as buckets I said, well what does that do? It’s a bucket So what? They should no, no Think about it If it’s a bucket and you have an 18 it won’t fit in a 17 gallon bucket An 18 gallon item has to go into the next biggest bucket That’s kind of good That’ll help this kind of human nature tendency to underestimate We have something big, we put a number on it, but then later when we break it down into the tasks, it explodes into something bigger Well this rounding up idea sounded pretty good– the bucketing idea Let’s try it So they went off and did that project They got about halfway through and it was going really well The next team came to me and said let’s call the experiment off we’ve learned how to estimate I said we’re not even close to done Go find a tweak Go do something different They came back They grumbled and complained They came back the next day and said we’ve got it We’re going to think of them as buckets, as well, but they’re not buckets of water, their buckets of sand I said what does that matter? They said, oh it matters a lot because if we use a 17 we actually can put 18 gallons of sand in a 17 gallon bucket It would be heaping This left them the capability to say things like lion and tiger A lion is just a little bit bigger than a tiger I want to call it the same number I said OK And I said, well have you defined how much is too big, that when does it roll up? They said no, no, no That’ll be decided in the context of each item It’s just that we feel that there’s room to go a little bigger

I said great Go that way So that team estimated that way and that turned out to be fairly successful The next team was doing– might not have been the next team It might have been a couple teams later– but one team was doing something and by then we settled on the Fibonacci sequence as a good set of numbers And this next team was explaining their estimate for their product owner And they said, OK this one here is a 21 and they talk about it Their product owner said, I’m impressed I’m really impressed that you know that’s a 21, not a 20 You must really know something to call it a 21 And the team said, Oh, no, no, no We use the Fibonacci sequence This is all back before The Divinci Code was out, right? It reminded us all what the Fibonacci sequence was, and the product owner said, the Fiba- what? And the team said, oh we’ll just call it a 20 And from there on we decided It looks like you’ve got some very precise knowledge you don’t have. This one’s a 34 That would be the next number So after while we’d deviate from the Fibonacci sequence So we use up to 13, and then 20, 40 and 100 So I do call this the “Hanks” sequence So in an estimating meeting we’re doing this and we get the first four estimators– a 3, 8, a 2, and a 5 I’d want to hear from Vidim why an eight and why a two? And Vidim says this is going to be hard to test. We’ve never tested anything like this This will be challenging And why a two? Well Vidim is right This will be harder to test than I was thinking My two is wrong I’m going to come up on the two But this will not be that hard to program I’ve done this before I did this on my last project No, the code’s not reusable but it’s not going to be that hard to do I think it’s fairly simple Everybody picks up their cards, they re- estimate, and we see that we get three fives and an eight At this point, I would ask Chris, give us an impassioned plea for eight The rest of us have fives, you tell us why this should be an eight This is not a vote We don’t go with majority rules We hired Susan, Vidim, and Ann yesterday Today’s their first day on the job Chris is the founding architect of the company He has a little more credibility Let’s go with whatever he or she says So this is not a vote It’s not a democracy We drive this until we get consensus I want to hear everybody say, I agree That’s the one number to put on here I’ve always asked, well what if that doesn’t happen? I don’t know I’ve never seen it happen Every team will get to consensus They may have some items that get kicked out Product owner, go get some more information We can’t tell That maybe if the case here where Chris is holding up a 100 We don’t know how we’re going to do this, we don’t have enough answers, go get some more information, and we’ll estimate this in a couple days But normally you will get the consensus The consensus may be false What I mean by that is Chris may go a couple more rounds and just say I fold, go with the five, but please–