Back to Blog
Management Frameworks

Agile Case Study (5/5): Closing the First Release

By  Jose Barato

May 26, 2021

22 minutes read

This is the fifth and last post on the chapter 23 of the book Agile Estimating and Planning, by Mike Cohn. In the previous post the team was about to start the first 2 week iteration, planned with 4 stories and 18 points. They also forecasted a project duration between 12 an 20 weeks. In this post we will read this:

  1. The team shows the outcomes of the 1st iteration (they complete 16 points out of 18) and they plan the 2nd iteration with 3 stories (18 points).
  2. They complete the 2nd iteration completely and they plan the 3rd one having into account 2 new exciter features, adding 30 and 35 points respectively. If they want to keep duration from 12-20 weeks, they have not to include an exciter feature of 30 points. Before starting 3rd iteration they have 133 story points (at the beginning they had 132).
  3. Finally, we read about the last iteration review. The project ended up in 22 weeks (2 weeks later compared to initial estimate, but with more value to the business).

This case study shows how agile methods can get software projects controlled.

2 Weeks later...

“I wish we could have finished everything. Still, I’m impressed with what we’ve got,” Sasha said to the others while waiting for the meeting to begin. “It’s unfortunate that you had to drop the ‘As a player, I want the computer to recognize a winning shape’ story. But I really needed your help on the move engine code, Sasha. Thanks,” Allan thanked the other programmer. “Not a problem, Allan. We’re all in this together and being able to demonstrate an engine that makes rings and bridges is more important,” Sasha said. “So, are we ready for the demo?” Frank asked as he walked into the room promptly at 9:00. He laid a box of bagels on the large table. “Absolutely,” Sasha said. “Do you know if any of the other executives or other project members are coming to watch the iteration review?” “I know Phil and Laura said they’d be here,” Frank said, referring to the Chief Executive Officer and the Chief Financial Officer. “And a couple of the engineers on other teams said they’d be here as well. They’re curious about our new agile development process,” Allan said. Over the course of the next half hour the team showed what they’d completed. They demonstrated the game engine making rings and bridges. They showed how two humans could play against each other, trading the keyboard between them to make moves. They’d even hooked the two pieces of functionality together so that a human could play against the computer. The human usually won but that wasn’t surprising against the easy move engine. The graphics were rudimentary, just Rose’s quick illustrations. But Phil, the CEO, was ecstatic to see that after only two weeks some small part of the game had been developed and was functional. He was even more pleased when Prasad handed him the keyboard and he was able to play the game himself, barely winning the new game. “This is fantastic, everyone,” Phil announced, rising from his seat. “Every two weeks you’ll show more?” “Absolutely. Every other week right here at 9:00,” Sasha said. “I’ll be back next time. I’m going to put these meetings on my schedule. This 30 minutes of seeing the actual software working is more useful than any status report I ever read. And how’s your schedule? Can I ask that now?” “You can,” Frank said. “Right now we’re sticking by our estimate of 12 to 20 weeks. We’re going to plan the next iteration after this meeting. We’ll know more in two to four weeks. At that time we should be able to commit to a schedule that we can tell publishers about.” “Great,” Phil replied. He and Laura left the conference room. So, too, did the half dozen others who had come to see the project.

Planning The Second Iteration

“If I recall correctly,” Frank said, “we planned a velocity of 18 story points.” “Correct,” said Delaney. “But we couldn’t finish the story about recognizing when the game had been won. So our velocity is less than 18. Do we give ourselves 1 point for doing half of that story?” Frank asked. “No,” Carlos said. “In general we want to apply an all-or-nothing approach to the story points. It’s too hard to give ourselves partial credit because we really don’t know how much work is left. Rather than guessing, we don’t get any credit until we get it all.” “So our velocity was 16 because we originally estimated the unfinished story as 2 points,” Rose said. “We have to plan on the project taking longer. Right?” Frank asked. “We’re still within the 12–20 weeks you told the executives,” Sasha said. “We started with 132 story points. We finished 16 so we have 116 left. Dividing 116 by 16, our velocity, gives us 7.25. To be safe, let’s call that 8 more two-week iterations. Including the two weeks we just spent, that’s a total of 18 weeks.” “That’s two weeks longer than four months, Frank. Does that matter?” Prasad asked. “Not really. The release isn’t tied to any specific date. We just want it out as soon as possible,” Frank said. “I haven’t had a chance to tell you this yet. We still haven’t finalized the contract for Deep Black & White with the Japanese publisher. They still want to publish the game. And we’re this close to signing the final contract.” He held his fingers millimeters apart. “But they don’t want to release it until they have time to gear up a big marketing campaign with ads in all the major magazines. That’s probably four months. And the way that works, we won’t see any royalties until the quarter after the game is released. We could be looking at eight months before Deep Black & White brings any real money in the door. Ideally, Havannah could be a quick win for us. Not on the scale of Deep Black & White. But since it won’t need the marketing ramp-up, the revenue will be more immediate.” “And, Frank, I’m sorry but because I had to do more testing than I expected, I didn’t do much product research during this iteration,” Delaney said. “OK. That’s fine but let’s make that a priority this time, even if it means you test less,” Frank said. “So, Frank, what would you most like to see finished in this next iteration?” The team planned their second iteration in the same way they’d planned their first. When finished they had committed to completing these user stories summing 18 points.
  1. As a player, I want the computer to recognize a winning shape (2)
  2. As a player, I can play against a weak engine that recognizes forks (8)
  3. As a player, I’d like to be able to choose between a wooden board and pieces and a metal board and pieces (8)
“So, if our velocity was 16 in the last iteration, why are we planning for 18 this time?” Frank asked. “We use velocity as a guide for measuring our progress against the release plan,” Carlos said. “We don’t automatically plan each new iteration to have exactly the velocity of the last iteration or the average velocity of the last few iterations. We plan each iteration based on how much work we can commit to. We only had up the points after we’ve figured out what we can commit to. When we do, the velocity should be somewhere close to our historical average, but it’s going to vary.” “Why?” “Two primary reasons,” Sasah began. “First, we’re making more effective use of Rose this time. She’s primarily an artist, not a tester. It was great that she helped with so much testing last time but this time she’s going to draw the art for the wooden and metal boards and pieces. I’ll do the coding to let a user choose between them and Prasad will probably be the one to test that story. But, other than that, most of that 8-point story comes from Rose. Second, we’ve got a bit of a head start on the story about recognizing a winning shape. We started it last iteration but didn’t get far. I feel pretty good about the stories we selected. Even though it’s 18 points again, it added up to 15 total hours less than the last iteration.” “I’m convinced,” Frank said. “Let’s do it.”

Two Weeks Later...

At the next iteration review the team again demonstrated good progress. They had completed each of the stories they’d planned. And Delaney had time to complete her product research. The executives, Phil and Laura, attended again. Because the news had spread about how much had been accomplished in their first two weeks, even more people from other teams attended the second review. “The game is coming together. I can’t believe your move engine can already beat me,” Phil said. “And, Rose, those first two boards look fantastic. Are we still looking at the same schedule? That would be 8–18 weeks from now.” “Our rate of progress is right about where we expected it to be,” Frank said. “So, we’re good there. Delaney did some more product research during this iteration and she’s found some interesting features we may need to include. I’ll make the call on that but I need to hear the team’s view first. I may suggest changing the schedule. We’re confident the new features will bring in more money. Now I need to weigh that against the extra development cost and the extra time.” “I like the part about bringing in more revenue,” Laura, the Chief Financial Officer said. “Me, too,” Phil agreed. “But I’m not wild about a possible slip already.” “It wouldn’t be a slip, Phil,” Frank said. “If you want to deliver the product in the originally planned 12–20 weeks, we’re feeling pretty good about that. What we’re going to discuss after this meeting is how much longer it would take to create a more valuable game. And then we’ll see if there are features we can drop that would get us that more valuable game in close to the same 20 weeks we gave as our upper end.” “I’m always interested in more valuable games. Let me know what you recommend,” Phil said as he stood and left the conference room. The rest of the guests left as well, leaving only the project team.

Reviewing The Release Plan

“Frank has already told you that I found some new exciters,” Delaney said. “A couple of you heard me talking on the phone with potential buyers about these. The good news is that our target audience really wants two features we haven’t thought about. First, they want the computer player to have different personalities who each play the game differently. One character may be very aggressive, another conservative. A character that a lot of people asked for is one who taunts them. It says mean things after a bad move, laughs at them when they lose. Stuff like that.” “That does sound fun,” Allan said. “It’ll change the move engine, but probably not much. It’s already configurable to allow me to tune it for more aggressive or more conservative play.” “That’s the first new feature,” Delaney continued. “The second is online play. People want to play this game online against other humans.” “That’s huge,” Sasha said. “We’d need to get space in a data center, acquire the hardware, hire the staff.” “We could partner for all that,” Frank said. “It’s not out of the question but you’re right that’s it’s big. Delaney, do you have any sense for which feature is more important?” “The characters. Good characters would be a big exciter. That is a feature that people really like but that they don’t expect,” Delaney said. “Delaney, you just told us that we’ve got scope creep. And we were already at the upper end of our 12–20 week estimate,” Prasad said. “Why did you say you have ‘good news’? Do you have something else to tell us?”
“No, that’s it, Prasad,” Delaney said. “But this is good news. We just found two new features that could help us sell more units and at a higher price. Yeah, if we do them the schedule will change. But that’s a good problem to have. We can ship the product based on today’s schedule and make a certain amount of money. Or we can change the schedule and plan on making more money. Plans are always point in time predictions about the future. We just found a better future to aim at. I’ve already written the user stories for both features. We should estimate them and decide if we want to change the plan to include either or both.” “Let’s do it,” said Frank. “I’m completely willing to change the date of the project if we can make enough more money and it’s not a big delay. It sounds like these features could be very lucrative.” “But, Delaney, why didn’t the questionnaire you sent out identify these features?” Prasad said.
“Questionnaires are good for prioritizing what we already know,” Delaney said. “They obviously aren’t good for discovering features we don’t know about. Some of the questionnaires we got back did mention characters and online play. But not enough that I thought there was a trend. I spent this past iteration talking to more likely buyers to confirm that these are good ideas and adjusting my financial models to show how much more revenue these features are likely to bring in.” “I like the sound of that,” Allan said. “If the game makes more money, maybe Frank will take us somewhere more expensive than Fatburger to celebrate this release.” “I liked burgers to celebrate the release of Deep Black & White,” Prasad said. “Not me. After this one, I say Frank springs for sushi,” Rose said. The team estimated the new stories Delaney had written. In doing so, they split some of her original stories and combined others. When done, the stories for online play added to thirty story points. They estimated creating the first character as fifteen story points. Each character after that would be five additional points. The team discussed it and agreed that five characters would be a reasonable number. This meant an additional 35 story points for developing the characters.
  1. As player, I’d like to play this game online against other humans (30)
  2. As player, I’d like the computer player to have different personalities who each play the game differently: character 1 (15)
  3. As player, I’d like the computer player to have different personalities who each play the game differently: character 2 (5)
  4. As player, I’d like the computer player to have different personalities who each play the game differently: character 3 (5)
  5. As player, I’d like the computer player to have different personalities who each play the game differently: character 4 (5)
  6. As player, I’d like the computer player to have different personalities who each play the game differently: character 5 (5)
Since Delaney’s financial forecasts showed that characters were likely to generate more revenue than online play, everyone agreed that they should adjust their plan to include characters but not to include an online version. “In the first two iterations we finished 16 and then 18 points. We just added 35 points. We’re going backwards!” Allan joked. “You’re right, Allan,” Carlos said, as he reached for a calculator. “We’ve got 133 points. With our current average velocity of 17 points per iteration that’s 7.8 more iterations.” “Eight more iterations plus the two we’ve already done and we finish in ten iterations,” Frank said. “It’s cutting it awfully close, Frank,” Sasha said. “Yes, it seems like we’ll finish in ten iterations. It’s a good thing our original schedule was a range and that we didn’t just tell people ‘eight iterations.’ However, if anything goes wrong—if a story is bigger than we think or if we slow down or if someone gets sick—we could take longer than eight more iterations.” “Are you saying we should change the schedule?” Frank asked. “Not necessarily,” Carlos answered. “If we want to keep the schedule at ten iterations, though, we should make sure we’re willing to drop some low priority stories if we start to run out of time. If you’re not willing to do that, we should convey the expectation that it may take longer than ten iterations.” “I can probably drop a few stories if necessary,” Frank said. “Good. Remember how I told you it’s important to know the stories at the top of your priority list and those at the bottom? This is a time when it’s important to know those at the bottom. I’ll feel more comfortable keeping expectations at no more than ten iterations if you can point out at least 17 points of stories that you could live without if it meant going long than ten iterations,” Carlos said. “Here are some stories I could live without:”
  1. As a new player, I’d like to be warned after making a horrible move and be given the chance to take it back (8 points)
  2. As a new player, I want to be able to view an interactive tutorial for the game (8 points)
  3. As a player, I want the system to play background music (5 points)
  4. As a player, I can select the background music played by the system (1 point)
“So if you start to run out of time by the tenth iteration, drop one or more of those,” Carlos said. “And if we’re on schedule and chose not to implement those stories we could release at least an iteration sooner, right? When would we release if we left out those stories?” Frank asked. “We’ve got 133 points planned. You’re willing to drop 22 points. That gives us 111,” Carlos said as he reached for the calculator. “With a velocity of 17, you need seven more iterations. You could be done in 18 instead of 20 weeks.” “Let’s keep those stories in the plan for now. I’ll drop them if we need to but they’re probably worth two additional weeks,” Frank said. “I want to go run the new plan by Phil since he asked to hear what we decided. Does anyone want to join me?” Delaney and Allan both said they did. It wasn’t a surprise that Delaney did. As the analyst, Delaney was often involved in product and schedule decisions. Frank was surprised by Allan’s interest, though. Normally Allan, one of the most hardcore technical people in all of Bomb Shelter wanted nothing to do with business discussions. He’d really engaged in those discussions, though, since starting this project four weeks ago. Frank appreciated that. It was leading to better product decisions all around. “I’m going up there now,” Frank said. “Let’s do it.” “Shouldn’t we prepare anything for Phil first?” Allan asked. “Nope. He’s got a whiteboard. We’ll just draw on that.”

Presenting The Revised Plan

“Hi, Phil. Do you still want to hear what we’ve decided about those new features in Havannah?” Frank asked. “Absolutely. These are the features Delaney mentioned this morning?” “Yes.” “I’ve got about 15 minutes now. Will that work.” “It should be plenty,” Frank said. He then asked Delaney to explain how her research had led to the discovery of the new ideas for online play and characters. “Phil, do you remember a few weeks ago when I told you how we are estimating in story points?” “Sure.” “And that story points are a measure of size. We started out with 132 story points. In the first iteration we finished 16. In the second we finished 18. Allan, can you draw that?” Frank asked, pointing to an empty space on Phil’s white board. Allan went to the white board and drew figure below:
“What you see here,” Frank continued, “is what we call a release burndown chart. From looking at it you can see our rate of progress. After two iterations we’re down 34 points. If you extended a trend line from this data you’d see that we are on pace to finish during the eighth iteration. That’s 16 weeks total.” While Frank was speaking, Allan drew the trendline shown in figure below:
“The whole Havannah team thinks that adding the characters Delaney identified is a good idea.” “How long will that take?” Phil asked. “We think we need five characters. It’s a total of 35 points,” Frank said. “That means that our burndown actually moves back up to here,” Allan said. He drew a vertical line up to 133 points on the white board, as shown in figure below:
“Hmm, that doesn’t look so good. After two iterations, you’re no better off than where you started.” “We’re actually much better off. We’ve made four good weeks of progress,” Allan said, pointing to the progress on the chart. “We’ve eliminated some of the riskiest parts of the project because I’ve demonstrated how I’m pursuing the move engine. I’ve only done the weak game engine, but I’m starting to see how we’ll write the strong engine. We’ve also learned a lot. We learned about a new feature that Delaney says is ‘exciting’ to our target audience. Normally we’d find that out much later. If I look at where we’re at, yes we still have as much work to do as we thought four weeks ago but now I’m more confident in the plan than I was four weeks ago. And every time we put a new data point on this burndown chart, I get more confident. Even if not every new point is good news. It will be news I can trust about the progress we’re making.” Allan paused. “We’re much better off than we were four weeks ago.” “So, how long will the project take if we add the characters?” Phil asked. Frank was surprised, but pleasantly, that the question had been directed at Allan. “Probably eight more iterations, or a total of twenty weeks. We’re completing 17 points of work per iteration. We call that our velocity. If we draw a trendline showing 17 points of progress per iteration, we finish in eight more iterations.” Allan drew the trendline shown in figure below:
“That cuts it pretty close so we’ve identified a few low priority pieces we could drop if we start to get behind or if we decide to release after 18 weeks instead of 20.” “And the game would sell more or be at a higher price?” Phil asked. “Yes, I’m certain of that,” Frank said. “I want to do a bit more research and modeling to determine if we should sell more copies at the current price or if we should increase the price by $5.” “I’m sold. Thanks for showing this to me,” Phil said. Allan started to erase the chart he’d drawn. “Don’t. Leave that up there,” Phil said. “I want to update that every two weeks.” “Thanks for your time, Phil,” Frank said as he, Delaney, and Allan left the office.

Eighteen Weeks Later...

“Great job, team. The game looks wonderful,” Frank said at the end of the final iteration review meeting. “I don’t see anything that’s missing. The engines play very well. And, Rose, I love the characters. The way that each character is tied to an engine is great. Players will select a character to play against rather than a level to play at. I admit I was skeptical of cartoonish characters rather than real characters at first, but they’re great. I like how the pirate taunts the player and how you hooked a friendly mermaid to the easy move engine. Her dialogue is going to definitely help new players learn the game. I think we’re ready to ship.” “I admit the game is great but I still don’t like the feeling that we’re late. We originally thought we’d be done in just over 14 weeks. But we told everyone 12– 20. Now we’ve slipped two weeks past that,” Prasad said. “Yes, and here we are done in 22 weeks,” Sasha said. “We never said we’d be done in 14 weeks. After the first iteration planning planning, we told everyone 12–20 weeks and that a more accurate estimate would come after six weeks.” “And, Prasad, keep in mind that we could have shipped two weeks ago if we’d wanted,” Frank said. “We chose to take time to incorporate additional changes into the characters based on the positive responses we were getting from beta testers. They loved the characters but gave us some great suggestions we wanted to incorporate. We made the pirate taunt a little more often based on their feedback. And we gave the rap singer more dialogue. Sure, the initial schedule was wrong but so was the product design. That game wouldn’t have made us anywhere near the return that this one will. You’ve played the new version. You know how much better it is now, only two weeks off our initial high-end estimate. If we could have released a version 1.0 without characters and then added characters later, I would have done that. But we can’t do that in our industry. I’m ecstatic with the product. It’s more than we expected and only two weeks later than an estimate given at the start of the project.” “And, don’t forget, that Deep Black & White was six months late, not two weeks.” “So, Frank, are you buying us burgers again to celebrate this release?” “No, this time I’m buying wherever you want.”
THE END   previous first

Related posts