Ping me on linked-in if you are interested in the beta release of pivotaltracker's new prediction system. Also, a huge thanks to Meenu Sharma from VMWare who is one of a few product managers who is great at sourcing amazing ideas from the community.
In this blog, let's go over two myths that I run into with Agile Estimation and Planning. As pivotaltracker.com gears up to launch the first predictive scheduling system, there are rules that should be followed to make it more accurate. Here are two myths I run into on nearly every team doing Agile/Scrum
Story Points Myth - Every story and bug we create, we must assign points to
Completion Myth - We must shoot for 100% completion of every iteration
Story Points Myth
When we first plan a project or plan out the next 3 months, every story we put in during the planning phase gets assigned points. To have accurate schedule predictions based on velocity it is important to follow a few rules and I will explain why below with example scenarios to make it crystal clear.
After, we lock a plan in, if a developer realizes there is more work and wants to add stories, all these stories get 0 points.
If a product manager asks the team for a 'new feature', that story gets assigned points.
If a bug comes in, that bug gets 0 points
Let’s draw out 2 scenarios where we plan out 100 points to understand benefit/drawbacks of points for late-added stories. The first scenario is how nearly every team does it today and why they can't use velocity to predict the future. Keep in mind that nothing is perfect but if you can draw an ideal scenario as the 'average project', you want to stick following that ideal scenario so your averages work out closer for schedule prediction.
Scenario 1 - Traditional Method
Let's say our previous project was 100 points, but we added 50 points of unaccounted for work. The total points to complete during the project is 150 points. Let’s say our velocity was 15 points/iteration.
Now, let's plan out our next project. After planning, let's say our next project was also 100 points to make the math easy. In this case, 150/15 is 10 iterations. The predictive model says we will be done in 100/15 = 6.6 iterations but it does not take into account there will be an ~equal amount of unaccounted for work. In reality 'equal' is anywhere from 30% more to 30% less work but is closer to the average we see in 'additional work' in every software project. This means we predicted the wrong number. In fact, let's assert that it just so happens, in this second project there was also 50 points of additional work. Let's also assume we averaged the same velocity of 15 points/iteration. This means, we took 10 iterations to complete the project not the 6.6 iterations predicted. We were way off by 34%. This is again an ideal scenario with perfect numbers where project 2 matches project 1 exactly on purpose for a better case study. An ideal scenario should have yielded a 100% success on prediction and did not. In fact, it painted a picture that was far too optimistic. Let's take another shot and let's again use ideal numbers.
Scenario 2 - Better Estimation
To keep things 1 to 1 with our our previous project, let's again say we just finished a project that was 100 points, and also added 50 points during the project except this time, the added work was all weighted at 0 points. Our velocity here ended up being 10 points per iteration(since scenario 1 took 10 iterations, this scenario to match also took 10 iterations).
Now, we go to start the next project and it is also 100 points. Our prediction then is 100/10=10 iterations. This prediction is now 100% match because we are taking late adds as a constant in the process and applying late adds to slow down the velocity to paint the truth each time we plan out a project. This paints a more accurate(albeit slower) velocity so predicts 10 iterations instead of 6.6 iterations. Based on the law of averages, this methodology should yield much better predictive results every time. As you begin to use the new pivotaltracker.com feature, keep these rules in mind. Also, another very important factor is which phase a team is in regarding team formation process
Forming - least accurate predictions
Storming
Norming
Performing - most accurate predictions
Another factor is brand new projects vs. existing projects that you are adding too. Brand new projects have many more unknowns so even with the above process, you are at a disadvantage so you may want to adjust predictions by buffering 30% in the case of brand new project or a team that is 'forming' vs. a 'performing' team.
The Transition
Let me take a wild guess in that your team is using scenario 1? If so, you need a transition plan as the team's velocity as we point out is inflated. I usually multiple the average velocity by 0.7 in this case going into the future until we get around 4 iterations complete with scenario 2 methodology. Please let me know how it works out.
One last way to think of each story and it's points (5 points or 3 points), is that they are simply gates for the team to get through. Each gate scores you points. You may discover additional work you want to track in a story, but those are not the original gates so you get no credit for those and only get credit for the gates you pass through. One real issue of the gantt chart was when one bar in the gantt chart was extended, all the other ones are collapsed but the real situation is
If software developers missed stories for the first bar in a gantt chart, they typically missed an equivalent amount of stories in the future bars meaning all bars need to be redrawn to accurately reflect the truth
Completion Myth
We were just talking about 'averages' above. I typically want 50% of the iterations to complete the entire iteration and the other 50% I want to barely fail completing the iteration. Let's look at two extremes. The first will be a team that 100% completes vvery iteration on-time. The second will be a team that fails every iteration(or 0% success).
Let's look at an ideal team that is completing 100% every iteration first. In this scenario, they not only complete the iteration but then start to grab from the next iteration before the iteration is complete. After all, they are not just going to stop working when the iteration is done. In this case though, the team is underplanning. Typically 2 or 3 days before the iteration, all work will be in-flight and one developer will finish his work and then should
check and see there is no work for the iteration
ask teammates if there is anything he can help to close the iteration
Pull from the next iteration
If we are hitting every iteration 100%, you will see really a lack of drive in the team as the goals are too easy. The planning is under-planned. This is not a scenario you actually want.
Next, let's look at a team that fails every iteration (0% success) which is many teams out there. In this case, the team keeps biting off more than they can chew. When the end of the iteration comes, teams like these are typically not even close to finish so again, they lose any last iteration drive to try to close out the iteration.
If you are in the team of 0% success, start with a real goal of trying to just succeed in 1 iteration by planning better. Bite off less work and push back on management if they are pressing for more work. It actually makes teams faster as when you are inches from the finish line at the end of the iteration, we (humans) are all motivated much more by being so close. If we underplan or overplan, then we just tend to work at the same velocity that we worked at all iteration.
Happy Scheduling!!!