Deliver faster – stop estimating

Software organizations have been creating estimates for more than 60 years. The industry is still not very good at it. Just by the law of large numbers we should have at least as many projects finishing early as we have projects that finish late, but that is not the case. 80% of projects finish late, and when they do finish, they only have 50% of the promised capability.

Is it eternal optimism?

It can’t be. After 60 years of being constantly wrong adjustments to estimates would be common, and the rate of success would improve. There is something systemic in the industry causing this, and the commonly implemented Agile Scrum planning and estimation are contributing to it rather than mitigating it.

SpeedShift^TM

At SphereOI we have incorporated Lean principles, Kanban, and software factory approaches to create our SpeedShift^TM process to reliably deliver software and AI systems on time AND 3X faster than most organizations.

The flaw in Scrum planning and estimation

Industry estimates and studies performed by Goldratt* show that a schedule at 80% confidence contains 100-200% safety time. OK, so if EVERY estimate is doubled or tripled, surely the project will complete at least on time if not early, right?

No, even with this practice (which has been in place for decades) projects are still late and do not deliver all the capabilities. Here is why:

Parkinson’s law: Work expands to fill the time allocated to it.

Arbitrary boundaries force people to add “safety” into estimates and schedule. These are compounded at every level. Once the schedule, including the safety, is in place, that time is used whether it is needed or not. 

An insidious example

Think of it this way – no one ever gives scheduled time back. So, let’s say I have a piece of work that I think will take me 3 days to finish. OK good, I add ½ day because I know I will get pulled off to do something else, or something unexpected will come up – so, 3.5 days. My tech Lead sees this estimate and wants to be sure we are not late, so they add another ½ day and we are up to 4 days. When we get to a roll-up level, the project lead adds a 10% safety buffer to everything, and now we are at 4.5 days. I start the work first thing on a Monday morning, and I was correct – it only takes me 3 days to finish the work – but the schedule has 4.5 days on it, so I have a little more time to tweak and test and make it that much better. I finish ahead of schedule on Friday morning at 10:00am – but it is a little late in the week, so I decide to do some clean-up work on my environment, review some code, and make some comments on stories in the backlog, and I still delivered on time. So, a 3-day task just took 5 days, and we claim we are “on schedule.”

But wait, what if my next task runs long? If I had stopped at 3 days on my last task, I would have two days left in surplus to apply to the next task – but given human nature, I used that time on something that is likely less important because the schedule said I had time to do it.

Be careful what you measure

Bad priorities and arbitrary boundaries lead to significant waste. The sprint and release boundaries are particularly devious. We have seen organizations that measure capacity and progress using story points. That is fine. Let’s say the team has a capacity of 50 points per sprint – so they plan for 50 points to be worked. By some miracle one of the team members finishes a day and a half early. They pick up 5 points from the prioritized backlog and start to work. They get some of it done, but not the entire story. Metrics now show only 50/55 points completed and the team is penalized. No kidding, they did extra work and the metrics got worse! Many organizations make it so hard to change even the Sprint plan that teams never pull in more work AND they are very conservative in their estimates to avoid missing targets.

Simplify, simplify, simplify

At SphereOI we take this to heart in everything we do, from user interfaces to software architectures, to network architectures, and even in system of systems problems. Here are the starting steps to make your teams deliver faster and more reliably:

1. Define a Strong Center – the reason the project deserves to exist
2. Break work into manageable capabilities with observable outputs
3. Prioritize everything
4. Only build what is essential to support the Strong Center
5. Strive for a continuous, stable flow of capabilities

This is not easy, but when we do these steps our cost and schedule become predictable, and our teams are able to absorb changes in direction and take unexpected difficulties in stride. By eliminating the boundaries, the estimates, and the wasted effort outside the prioritized list, we can complete projects in 1/3 the time.

*Goldratt, Eliyahu M., Cox, Jeff. The Goal; A Process of Ongoing Improvement. North River Press, June 2014.