No estimates!
…So you are to manage some software creation. Then you must understand you cannot simply employ the same techniques used to manage other domains. Software creation is strange – many counterintuitive truths about it. You will fail unless you learn about these weird realities.
This series contains these posts:
- How to hire a development team
- Counterintuitive facts of software development
- No estimates
- On the use of harmful tools
Estimation is impossible
From other fields came the notion that deadlines can be set. But one of the most striking bizarre facts about software creation is this:
It is impossible to anticipate when the thing will be done with any reasonable amount of certainty.
This was gradually discovered during the history of software development. It is impossible to estimate development effort, because in programming, difficulty is accidental, it is not predictable. One does not know what the trouble is going to be.
If you cook pierogi once, you have some idea of how long it will take you next time. But new software is like writing a new book or researching new tech. Treating it like pierogi is so wrong, it is offensive. Only in very few cases is software so repetitive, and in those cases, it is not very valuable. Valuable software does something new enough that you can't estimate it.
In such cases, I don't know how long it's going to take, because I haven't written it yet.
History
Since the 70s, devs marveled at how much their estimates were off. So they concluded: "we should improve our estimation skills. Learn to estimate better through deeper thinking". Research on this was done in the 80s, and they came up with better techniques. Then they saw the results were equally off. It wasn't until the turn of the century that a few enlightened minds decided estimation was a waste of time. The result of the estimation was garbage, so they wouldn't do it anymore. They wouldn't lie to management.
This idea only began to spread about 10 years ago. That's the #NoEstimates movement.
Rubbish
The last time I estimated a story with Raphael, we spent 3 hours and came to the same conclusion, so we felt good about it, although we were extremely tired. But then a better alternative came up and plans changed, so the 6 man-hours were thrown away. The estimate would need to be redone, and in all that time, nothing useful was being done for the user/client.
The 6 man-hours were pure waste: doing the estimation hurt the speed of the team. The estimation was probably incorrect even if we felt good about it, and even if it weren't so, it had no value once we had a better idea and changed our plans. If you want your team to be fast in delivering working software, requiring estimates is a great way to ensure they will be slow.
So, you see, in software, estimation doesn't work, its output is garbage and management decisions, business decisions, must not be based on such poor quality information.
Developers always knew estimates are rubbish, but managers always insisted they need it. Managers won, but nothing could be done about the fact that, in software, estimates are garbage.
Software creation is an interaction
When we estimate, we sort of write the software in our heads. We start scribbling that, to finish this feature, we need to write these new database tables, this database migration, these business rules, these API methods, use this or that library, add this and that to the GUI etc.
But that's not how it actually works. When we sit down to do the actual work, we are faced with the entire iceberg, not just the tip we could see.
Initial estimates often assume a "greenfield" scenario, but in reality, we must integrate with legacy parts of the system, poorly documented code, or unexpected architectural bottlenecks.
We make bets. After some software is written, we start to realize whether our bets were off. Only through building can we understand the true problem and discover better solutions. We are now interacting with all sorts of unanticipated forces:
- Requirements were vague, evolving, or misunderstood.
- Real needs become clearer only as the product takes shape.
- Underdeveloped designs.
- Edge cases, usability concerns, or integration issues emerge.
- Unforeseen technical debt.
- Models may be different than how we remembered them.
- Libraries are at least partially unknown.
- The compiler.
- Versions.
- Discovery of steps we didn't anticipate we'd have to do.
- A developer doesn't just develop. She gets sick, she must go into meetings, she needs to pause to fix a bug, she needs to do some planning, she needs to answer e-mails and someone else's questions, she needs to research something before the next sprint... None of this gets factored into an estimate.
In short, writing actual software is a struggle against realities that were completely unknown at the time of estimation.
Implementation is discovery, not execution. This is why Agile emphasizes iterative progress, continuous feedback, and adapting to change.
Managers do not understand the word "estimate"
When you use estimates, you set yourself up for this eternally repeating conversation:
"The feature is not ready??? You said it would only take a week."
"That's not what I said. I gave you an ESTIMATE."
"That's bullshit!!!"
"You got that right!!!"
"How can I meet my goals like this???"
"What do I know about YOUR job??? My plate is already full of this letter soup!"
What to do
Developers give estimates just so managers will stop pestering them and go away. That is an irresponsible thing to do. The only responsible thing is to deny estimation.
However, businessmen still need information to be able to make business decisions. There's an event later this year, that is an important business opportunity, and we need to be able to predict what can be ready by then.
That is why people have been studying how to make management decisions in the absence of estimates. Based on other criteria.
Not convinced?
I do not expect you are convinced. I was only convinced after weeks of consideration. But this 37-minute presentation by Allen Holub is what finally convinced me.
After 15 minutes, he starts to show how to manage a software project in the absence of estimates, including how to count stories to make graphs and calculate timetables. After 31 minutes he recommends Story Mapping to organize the backlog and prioritize features for the next release.
Other notable authors about NoEstimates are Woody Zuill and Vasco Duarte.