The Illusionary Premises of Software Estimation
When I left the dentist’s office today I was asked to make new appointments, one for tooth cleaning and another one to get a small filling. The dentist handed me a small note to show to the receptionist. It had scribbled on it the estimated duration of the session to make the filling: 30 minutes.
How come the dentist was able to make this estimation? How come the receptionist also knew how much time the tooth cleaning would take?
The circumstances for my treatments are so clear and similar that it’s a no brainer for them to predict how long they are going to take from past experience. That’s good craftsmanship, I’d say.
Unfortunately software developers are no good craftsmen in that regard. Their estimations are usually pretty bad, i.e. far from reality. The deviation is not maybe 5% but 25% or 50% or even 200%. That’s a ridiculous deviation.
Still, though, software developers are regularly asked for more of the same misleading estimations. Crazy, isn’t it?
But maybe the deviation is curable? Maybe software developers could learn to do better estimations?
Unfortunately I doubt it. After todays experience at the dentist’s office even more. There is a huge difference between dental craftsmen and software craftsmen. And these differences are very fundamental.
Alas, the people asking software developers for estimations are not aware of these differences. They view software developers as craftsmen (or engineers) who are capable by definition to do pretty accurate estimations.
This view, though, is wrong. It’s suffering from illusionary premises. People holding it cannot see clearly a software developer’s reality.
The Illusion of Control
First and foremost estimations like the dentist’s office made require control. You need control over material, tools, circumstances, situation in order to accomplish the work according to an estimation.
If I asked you how long it would take you to cook dinner and you said it would take you 60 minutes, then this is and estimation under some implicit preconditions. You’d expect to have a material and tools handy and in good shape. You’d also expect to not be disturbed.
But what if you started cooking and discovered the knifes to cut the meat were blunt? What if your friend showed up for a chat? What if some herbs were missing? What if I came in after some 20 minutes and asked you to change one of the side dishes?
You’d loose control. You’d deviate from the estimated time. Sharpening the knife would take 3 minutes, even telling your friend you have no time for a chat would take 2 minutes, running to the grocery store to fetch the herbs and the vegetables for the new side dish would take 10 minutes, doing the new side dish would take 5 minutes (not to mention wasting the original side dish). All in all you’d be 20 minutes or 33% over schedule.
And this is happening all the time to software developers. The do not really control material, tools, circumstance, or situation in a way the imagine during estimation. Tools are not working as expected, prerequisites are missing, interruptions are frequent, changes to the original requirements creep in.
It’s illusory even delusional to assume control over software development beyond simple tasks and/or short tasks.
The dentist has much, much more control. He sees clearly what has to be done; that’s why he did an x-ray and did an assessment of my dental situation first. His materials and tools are in pristine order. And except for rare cases he will not be interrupted during a treatment session.
The Illusion of Experience
In addition to control the dentist possesses truly vast experience. Not only does he know exactly how long certain treatments take him, he also can fall back on numbers from decades, even centuries of his trade. Especially in the past years all health professionals have come under increased pressure to become more efficient. So they are tracking pretty precisely how long things take.
So when the dentist discovered my cavity he knew right away from its size, its location, my general dental situation how long it would take to make the filling. In the past year alone he’s probably done at least 1000 treatments with lots of fillings but also doing caps and some extractions. And each time he knew exactly how long it took him.
Compare that to the situation of a software developer. Sure, a software developer is spending thousands of hours, well, developing software each year. But what kind of experience does she gain thereby? Tacit experience. It’s not explicit, it’s not quantified. No time tracking is taking place for accomplishing tasks. And she often does not even know when a task is truly accomplished.
On top of that the tasks a software developer is working on are much, much less comparable to each other that what a dentist is doing. “Making a filling” might range from 30 minutes to 60 minutes. If a treatment is going to take longer, than it’s called differently, e.g. “Doing a root canal” which then might range from 60 minutes to 180 minutes.
But how long does a bug fix take? Maybe 5 minutes, or maybe 3 days. The variation in effort for tasks in software development is huge. And that’s the case because there hardly exists and classification for tasks.
We can distinguish between UI or domain logic or data access tasks. Or we can distinguish between bug fixing and new features. We can distinguish between one more data entry field and one more report. But even though there might be some categories to assign to tasks, this is not done routinely. And it is not tracked what that means with regard to effort.
Thus software developers do not really build up explicit experience which would allow them to make credible estimations.
The Illusion of Quality
The dentist knows when he’s finished a treatment. There are clearcut criteria for that. Short term and long term quality are defined by processes, materials, and metrics. That’s why a dentist has to study all those years and practice under supervision and go to all those advanced trainings even after he has set up his own shop.
Compare this to software development. Once you started work on an estimated task and you’re starting to run out of budget… what do you do to stay within the budget? Because that’s what estimation is about: budget. “Do not run over budget!” is the first commandment of project management.
So what do you do? You start cutting corners. It’s as easy as that. Even though you don’t what to you finally will cave in. You’ll leave out some functionality here, some performance there, security hasn’t be specified exactly anyway, and nobody is going to miss non-existent tests anyway. Not to mention the elusive notion of “clean code”. You’re not going to receive a pad on the back for nicely refactored code. But you will get slapped for not delivering within budget.
This is what thus happens usually: Delivery is over budget (due to lack of control and experience) and with less quality that expected. And your one true learning is: Nobody actually cares. Some finger pointing will happen and will pass. Then on to the next estimation which will not be honored.
Except for glaring gaps in functional requirements or efficiency requirements lack of quality will be of little concern as long as there was no huge budget overrun.
And even if the budget had been overrun my much, and even if much behavior is missing, and even if (hard to detect) code smells were created… No post mortem will be done. Nobody will actually ask “Why did this happen? Why was the estimation so far off?” Nobody will actually record not even two simple numbers: the estimation vs. the real budget used. Let alone and deviation in quality.
***
Given these three big, fundamental illusionary premises why should software estimation broadly and reliably work? I don’t have a clue. And I don’t see it working except in rare cases.
But not all is lost! Now that the illusions have been spotted maybe there is a way to get to more reliable software estimations - if you must.
3 Tips to Improve Your Estimations
Tip #1: Get in Control
First and foremost refuse estimations as long as you don’t feel in total control of material, tools, situation, circumstances. Get to know your trade. Get to know the code base. Truly understand requirements. And finally: Truly focus while working on estimated tasks. Do not allow deviations or interruptions.
Tip #2: Collect Experience
Whatever you do be very conscious about it. Feel the flow (or the lack of it). Record what you’re doing: is everything going according to plan or are you running into obstacles? Do you need to troubleshoot, learn, wait for others, do re-work? Track how long you expected to work on something and how it actually took you. Reflect on the categories task might fall into. Do you see any patterns?
Compile a database for reality checks. It’s easy to deceive yourself. But if you record hard facts you truly build experience to rely on.
Tip #3: Measure Quality
Be sure to know what the true scope is you’re supposed to deliver within an estimated budget. What are the functional requirement? What are the efficiency requirements (e.g. performance, scalability, security, usability, fault tolerance…)? What are the evolvability and correctness requirements?
Then measure how much of all that you deliver within budget. That should be easy for functional and efficiency requirements. But what about evolvability? That’s hard. That’s why you’re prone to deliver less, even much less than expected. If you’re serious about evolvability and correctness, though, you’ve to take measures to keep them up even while you’re moving fast and are stressed out. Otherwise you won’t live up to your estimation - even though someone might pad you on the back for a quick win.