Magically Predictable Software Production
Here’s a story, or a thought experiment, of you like: Imagine you as a Product Owner (PO) find a black box which magically transforms written requirements into high quality executable code.
You just need to hook-up the black box to a GitHub repository. Once that’s done, the rest is easy: you feed it requirements as GitHub issues, and after some time you receive the generated code as a GitHub release.
No more time consuming discussions. No more whining developers. Instead an obedient black box to produce high quality code. The wet dream of any Product Owner, isn’t it?
Oh, sorry, before I forget: there’s just one glitch in this rosy picture. It’s not known how long it will take the black box to produce a high quality release. Sometimes a release happens after just a couple of hours, sometimes it takes a day or two, sometimes it even takes some weeks.
What a bummer.
Or maybe not? Would you not want to use this magic black box? It hasn’t cost you anything. It’s a present from some merciful Agile deity who wants to improve the life of poor Product Owners like you, aching under a load of erratic wishes streaming in from customers.
What do you think? Would you rather go back to working with human developers? Really?
Or do you need some more facts to make a final decision? Here’s how long it took the black box to transform issues into releases in a past project:
Imagine yourself in the position of that project’s new PO. (The former PO left because he got bored.) What would you do if your customer requested another feature? How would you answer her question „When will it be done?“ or „How much will it cost?“ (Of course your company would still charge its customers a hefty sum even though it gets the code produced for free by the magical black box.)
This is a tough one, isn’t it? No one to turn to. The black box is mute. It’s just quietly waiting for the next issue.
Maybe you should grab a screw driver and try to open the black box? There could be something inside it to give you a hint as to how long an issue will take to implement. But I can tell you: You won’t find anything helpful. You’d only destroy the black box’s magic. You won’t be able to get it back to working as diligently and flawlessly as it has before. It would be permanently broken in some way.
Maybe you should quit then? No developers to coerce into answering the customer’s question for you. No machine to dissect and tinker with. Aren’t you entitled to some help here?
Or… Are you a man, or a mouse? Take up the challenge!
Answering „the question“
It’s the question every developer fears, it’s even the question every PO fears, I’d say: „When will it be done?“ (By the way, I recommend reading the like named book by Daniel Vacantie!) But as a real man (or woman) you’ll come up with a reasonable answer, even though you’re all alone with it. The only companion in your job being a mute and diligent, but unfortunately seemingly unpredictable black box.
You remember that solutions to hard problems often require to question every assumption. So what are your assumptions?
- Assumption #1: The black box is mute. Is that true? Yes, it is. No way you can ask the black box anything about its future performance.
- Assumption #2: The black box is diligent. Is that true? Yes, it is. Whenever you create a new issue it almost immediately starts to work on it (you know that because it turns from silent to rattling). And it will inevitably produce high quality code. The customer has never been happier; only few bugs leave the black box.
- Assumption #3: The black box is unpredictable. Is that true? Yes, because it does not tell you when it will be done with producing the release for an issue.
But is that really the definition of predictability? To be able to tell when one will be done? Can only people who have a mouth be predictable, because they can make a statement about how long they think something they are supposed to do will take?
Really? I don’t think so. I can’t even imagine you thinking so.
My guess is you’re running around all day making predictions of all sorts about the world without asking the world. You’re making predictions about the weather or traffic or your spouse or your boss or a line in front of the cash desk in a supermarket.
How can you do that?
You compile experiences. You observe, you remember, you derive conclusions from that. In short: you develop a more or less fine tuned „gut feeling“ for these situations.
Or should I say „a database of likelihoods“ instead of „gut feeling“? Because that’s what it is. You look at the sky and assess the likelihood of rain in the afternoon from cloud and wind patterns you observe. You look at the line in front of a cash desk and assess the likelihood of being served more quickly compared to another line from how many people are in it, how old they are, how many items they carry.
All this happens instantaneously and pretty unconsciously. You’re used to that. This is a reliable way to deal with the contingencies of life.
Your „database of likelihoods“ makes the world pretty predictable to you. Predictable enough, at least, so you feel comfortable (most of the time). No 100% guarantees needed. A high enough chance for a beneficial development is good enough. Or to put it the other way around: a low enough risk for a frustrating outcome is good enough.
What a low enough risk is, is up to you. People are different. Some like gambling: high risk, but also high potential reward. Some don’t like gambling. Ass the ancient Greeks said: Know thyself! What kind of „risk personality“ are you?
Ok, back to software development. Back to the customer’s question „When will it be done?“ Back to the ominous black box.
How are you going to answer the question? Well, I suggest you question your assumption #3. Is the black box really unpredictable? Maybe you should look more closely.
Here is the same historical release data as above, but now not in chronological ordern, but ordered according to how long it took for each issue to be turned into a release:
As you can see, the time to release ranges from 0.5 days up to 10 days. But the durations don’t appear equally often. There is a certain unequal distribution to their frequencies. Not all cycle times are equally likely.
Most often the black box needs 1, 2, 3, or 4 days to produce a release. Very rarely, though, it might take 9 or 10 days.
How’s that for an compilation of experiences? It’s hard facts. Undeniable reality. And not as erratic and unpredictable as it appeared at first, I’d say.
How much would you bet on the next roll of a dice resulting in a 6, if you could double your money (or lose it)? Your chance for winning would be 1/6=0.1667 or roughly 17%. The risk of loosing would be 1-1/6=0.833 or roughly 83%. Would that be a safe bet for you? Would you bet 1€ or 10€? How much would the potential gain need to change in order for your to bet?
Or how about a bet on the next roll of a dice being a number from 1 to 5? Again, you could double your money or lose it. Your chance for winning would be 5/6=0.833 (83%), your risk of losing 1-5/6=0.1667 (17%). How much would you bet? 1€ or 10€ or even 50€?
I don’t know you, but I bet you’d go for the second game with much lower risk. Even a much lower risk than 50:50.
Since you’re familiar with assessing chances/risks in gambling to either stay away or „invest“, I believe you are capable of dealing with the magical black box, too.
Look at the historical data. It delivers likelihoods (chances) and risks like the fictitious dice games.
If you’re like me, you’d risk a game with a roughly 83% chance of winning. So let’s see what that would mean when „playing“ with the black box.
The p-column shows probabilities for a certain number of days until release. A release in just 1 day is quite unlikely: just a 21% chance. But a release after 7 days is as likely as winning the second dice game, even more likely: a 84% chance. With two days more you even get a 95% chance or mere 5% risk.
How’s that for an answer to the customer? „Dear Ms. Bryant, we’ll be able to deliver the feature you requested with a 50:50 (50%) chance within 3 days, but if you prefer more reliability, you might want to expect it to take 7 days (84%).“
Or if you want to answer the cost question think of it like this:
- The costs per day (aside from the free black box) might be 500€.
- You can charge your customer 1000€ per day.
Internally you might go for a 16% risk = 7 days. Your costs would be 3500€.
But you cannot charge the customer 7000€. So you tell her 4500€ which still equals a 28% profit margin.
If all goes according to plan the feature takes 7 days and you earn 1000€. But maybe the black box only needs 4 days; there is an 68% chance for that. Then you make 2500€! Not bad, eh!?
However… it’s also possible the black box takes unusually long and the release only appears after 9 days. There’s a 11% chance for that. Then you earn nothing - but also still don’t lose any money.
Let’s not kid ourselves: even if it seems as if 10 days would equal 100% or total certainty, they do not. 10 days are just the current maximum duration observed. There could be an unknown risk of the black box sometimes needing even more than that. Maybe the feature in question will take 13 days? You don’t know until you feed it to the black box.
That’s how life is. It’s full of uncertainties, even though you might see a pattern (or probability distribution).
What do you do in the face of such uncertainty? You boldly decide! You don’t flee, you don’t freeze. You fight by taking your chances. Not blindly, but informed. A list of historical data like above helps. It’s better than gut feelings coming from unknown sources.
And the beauty of it: with each new experience (recorded time to release for an issue) the uncertainty is lowered (if there indeed is a pattern behind what the black box does). Each new data point improves the fact base and thus your capability to make predictions. It’s an explicit memory of past performances which is likely to resemble future performances.
Lowering the risk with classification
Now that you realized the black box is not that unpredictable, that there is „method to the madness“, that there is a pattern (frequency distribution), you might want to look even more closely.
You still cannot ask the black box anything. You’re still on your own. But maybe you realize, that not all issues are created equal. Maybe you find they belong to different classes (or categories).
Here are the same historical data as before but this time classified with two colors:
How you define such classes is up to you. You might classify according to kind of functionality or technology involved or your own idea of complexity of an issue or in which mood the customer was when she submitted the requirements. Be creative. Try out a few ideas - and see if they lead to more refined patterns.
Then classify the new feature request before you answer the question. It might change your answer drastically:
If the customer asked for a yellow feature, you could tell her with a 91% confidence that she would receive a release after 3 days. But if the feature in question was a red one… a comparable level of certainty would lead to 9 days as an answer.
Classification can help you to give more realistic answers - but be sure to not design the issue classes too sophisticated. It should be fairly easy to assign them. You, as the PO, need to be able to do it before the black box starts working on the issue.
Conclusion
What a story, what a thought experiment. Weird, wasn’t it? There are no such magical black boxes in the world.
Or are there?
I can tell you, I have seen such magical black boxes. Lot’s of them. But, alas, most have been tampered with. Customers, managers, POs pried them open to gain some insights into their workings, even to optimize them… but what they got was less performance, less quality. They couldn’t put the black boxes back together again. Sad, isn’t it.
So, in the - of course unlikely - event of coming across a magical programming black box, try to bridle your curiosity. Don’t try to improve a complex machine; you’ll fail. Accept its diligence and high quality work, even though you have no control of its performance. You’ll see, that does not mean it’s not predictable. Right to the contrary. You just need to look at it the right way.