[The Ecosystem of IT Workplaces] The True Nature of 'Developer Bias/Overly Optimistic Estimates' That Corrodes the Workplace

The weight of the words, 'It's just a few final tweaks'

On a Friday evening, the sound of rhythmic typing echoes through the office. When asked about his progress, he replied with a smile:
'It's almost done. It's just a few final tweaks, so I'll have it ready for you to check by Monday morning.'

However, on Monday morning, it was him at his desk with bloodshot eyes. The screen was filled with countless error logs.
'Well, it turned out to be a bit tougher than I thought...'

In the IT workplace, we have seen this scene, like something out of a movie, repeated over and over again.
'Progress is at 80%'
Why does it take more than half of the total time to finish after being told that? This is by no means because they are slacking off or lack the ability.

It is because of a clever trap set by the human brain: 'developer bias'.

The 'drug called optimism' happening inside an engineer's brain

There are three major misunderstandings lurking behind why estimates become overly optimistic.

The biggest one is likely the unconscious tendency to only envision the 'happy path'.
The happy path is a route where no errors occur, there are no specification changes, and everything proceeds exactly as ideally planned. To use a travel analogy, it's like calculating your arrival time on the assumption that you won't hit a single red light and there will be zero traffic. Such a drive is impossible in reality, but for some reason, in the brain before writing code, it becomes the 'standard'.

The second is the illusion that 'the only development work is the time spent writing code'.
Actual work includes a mountain of time spent deciphering past code, researching how to do things, and waiting for reviews. Even though typing on the keyboard is only a small fraction of the total, at the moment of estimation, we only think of the 'time spent typing on the keyboard'.

And the third is the pressure from those around us and the vanity of 'wanting to be seen as a capable person'.
When asked, 'How long do you think it will take?', we can't refuse and end up answering, 'If I work hard, it'll take two days!' The atmosphere in the room softens at that moment, but you are just saddling your future self with a massive debt.

Abandon grit and use systems to win

So, how should we confront this habit of the brain?
Even if you try to 'estimate more carefully' with sheer willpower, the bias won't disappear. What is needed is a system that suspects yourself from the start.

First, I want you to try creating a rule where you mechanically submit 1.5 times the amount of time you think it will take to 'get it done'. If you feel it will take '3 days', convert it in your brain to '4.5 days' and communicate that. This slightly lonely level of detachment—not trusting your own intuition at all—is effective.

Another thing is to break down tasks into the smallest possible pieces. If you keep it as a large chunk like 'create an inquiry form', the estimate will inevitably be rough. Try breaking this down into 'screen layout', 'input validation', 'email sending process', and 'display when an error occurs', down to a level that can be finished in one to several hours. The more finely you break it down, the more the hidden mysteries of 'Wait, how do I do this part again?' will surface.

Before starting work, it is also recommended to spend just one minute imagining, 'If this task were to be delayed, what would be the cause?' You will be able to notice small pitfalls in advance, such as 'Oh, I haven't properly read the specifications for that external service yet'.

'Finishing as promised' is much better than 'finishing early'

What is important for an engineer is not to say smooth things like 'I can do it quickly', but to 'deliver on the day you said you would'. Adding buffer time is not cheating or slacking off. It is a proper form of quality control to protect the project from unexpected trouble.

There is also something I would like the business people and managers around us to know. Asking an engineer, 'How long will this take?' and expecting a single number can be a bit harsh. Simply asking, 'How long would it take in the best-case scenario, and how long if you run into any snags?' and allowing for a range will significantly reduce stress for everyone.

When you receive an estimate that includes plenty of buffer days, please do not press them by asking, 'Can't you do it faster?' That is a sign of their sincerity, showing that they have seriously confronted the risks.

No one can perfectly predict the future.
That is precisely why I want us to be a team that can laugh together and support one another, while factoring in the 'optimism' of our own brains in advance.

[Casual Memo] 'Developer Bias' and 'How to Overcome It'

1. What is 'Developer Bias'?

This is a classification of psychological and environmental factors that cause developers to unconsciously become optimistic or drift away from reality when creating estimates.

(1) Psychological and Cognitive Biases (Brain Habits)

・Planning Fallacy
The tendency to assume that 'this time it will go smoothly,' even if you have experienced delays with similar tasks in the past.

・Self-Serving Bias / Overconfidence Effect
Overestimating one's own technical skills or ability to handle trouble, and scheduling under the assumption that no errors or bottlenecks will occur.

・Optimism Bias
Excluding 'obstacles caused by external factors' such as network failures, API specification changes, or sudden requirement changes.

・Tunnel Vision (Happy Path Orientation)
Calculating only the amount of code for the 'normal path (the flow where everything works correctly)' and failing to account for exception handling, testing, and refactoring time.

(2) Lack of Understanding of Technical and Operational Structures

・The Illusion that 'Coding Time' = 'Development Time'
Forgetting peripheral work time such as research, design, building, verification, code reviews, documentation, and deployment preparation.

・Underestimating Unknown-Unknowns
When introducing new technologies or external libraries, estimating research and verification time at only a few hours, assuming 'I can figure it out if I look it up.'

・Ignoring Technical Debt
Failing to include time for deciphering existing code or investigating the scope of impact caused by modifications.

(3) Organizational and Interpersonal Pressure (Environmental Bias)

・The Desire to 'Show Results Quickly / Be Recognized'
Presenting a wishful-thinking schedule due to the psychological desire to appear competent or to please clients and superiors.

　・The 'Pressure-Induced Compliance' Bias
　　When asked by sales or PMs, 'Can you get this done by X date?', you are unable to say no and end up stating the bare minimum number of days, saying 'I can do it if I try hard enough.'

2. How to Overcome Developer Bias

　Since bias cannot be solved by sheer willpower, we will categorize countermeasures from the perspectives of 'systems,' 'methods,' and 'communication.'

　(1) Systematizing Estimation Methods and Calculation Rules

　・Task Breakdown (Thorough WBS)
　　Break down tasks until each one can be completed in half a day to a full day (e.g., within 4 to 8 hours). The smaller the breakdown, the fewer cognitive oversights occur.

　・Standardizing Buffer Rules (e.g., 'Three-Point Estimation')
　　　・Provide three scenarios—best case, standard case, and worst case—and calculate a weighted average (PERT method).
　　　・Establish an organizational rule to use 'calculated estimate × 1.3 to 1.5' as the standard buffer.

　・Comparison with 'Past Performance Data' (Analogous Estimation)
　　Instead of relying on your own intuition, pull the 'actual time spent' on similar past tasks from logs (Jira, Trello, etc.) and apply that data.

　(2) Reviews and Organizational Processes (Introducing External Perspectives)

　・Peer Estimation (Comparative Estimation/Planning Poker)
　　Do not rely on one person's intuition; have multiple team members provide estimates and align perceptions where there are discrepancies.

　・'Devil's Advocate' (Pre-mortem) Session
　　Before finalizing an estimate, discuss 'If we assume this project didn't finish on schedule, what would be the reason?' and incorporate risk time in advance.

　(3) Improving Mindset and Communication

　・Distinguishing Between 'Development Time' and 'Working Time'
　　Even with an 8-hour workday, calculate based on the premise that actual focused coding time is only '4 to 5 hours' due to meetings, chat replies, reviews, etc.

　・Lowering the Psychological Hurdle for Presenting Buffers
　　Build a culture where bosses and clients understand that the value lies not in 'fast estimates' but in 'estimates that can be reliably met.'

[Related Articles]

☆The Ecosystem of IT Workplaces

☆The Path for Project Managers

☆The Small Path of Project Management

Book Introduction

・The Mythical Man-Month
・Frederick P. Brooks, Jr. (Author), Toru Takizawa (Translator), Yuko Makino (Translator), Noboru Tomizawa (Translator) Format: Kindle Edition
・Maruzen Publishing 2014/4/20
A masterpiece read as a classic of software engineering in large-scale development projects. Brooks' Law, which states that 'adding manpower to a late software project makes it later,' is famous. It points out the problems of applying the 'man-months' schedule estimation in development.
(* Book descriptions, etc., are generally based on the descriptions on the linked site (Amazon). Please refer to the link for detailed content.)

*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

・A Guide to the Project Management Body of Knowledge (PMBOK Guide) 7th Edition Kindle Edition
+ The Standard for Project Management: Supervised translation by PMI Japan Chapter
Project Management Institute (PMI) (Author)
General Incorporated Association PMI Japan Chapter (2023/1/6)
(* Book descriptions, etc., are generally based on the descriptions on the linked site (Amazon). Please refer to the link for detailed content.)
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

・Illustrated Instant Combat: A Textbook to Thoroughly Understand PMBOK 7th Edition Knowledge and Methods in One Book Kindle Edition
Kazuya Maeda (Author)
Shoeisha (2024/9/20)
This is a commentary book on the PMBOK Guide 7th Edition, known as the global standard for project management. In addition to the basic knowledge of project management such as 'Project Basics,' 'Value Delivery System,' and '12 Principles,' it explains in detail the key points of the '8 Performance Domains,' which can be called the main theme of PMBOK 7th Edition. It is a book recommended not only for studying project management but also as a first step for PMP exam preparation.
(* Book descriptions, etc., are generally based on the descriptions on the linked site (Amazon). Please refer to the link for detailed content.)

*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

・Illustrated Introduction: Easy-to-Understand Basics of the Latest PMBOK 6th Edition Kindle Edition
Yasuji Suzuki (Author)
Shuwa System (2018/3/23)
The PMBOK Guide has been translated and published in 11 languages worldwide, including Japanese, by the Project Management Institute in the United States. Even when translated, it is difficult to understand due to the many technical terms and katakana terminology. The author, who is also the leader of the translation and supervision team for the 'PMBOK Guide 6th Edition,' explains the original meanings as simply as possible. It uses plenty of illustrations to make it easy to visualize, making it easy to understand for everyone from beginners to veterans.
(* Book descriptions, etc., are generally based on the descriptions on the linked site (Amazon). Please refer to the link for detailed content.)

*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

・A Book to Thoroughly Master the Basics of Project Management in One Volume Kindle Edition
Kazuya Maeda (Author)
Gijutsu-Hyohron Co., Ltd. (2022/06)
This book is an introductory guide where you can learn about project management from the basics. After explaining the fundamental knowledge of project management, it divides the project into five stages: 'Initiating,' 'Planning,' 'Executing,' 'Monitoring and Controlling,' and 'Closing,' and carefully explains what should be done and the points to be careful about at each stage.
(* Book descriptions, etc., are generally based on the descriptions on the linked site (Amazon). Please refer to the link for detailed content.)

*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-