Is finding bugs earlier really cheaper?

Finding bugs early is one of the most common recommendations in software testing. The reasoning seems simple: a defect found during development should take less effort to fix than the same problem found after customers start using the product. That idea has shaped testing strategies for decades and sits at the center of the shift-left approach.

Discussions about defect costs often go further and claim that a bug found in production can cost 100 times more to fix than one found during development. The figure gets repeated a lot, and it deserves a closer look.

The research we reviewed suggests that detection timing matters, but how much depends on the defect, the software environment, and the work needed to fix it. This article goes through what that research says and what QA teams can do with it.

Where the 100x figure comes from

The link between when a defect is found and what it costs to fix has been discussed since the early days of software engineering. Barry Boehm's 1981 book Software Engineering Economics presented evidence that fixing problems later in development could take much more effort, especially in large projects.

It is easy to see why. A misunderstood requirement caught during planning might need only a conversation and a document update. If the same misunderstanding reaches production, fixing it could mean changing code, updating tests, coordinating a release, and handling customer complaints.

Over time, observations like these were often reduced to fixed multipliers. In The Leprechauns of Software Engineering, Laurent Bossavit traced the idea that defects cost more the later they are found, which sits behind figures like 100x, back to its sources. He found the evidence behind Boehm's cost curve weak. The few studies it cited were misrepresented, and one of them pointed the opposite way. In later decades the claim hardened into a general cost-of-change curve that was no better supported.

Even if the early findings were solid, they could not predict the cost of every modern defect.

A small interface issue, a database error, and a security vulnerability can take very different amounts of work to fix, no matter when they are discovered.

What recent research shows

A 2026 preprint, What Makes Software Bugs Escape Testing?, looks at this in more detail. Domenico Cotroneo and colleagues analyzed more than 14,000 defects from open-source C/C++ and Java projects and compared defects found before release with defects that escaped testing. The paper is posted on arXiv and covers open-source projects only.

The authors found that post-release defects tended to sit in older, frequently modified components with more complicated change histories.

• C/C++ projects: issues for these defects took longer to close, with a median of 25 days against 14 for defects found before release. The code changes needed to fix them were only slightly more complex.
• Java projects: the median also rose, from 14 to 28 days, but the most common closure time was the same in both groups, and the authors describe the gap as weaker.

Closure time shows how long an issue stayed open, which says little about the hours of work behind the fix.

The study did not try to establish a cost multiplier. It compares different defects rather than the same defect found at different times, and the authors note that many pre-release defects are simple and close quickly. We read this as a limit of the study design, because it cannot show by itself that early detection saves effort. It does suggest that defects which escape testing are a different group, and one that can take longer to close.

The authors suggest giving components with heavy change histories, high churn, and strong coupling to other parts of the system deeper regression and integration testing. Knowing where defects are most likely to escape helps a team decide where to test hardest.

Early tests only help when they can be trusted

Shift-left testing aims to find problems earlier, but early detection only helps when the tests give reliable information.

A 2015 study by Arash Vahabzadeh, Amin Milani Fard, and Ali Mesbah looked at bugs in test code. The researchers examined 448 Apache Software Foundation sub-projects, and 211 of them had bugs in their tests. Of the 5,556 test-related bug reports they found, they classified 443 by hand.

• About 97% were false alarms, meaning the test failed even though the production code was correct.
• The other 3%, or 15 reports, were tests that could pass despite a defect in the software, and incorrect or missing assertions were a major cause. The authors note that such bugs are harder to detect and report, and the study counted only bugs that were reported and fixed.

The study does not discuss shift-left. Our takeaway is that running tests earlier does not necessarily save effort if the tests are unreliable. False alarms can send developers after problems that do not exist, and a wrong assertion can let a real defect through.

Finding a bug is only the start of the work

Finding a defect early gives a team the chance to fix it efficiently, but what happens next matters too. In Bug Tracking Process Smells in Practice, Erdem Tuna and colleagues studied bug tracking in four JetBrains projects. They looked for deviations from recommended practices, such as bugs with no assignee or no link to a fixing commit. These deviations did occur, but developers disagreed about how serious many of them were, and only 24 developers answered the survey.

Take a checkout defect found before release. If the bug report has clear reproduction steps, the affected environment, and the expected result, developers have a good starting point. Without that information, they may spend extra time just trying to reproduce the issue.

Arti Rana and colleagues, in Process of Finding Defects in Software Testing, describe related problems, such as duplicate reports and bugs that cannot be reproduced because of differences in server, environment, test data, or build, and recommend recording those details in the bug report. It is a descriptive paper, so we use it for examples only.

Whether an early find pays off depends on test reliability, communication, and how defects are managed.

Making shift-left testing work in practice

Larry Smith coined the term shift-left testing in a 2001 article in Dr. Dobb's Journal. The idea is to look for quality problems throughout development, not only close to release.

In practice, this can start before any code is written. Reviewing requirements and discussing unclear expectations can stop a misunderstanding from becoming an implementation problem. During development, unit and integration tests give faster feedback, and risk-based testing can focus effort on critical features and frequently modified components. The studies above did not test these practices.

Teams should also learn from defects that reach production. Customer reports, recurring issues, and production incidents show where test coverage is weak and where future testing should concentrate.

An organization can measure its own results. Tracking defect resolution time, escaped defects, unreliable test failures, and the effort needed to reproduce issues shows whether earlier testing is paying off. These numbers will not capture every consequence of a failure, but they give teams a better basis for deciding where to put their testing effort.

What the evidence supports

None of the studies above measures the cost of finding the same bug early versus late. Taken together, they suggest that late-found defects can take longer to resolve, that the gap differs between ecosystems, and that early tests only help when they can be trusted.

Finding bugs earlier can still reduce effort, especially when a problem could spread across components or reach customers, but the size of the benefit depends on the defect and on how the team handles it. The points above that go beyond what the studies report are our own reading.

A practical first step is to choose one component that has a history of escaped defects, add earlier checks there, and track the numbers above before and after the change. That gives a team evidence from its own product.

References

• Boehm, B. W. (1981). Software Engineering Economics. Prentice Hall.
• Bossavit, L. (2013). The Leprechauns of Software Engineering: How Folklore Turns into Fact and What to Do About It. Laurent Bossavit.
• Smith, L. (2001, September). Shift-Left Testing. Dr. Dobb's Journal, 26(9), 56, 62.
• Cotroneo, D., De Rosa, G., Improta, C., & Varriale, B. G. (2026). What Makes Software Bugs Escape Testing? Evidence from a Large-Scale Empirical Study. arXiv preprint arXiv:2604.26672.
• Vahabzadeh, A., Milani Fard, A., & Mesbah, A. (2015). An Empirical Study of Bugs in Test Code. In Proceedings of ICSME 2015, 101–110.
• Tuna, E., Kovalenko, V., & Tüzün, E. (2022). Bug Tracking Process Smells in Practice. In Proceedings of ICSE-SEIP 2022, 77–86. https://doi.org/10.1145/3510457.3513080
• Rana, A., Rawat, A. S., & Bijalwan, A. (2017). Process of Finding Defects in Software Testing. Proceedings of RICE 2017, ACSIS, vol. 10, 297–300. DOI: 10.15439/2017R18.