Your users are not your QA team

A user launches your application, presses a button, and nothing happens. They refresh the page. Still nothing. If you're lucky, they send a message: "This isn't working."

That message is useful. But it is not a test result.

By the time a customer finds a bug, it has already reached production. The customer is no longer helping you prevent a problem. They are experiencing it.

For small, fast-moving teams it can be tempting to treat production as the final testing environment: ship the feature, see what happens, fix whatever gets reported. That is not QA. It is making your customers do QA for free.

Finding a bug is not the same as testing

Users are very good at one thing: telling you what happened to them. "The checkout button did nothing." "The page crashed." "My order disappeared." Reproducing and diagnosing a bug requires a different kind of information.

A study in the Journal of Systems and Software manually analyzed 1,000 bug reports from ten open-source web applications on GitHub, comparing what was known when a report was filed against what was added as the issue was investigated. These were not ordinary customer messages, and the study could not reliably separate end users from more technical contributors. Even there, important information usually arrived late:

• Diagnostic suggestions were added to 27% of reports after filing.
• End-user usage information was added to 178 of the 1,000 reports.
• Setup data was added to 116.
• Additional reproduction steps were added to 64.

This does not mean the initial reports were bad. More than half already included steps to reproduce, outcome information, and setup data. Structured templates helped teams collect more up front, but could not close the gap entirely, because different bugs need different evidence.

A user describes an experience. A tester investigates a system. Those are different jobs.

If a structured bug report can still need a follow-up investigation, "checkout doesn't work" was never going to replace testing.

Production is too late to start testing

"We'll fix it when users report it" has a second problem: you are choosing to find defects at the end of the process.

An IEEE case study of Fannie Mae's Agile-DevOps transformation is a useful, narrower example. It looked at structural-quality analysis specifically, not QA as a whole, so it is not evidence that QA alone produced the results. Before that analysis ran frequently, structural flaws were mostly found after release rather than during testing. Once teams scanned builds during the sprint, they caught critical issues before shipping, and teams scanning daily could fix critical defects within a day or two instead of waiting for the next sprint.

Across the transformation, applications improved 30% to 48% on critical defects per Automated Function Point. Teams that combined repeatable Agile practices, structural-quality analysis, test automation, and continuous delivery averaged a 28% productivity gain, and one team cut defects per Automated Function Point by 41% while shipping more in shorter cycles.

None of that means adding QA makes any given startup 28% faster — several practices changed at once, so the number can't be attributed to one of them. But the direction holds:

The earlier you build a quality feedback loop, the earlier you get to act on what it finds. Waiting for customers just moves that loop into production.

A working feature is not a quality feature

This is an easy trap for a small team. The developer builds it, it works locally, the happy path works in staging, it ships. Done.

Except "does it work when used correctly" is only one part of quality.

ISO/IEC 5055, the standard for automated source-code quality measurement, scores four structural characteristics: reliability, security, performance efficiency, and maintainability. It is not a QA standard and static analysis is not testing, but it makes a point worth keeping: most dimensions of quality exist before a customer ever produces a symptom.

• A user does not evaluate your architecture for maintainability. They notice the app keeps breaking.
• They do not measure performance efficiency. They notice checkout takes too long.
• They do not systematically test reliability. They notice something that worked yesterday doesn't work today.

ISO/IEC 5055 goes further and marks some security and reliability weaknesses as unacceptable, meaning the software should not ship until they are removed. That is a different philosophy from "ship it and see what users find." Quality needs a checkpoint before production, not an apology after it.

Feedback and QA answer different questions

None of this makes user feedback less valuable. It tells you whether people understand the product, where they get stuck, what they need, and how the software behaves in situations you never anticipated. You need that.

But QA is asking something else. User feedback asks what happened. QA asks what could happen.

A tester deliberately looks for edge cases, unexpected states, regression failures, and interactions between features. The goal is not to behave like a normal customer — it is to find the problems a normal customer should never have to hit. Ask your users what the product should improve next. Do not make them the system that decides whether the current release works.

Your customers should use the product, not test it

Small teams do not skip QA because they don't care about quality. Usually they are trying to move fast, and a regression cycle looks like something standing between a feature and its release.

But removing QA does not remove the work. It moves it.

Instead of a problem found before release, a customer finds it after. Instead of a controlled test result, the team gets a support message, and now someone has to reproduce it, collect the missing context, diagnose it, fix it, and verify the fix didn't break something else.

You did not save the testing time. You delayed it until the bug had a customer attached to it.

That is why QA should not sit as a final obstacle between developers and release. Whether it lives inside the team or with a specialized partner, its job is an independent feedback loop before production.

At GetQA, that is the role we think QA should play: developers keep building, while QA systematically challenges what's being built before customers have to. Because your users already have a job. Use the product. Not test it.

References

• Wang, D., Galster, M., & Morales-Trujillo, M. (2024). Information Needs in Bug Reports for Web Applications. Journal of Systems and Software.
• Snyder, B., & Curtis, B. Using Analytics to Guide Improvement during an Agile–DevOps Transformation. IEEE.
• Curtis, B., Martin, R. A., & Douziech, P.-E. (2022). Measuring the Structural Quality of Software Systems. IEEE Computer.
• ISO/IEC 5055:2021. Automated Source Code Quality Measures.