Engineering economics / Quality assurance
Make the case for QA with the cost of getting it wrong.
Quality assurance uses time, people and infrastructure. Software failures use those resources too, often while interrupting customers or planned work. A useful investment case compares the relevant costs and risks, identifies what a proposed check can establish, and keeps observed results separate from hypothetical savings.
Start with a consequential failure
The business case needs more than a test count.
Consider an illustrative order workflow that occasionally creates duplicate records. A test that detects the repeated request may protect an important behavior. Its value depends on whether that failure can occur in the real workflow, the cost of correction, the effort to maintain the check and the other controls already in place.
- Quality work has a full cost
- Include planning, checking, environments, maintenance and investigation of misleading failures. The test tool's invoice is only one possible part.
- Failure work has a different cost
- Separate problems corrected before delivery from failures that reach users. Record the actual repair, support and disruption involved.
- A benefit claim needs its own evidence
- A caught defect establishes that the check detected that defect. It does not establish how many customers would have encountered it or what revenue would have been lost.
Six investment questions
Tie the proposed QA work to an observable consequence.
Choose the depth of verification according to the behavior, exposure and consequence. Safety, security and contractual obligations may require work independently of a short-term return calculation.
Rework before release
01Investment question: Which recurring defects consume repair and review effort?
- Business concern
- Repeated correction can interrupt planned work even when the problem never reaches a customer.
- Starting evidence
- Defect records, root causes, repair effort and the changes affected.
- QA contribution
- For the duplicate-order example, express the intended behavior and add a check at the boundary where the repeat is handled. Verify that the known incorrect behavior would fail it.
- Decision owner
- Engineering validates the cause and check; the product owner assesses displaced work.
- Evidence to review
- Relevant failures detected, correction effort and repeat occurrences over a stated period.
- Attribution limit
- A lower defect count can reflect less change, different reporting or a changed classification.
- Ownership implication
- Keep the regression check understandable and assign responsibility for updating it when the contract changes.
Outages and degraded service
02Investment question: Which service failures matter enough to change release priorities?
- Business concern
- A service interruption has different consequences depending on who is affected and what work stops.
- Starting evidence
- Incident records, affected tasks, service objectives and recovery evidence.
- QA contribution
- Test relevant failure and recovery behavior in an authorized environment. Use operating evidence to identify gaps that pre-release checks did not cover.
- Decision owner
- Service owners set reliability priorities; release and incident owners manage operational decisions.
- Evidence to review
- The tested scenario, observed service impact, recovery result and unresolved exposure.
- Attribution limit
- A period without an outage does not prove that a new test prevented one.
- Ownership implication
- Agree when reliability work takes priority and who can resolve a disagreement about the evidence.
Broken customer journeys
03Investment question: Can a suitable user complete the action the product depends on?
- Business concern
- A functional or interaction defect can block an inquiry, order or other important task.
- Starting evidence
- A defined user journey, failure reports and the actual receiving-system behavior.
- QA contribution
- Exercise the critical journey, including invalid inputs, repeated actions and relevant failure states. Check the result beyond the visible confirmation screen.
- Decision owner
- Product owns the intended outcome; engineering and QA verify behavior and investigate failures.
- Evidence to review
- A reproducible blocked task, the correction and successful completion in the tested conditions.
- Attribution limit
- Fixing a blocked task does not prove an increase in qualified demand, conversion or revenue.
- Ownership implication
- Keep the journey check aligned with the business process and its integrations.
Feedback and delivery flow
04Investment question: Where does the team wait for trustworthy information about a change?
- Business concern
- Slow or unreliable feedback can create queues and repeated context switching.
- Starting evidence
- Time to meaningful feedback, failed-check causes, reruns and release handoffs.
- QA contribution
- Keep useful automated checks close to the change while retaining relevant exploratory and acceptance work. Investigate flaky results instead of routinely rerunning until green.
- Decision owner
- Engineering owns the test system; QA contributes scenarios and investigation expertise.
- Evidence to review
- Feedback time, genuine defect signals, avoidable reruns and the remaining manual work.
- Attribution limit
- A faster suite alone does not establish faster end-to-end delivery.
- Ownership implication
- Budget for the people who maintain the checks and diagnose their failures.
The cost of changing the system
05Investment question: Will another owner be able to change this behavior and verify it?
- Business concern
- A difficult test setup or opaque dependency can make later work expensive.
- Starting evidence
- Setup instructions, test data, contracts, dependency behavior and maintenance effort.
- QA contribution
- Check whether another contributor can reproduce a relevant failure and verify the correction. Keep the test focused on the contract rather than incidental implementation details.
- Decision owner
- Maintainers decide the test boundary and what knowledge must survive a handover.
- Evidence to review
- A usable reproduction path, current instructions and recorded maintenance problems.
- Attribution limit
- More test code can add ownership burden if it is brittle or no longer relevant.
- Ownership implication
- Treat test data, environments and documentation as maintained parts of the software.
The next increment of QA investment
06Investment question: What specific uncertainty would the additional work resolve?
- Business concern
- A proposal should explain why this work deserves attention alongside other engineering needs.
- Starting evidence
- The failure mode, existing controls, proposed scope, full cost and decision deadline.
- QA contribution
- Compare a bounded check or exercise with the current process. Define what evidence would justify continuing, changing or stopping the approach.
- Decision owner
- The business owner chooses the investment with engineering and finance input.
- Evidence to review
- An agreed scope, cost basis, acceptance result and review decision.
- Attribution limit
- An assumed avoided loss should not be reported as a measured saving.
- Ownership implication
- Revisit the case when usage, architecture or the cost of maintaining the check changes.
Keep the economic record readable
Do not mix cash, capacity and hypothetical loss.
These categories support a decision. They are not amounts to add together automatically, and the same incident or staff hour should not appear twice.
| Record | What to capture | Keep separate | Validation owner | Claim limit |
|---|---|---|---|---|
| QA investment | People, tools, environments | Initial and recurring cost | Engineering and finance | Include maintenance |
| Repair effort | Actual correction work | Effort and elapsed time | Engineering | Less effort is not always less spend |
| User-facing failure | Affected work and support | Observed and estimated impact | Service owner | Do not count the same loss twice |
| Recovered capacity | Time available for other work | Capacity and cash saving | Delivery owner | Record how capacity was used |
| Commercial outcome | Relevant business records | Demand and product changes | Business and finance | Correlation is not attribution |
| Avoided-loss scenario | Assumptions and sensitivity | Forecast and realized result | Risk and finance | Keep uncertainty visible |
Build a bounded investment case
Choose a decision period and a testable proposition.
Use the business's own evidence and cost definitions. Where the evidence is missing, explain what would need to be measured before making a monetary claim.
- 01
Name the risk and affected work
Describe the incorrect behavior, who is exposed and the consequence. Distinguish a recurring observed problem from a plausible future scenario.
- 02
Establish a comparable baseline
Record the current workload, release activity, failures and correction effort. Note changes in traffic, scope or reporting that would make later comparisons misleading.
- 03
Cost the proposed quality work
Include setup, execution, infrastructure, investigation and ongoing maintenance. Compare incremental costs over the same period as the proposed benefit.
- 04
Verify the mechanism
Show how the check detects the relevant wrong behavior or how the exercise reveals a recovery gap. Record what the evidence cannot cover.
- 05
Review the observed result
Compare the scoped outcomes and full costs. Keep a separate account of recovered capacity, cash effects and modeled risk, then decide whether to extend or adjust the work.
Protect the reasoning
Make uncertainty part of the investment record.
A credible QA proposal can contain unknowns. It should make them visible enough that the owner can decide what to investigate next.
- Avoid universal defect multipliers
- The effort to fix a problem depends on its cause, architecture, exposure and stage of discovery. Use local evidence rather than claiming every late defect costs a fixed multiple.
- Count each effect once
- A support hour, refund and revenue effect may be related but are not interchangeable. Agree the accounting treatment and remove overlapping estimates.
- Keep mandatory work explicit
- Some risk controls are required by the nature of the system or its obligations. A speculative short-term return does not decide whether those responsibilities can be ignored.
- Maintain a trustworthy suite
- Review checks that are flaky, redundant or expensive to maintain. Improve the signal and coverage; do not hide a real failure by weakening the expectation.
Questions about QA value
Ask what the return calculation actually contains.
The calculation is only as useful as its scope, cost basis and evidence.
- How is QA ROI calculated?
- For a defined incremental investment, a simple ROI calculation divides attributable monetary benefits minus incremental QA costs by those QA costs over the same period. The difficult part is establishing the inputs. Keep forecasts separate from observed results, and do not present staff capacity as cash savings without a justified financial effect.
- Does every caught defect represent avoided revenue loss?
- No. The defect may never have reached a user, another control may have caught it, or its commercial effect may be unknown. Record the detected behavior and its plausible consequence without inventing the counterfactual.
- Can better QA reduce delivery speed?
- Additional checks can add execution and maintenance time. Useful feedback can also reduce later investigation or repair. Measure the complete flow, including queues and reruns, rather than assuming that either more or fewer tests is always faster.
- Where should a team with little coverage begin?
- Start with consequential behavior and the boundaries other work depends on. A small set of meaningful checks can establish an initial safety net, then grow with the system. Coverage should follow risk and change, not an arbitrary test-count target.
- Is a passing suite enough to release?
- It establishes that the selected checks passed in their tested conditions. Release readiness also depends on configuration, operational responsibilities, unresolved defects and relevant manual or specialist review. A suite is one part of the decision evidence.
Source basis
Sources behind the control model.
- 01
American Society for Quality
Cost of QualityDistinguishes prevention, appraisal, internal failure and external failure costs. The article applies that distinction to software; source statistics and case-study savings are not adopted.
- 02
DORA
Test automationUpdated July 2025. Discusses continuous feedback, shared testing work and maintaining reliable suites. Its research is not a project-specific ROI forecast.
- 03
Google SRE
Testing for ReliabilityThe sections on test limits, test costs and establishing a test environment support scoped, consequence-aware verification. Production examples require separate operating authority.
- 04
Google SRE Workbook
Example Error Budget PolicyA 2018 example shows how service evidence can redirect attention to reliability. Its thresholds, schedules and escalation roles are illustrative, not requirements for every organization.
Start with one real workflow
A Systems Audit is the usual starting point. If the opportunity is already clear, we can move directly into a focused build.
Show Us the WorkflowStart with the free automation readiness checklistOBSERVEQUANTIFYDECIDEBUILD