The remediation truth gap

The research question

When an organization says a vulnerability is fixed, what evidence supports that statement?

The answer may come from a developer, a ticketing system, a new scanner result, a manual retest or a repeated exploit attempt. Those sources are not equivalent, but security programs often collapse them into the same closed status.

The remediation truth gap is the distance between the organization's belief that an issue is fixed and the evidence that the original security condition has actually changed.

This article defines a framework for studying that gap. It does not present empirical findings.

Different meanings of closed

Developer-complete

The engineer changed the code, configuration or permission believed to cause the issue. This is important evidence of action, but it does not yet prove the security outcome changed.

Ticket-complete

The task moved into a closed state in the project-management system. This is evidence that the workflow progressed, not that the exploit disappeared.

Detection-cleared

A scanner or rule no longer detects the weakness. That can be strong evidence when the detection accurately represents the security condition, but it may still differ from replaying the original exploit.

Adversarially retested

The relevant attack is repeated after remediation and the original security outcome is no longer reproducible.

The question is not whether the team did the work. The question is which event changed the security state.

Why the gap matters

Remediation reporting often flows upward through abstract status. Leadership sees open and closed findings. Security sees severity and age. Engineering sees tickets. Each view is useful, but none automatically proves the original attack stopped working.

This creates a subtle form of security debt. The organization may believe exposure is removed while the attacker path remains viable.

The gap also works in the other direction. A finding may be genuinely fixed while administrative systems remain stale. That means research should not assume tickets are always overly optimistic. It should measure the relationship between workflow state and technical state.

A remediation evidence ladder

The framework can classify closure evidence by how directly it tests the original security condition.

• Statement of intended remediation
• Code or configuration change
• Ticket closure
• Detection rule no longer triggers
• Targeted validation of the original condition
• Reproduction attempt fails with preserved evidence

The purpose is not to declare that every finding requires the same depth of retesting. Low-consequence configuration issues may not justify the same validation effort as material exploitation paths.

The research question is where organizations currently draw that line and how often the chosen evidence supports the confidence expressed in the final status.

What would need to be measured

• Original finding type
• Original evidence quality
• Remediation action
• Event that triggered closure
• Whether a targeted retest occurred
• Retest result
• Whether the finding later recurred
• Whether the organization changed its closure decision after retesting

The study would also need to record the time between remediation and retest, because a system can change after the original fix for unrelated reasons.

The role of findings state

A research design needs to distinguish security state from task state. If the only available record is the ticket, the study cannot reconstruct what the security system believed before and after remediation.

A stateful finding should preserve the original evidence, owner, remediation event, retest and final status. That history allows researchers to compare what the workflow said with what the technical validation showed.

Failure modes a study should distinguish

A failed remediation can mean several different things. The proposed fix may not address the real root cause. The fix may work for the reported request but leave an equivalent route open. The environment may have reintroduced the weakness after the original change. Or the retest itself may be invalid because the authorized context changed. Those cases should not be collapsed into a generic "reopened" category.

A useful study needs enough technical history to explain why the retest disagreed with the closure decision. That distinction is essential if the research is meant to improve remediation practice rather than merely criticize ticketing systems.

Organizational questions worth testing

The technical retest is only part of remediation truth. A future study should also examine who is allowed to declare closure, whether security can reopen work after engineering completion and how disagreements between task state and technical state are resolved.

Those governance questions matter because a strong retest process can still fail operationally if the organization has no owner for the final security decision. Research should therefore capture both the technical evidence and the authority model around closure.

Why this matters to RedMaw

RedMaw is designed so a finding closes by proof rather than a checkbox. That makes the remediation truth gap directly relevant to the product thesis.

The honest research contribution is not to assume the product thesis is universally correct. It is to define the evidence states clearly enough that organizations can measure how they currently decide that security work is complete.

A future benchmark could then examine which remediation signals correlate most closely with adversarial retest outcomes. Until that dataset exists, the responsible output is the framework, not a statistic.