9 Realms Cybersecurity
All Posts
Security Operations

: From Assessment to Action: Why Point-in-Time Testing Is Not Enough

Chuck Flynn
: From Assessment to Action: Why Point-in-Time Testing Is Not Enough

The most expensive security assessment is the one nobody acts on.

I have reviewed hundreds of security programs over two decades. The pattern that costs organizations the most is not the absence of testing. It is testing that produces findings, findings that sit in a PDF, and a PDF that lives in SharePoint until the next renewal cycle forces someone to open it again.

Point-in-time testing is necessary. It is not sufficient. Understanding the difference between those two things is the bridge most mid-market security programs are missing.

Why Point-in-Time Testing Cannot Stand Alone

A penetration test is a photograph. It captures what was true in your environment on a specific date, against a specific scope, by a specific team using specific techniques. That photograph is genuinely valuable. It tells you what your attack surface looked like, which vulnerabilities are exploitable in sequence rather than in isolation, and where a real adversary would likely focus first.

It does not tell you what is true today.

The gap between what your pen test found and what your environment looks like right now is not hypothetical. It grows from the moment the test ends. New assets come online. Users are provisioned and deprovisioned. Third-party integrations are added. Cloud resources spin up outside of change management. What was scoped six months ago is already a partial picture of a network that has kept moving.

Three specific categories of drift account for most of the exposure that opens between tests.

Asset count. Shadow IT is not a failure of policy. It is a predictable consequence of how businesses operate. SaaS tools get connected to corporate identity. A developer spins up an EC2 instance. An acquired company's assets never fully merge into the known inventory. Every asset outside your current scope is an asset outside your current testing coverage.

Identity drift. Privileged accounts accumulate permissions over time. Service accounts created for a specific integration outlive the integration itself. A departed employee's credentials stay active in a system nobody reviews. Identity is one of the fastest-moving surfaces in any environment and one of the least frequently re-assessed between formal testing cycles.

Third-party trust. Your pen test scope almost certainly excluded your vendors, your cloud providers, and your SaaS stack. The connections between your environment and those third parties represent real attack paths. The SolarWinds compromise did not require attacking SolarWinds customers directly. It required compromising the update mechanism those customers already trusted.

None of this is a criticism of penetration testing. It is a description of what point-in-time testing was designed to answer and what it was not designed to answer. The problem is not the tool. It is treating the tool as a complete program.

What Continuous Monitoring Actually Means at Mid-Market Scale

Continuous monitoring is not a synonym for buying a SIEM. A SIEM is a platform. Monitoring is an operational discipline that requires analysts, tuning, defined playbooks, and someone with the authority to act when something fires.

At mid-market scale, continuous monitoring means three concrete things. First, your environment produces telemetry across endpoints, network, identity, and cloud, and that telemetry feeds into a correlated detection capability rather than separate siloed tools. Second, someone is reviewing that telemetry continuously, not checking in on a schedule. Third, when detection thresholds are crossed, there is a defined response process with clear authority, not a notification that waits for the right person to be available.

That third element is where most programs have the widest gap.

The Bridge: Finding, Prioritization, Closure, Re-validation

Testing and monitoring only produce value when they connect to a closed loop. The loop has four steps, and most programs execute the first one reasonably well.

Finding is the output of your pen test or your vulnerability scan. You have a list. The list has severity ratings, usually in CVSS score form, and some description of what was found.

Prioritization is where the first break occurs. CVSS scores measure theoretical severity. They do not measure exploitability in your specific environment, the asset criticality of what is at risk, or whether compensating controls are already partially reducing the exposure. A critical CVSS score on an isolated legacy system with no network path to production data is not the same risk as a medium score on a box that sits between your VPN and your domain controller. Prioritization requires context that a scan cannot generate on its own.

Closure is patching, reconfiguring, or accepting the risk with documentation. This step requires ownership, tracking, and a defined remediation window. Most organizations have a patch management process. Fewer have a remediation tracking process tied directly to pen test findings.

Re-validation is the step that almost never happens before the next scheduled test. After you remediate a finding, you should verify that the remediation worked. Not assume it worked. Not trust that the ticket was closed. Verify that the vulnerability is no longer exploitable in the way it was when it was found. Without re-validation, your finding closure rate is a measure of tickets closed, not of risk actually reduced.

What MDR Adds That Vulnerability Management Does Not

Vulnerability management tells you what is exposed. MDR tells you when something is being exploited, including things your vulnerability management program did not flag.

The practical difference: a vulnerability scanner identifies that a system is running an outdated version of software with a known CVE. MDR detects that someone is attempting to exploit that CVE in real time, or that an attacker is using a technique that does not correspond to any CVE at all because it is a logic flaw in your specific configuration.

MDR also covers the time between tests. When an attacker is moving laterally through your environment on a Thursday night four months after your last pen test, no vulnerability scanner is going to catch that. A managed detection program with properly tuned detection rules, correlated telemetry, and an analyst on shift will.

The two capabilities are additive. Testing tells you where the holes are. Monitoring tells you when someone is using them, and everything in between.

Different Tiles, Not Substitutes

At 9 Realms, we describe our service model as a mosaic. Each capability is complete on its own. Vulnerability assessments and penetration testing produce value independently of whether you have managed monitoring. Managed monitoring produces value independently of whether you have run a recent assessment. But the full picture only emerges when both are in place and connected to each other.

The organizations that recover fastest from incidents are not the ones with the biggest tool budgets. They are the ones where testing findings inform what monitoring watches for, and where monitoring observations tell the testing team where to look harder next time. That feedback loop is the bridge between assessment and action.

Ready to Close the Loop?

If your organization has completed a vulnerability assessment or penetration test in the last 12 months and wants to understand what monitoring coverage should look like against those specific findings, we offer a close-the-loop assessment workshop. We review your existing test output, map it against your current monitoring posture, and give you a prioritized gap analysis with specific remediation steps.

No tool purchase required. No long-term commitment. Just clarity on where the loop is open and what it would take to close it.

Visit [9realmssecurity.com] to schedule a conversation.

Tags:pen test, continuous monitoring, MDR, vulnerability management, 9 Realms