What Penetration Testing Actually Is (And Why Your Scanner Report Does Not Count)
Here is what our penetration testers find in the first four hours of an engagement. Not the sophisticated stuff. The boring stuff.
Default credentials on network devices. VPN concentrators running firmware from 2021. Internal file shares with no authentication on the same subnet as client workstations. Printers with web admin portals that have never had the default password changed.
Before we try a single exploit, we have usually found three to five ways into the environment that require no special skill whatsoever.
This is not unique to the organizations we test. It is the baseline condition of most mid-market environments, not because IT teams are negligent, but because nobody is looking at the environment the way an attacker looks at it. That is exactly what a penetration test is designed to do.
What a scanner finds versus what a pen tester finds
An automated vulnerability scanner looks for known CVEs against known software versions. It tells you: this version of Apache has a known vulnerability. It does not tell you: that vulnerability, combined with this misconfiguration, combined with that service account's broad permissions, creates a direct path to your domain controller.
A penetration test is adversarial. The tester is trying to accomplish a specific goal. Gain domain admin. Exfiltrate sample data. Pivot from the guest network into the internal environment. The CVEs are a starting point, not the deliverable. The deliverable is a map of what a real attacker would do, in what order, with what tooling.
Scanners answer the question: what software versions do I have and which ones have known vulnerabilities? Pen testers answer the question: given everything in this environment, how far can I get and what would it take to stop me?
Both questions matter. They are not interchangeable.
What PTES methodology means in practice
The Penetration Testing Execution Standard, PTES, provides a framework that qualified testers follow across six phases: pre-engagement, intelligence gathering, threat modeling, vulnerability analysis, exploitation, and post-exploitation. This is not a checklist. It is a structured approach to thinking like an attacker across the full lifecycle of an attack.
What this means practically is that a tester following PTES does not stop at finding a vulnerability. They attempt to exploit it, then ask what they can do from that position. Can they move laterally? Can they elevate privileges? Can they reach the data that matters? The answer to those questions is what separates a useful pen test from a scanner report with a narrative attached.
When you are evaluating a penetration testing provider, ask them to describe their methodology. If the answer sounds like a tool list rather than a process, keep asking.
Three certifications that signal a real tester
Certifications are not a guarantee of quality, but they are a reasonable filter. Three that matter in this space:
OSCP, the Offensive Security Certified Professional, requires candidates to complete a hands-on exam in which they compromise a set of machines within a time limit. There is no multiple choice. You either get in or you do not. It is one of the few certifications in security that cannot be passed by memorizing a study guide.
GPEN, the GIAC Penetration Tester, covers network penetration testing methodology in depth and requires demonstrated knowledge across scanning, exploitation, and password attacks. It carries the weight of the GIAC brand behind it, which has been a standard in the industry for over two decades.
GWAPT, the GIAC Web Application Penetration Tester, is the relevant credential for organizations with public-facing web applications. Web application attacks are a distinct discipline from network penetration testing, and the two are not interchangeable.
If your provider cannot tell you which certifications their testers hold, ask directly. If they redirect you to company certifications rather than individual tester credentials, that is worth noting.
What a proper deliverable looks like
A penetration test deliverable should contain three things a scanner report cannot produce: an attack narrative, a prioritized risk register, and a remediation roadmap.
The attack narrative describes what the tester did, in what order, using what techniques, and what they were able to accomplish. It should read like a structured account of an intrusion because that is what it is. If the report reads like a CVE list with severity ratings, you received a scan.
The risk register prioritizes findings not just by technical severity but by exploitability and business impact. A critical CVE on an isolated legacy system that cannot be reached from the network is not the same risk as a medium CVE on a system with direct access to your production database. The prioritization should reflect that.
The remediation roadmap gives your team a sequence. Fix this first because it is both exploitable and directly in the path to your most sensitive data. Then this. Then this. That ordering is judgment that a scanner cannot exercise.
What cyber insurance carriers are actually requiring now
Underwriters have changed their requirements, and not gradually. Most carriers now require evidence of annual penetration testing as a condition of coverage. Not just a scan. Not just a report. Evidence that a qualified practitioner actually attempted to compromise your environment and documented what they found.
Several carriers are now requesting the pen test report itself as part of the renewal package. They want to see the methodology, the tester credentials, and the findings. A scanner output repackaged as a penetration test will not satisfy this requirement, and underwriters are becoming more sophisticated about telling the difference.
The coverage implications are significant. Organizations that cannot demonstrate adequate testing may face premium increases, exclusions for specific attack types, or denial of renewal on existing terms. If your renewal is in the next 90 days, this is not a procurement decision you can defer.
The three questions to answer before your renewal: When was your last adversarial penetration test? Do you have a report that documents methodology and tester credentials? Are those findings remediated and documented?
If the answer to any of those is unclear, get clarity before your carrier asks the same questions.
What to ask before you hire someone
Three questions that cut through the noise in any vendor evaluation:
Are your testers certified, specifically OSCP, GPEN, or GWAPT? Individual tester credentials, not company-level certifications.
Do you operate under a signed Rules of Engagement document? This defines the scope, the timeline, the systems that are in and out of scope, and what happens if something breaks. Any legitimate provider will have one. If they do not, walk away.
Is the deliverable adversarial? Meaning someone actually tried to break in, documented what they did and how far they got, and produced a report that reflects that. Or is it automated output with a cover page?
If you are evaluating penetration testing providers and want a direct conversation about methodology, reach out. We will tell you whether we are the right fit, and if we are not, we will tell you that too.
Visit 9realmssecurity.com/services to learn more about how 9 Realms Security delivers penetration testing for mid-market organizations.