What Makes a “Great” Penetration Test? Webinar Recap

BARR Advisory Cybersecurity Consulting Manager Larry Kinkaid was recently joined by penetration testing experts at the cybersecurity assessment and advisory firm Psicurity for a comprehensive discussion on the ins and outs of web application penetration tests.

During their talk, the experts explained what a high-quality web application pentest should look like—highlighting how effective pentests go beyond surface-level scans by carefully scoping the work, applying rigorous standards, providing actionable recommendations, and ultimately helping organizations strengthen security while building trust with customers, partners, and auditors.

Low-quality pentests can provide a “false sense of security,” said Neil Anderson, CEO and co-founder of Psicurity. This can result in mounting technical and security debt. “It can also impact trust with your customers and partners,” Anderson said.

“Ultimately, our view is that application pentests should be designed to find as many vulnerabilities as possible in the time available,” he explained. “You really want to do a good job of ensuring that the testing and results are thorough and that you’re not being given a cut-rate product.”

“The attackers have all the time in the world to try to hammer on things and find things, but we only have so long,” noted Gerrit Padgham, CTO, COO, and co-founder at Psicurity.

For SaaS organizations, this means it’s important to identify a pentest provider that does the “real, manual work” to offer actionable and relevant recommendations. “Are they providing detailed recommendations? Are they trying to actually get in, or are they just giving you a list of things they found and printing out a bulk report with hundreds of pages?” Anderson pressed.

“A pentest is not a vulnerability scan,” Anderson added. “A [quality] pentester is going to provide suggested fixes, detailed descriptions of the problem, steps on how to reproduce or exploit that problem if they’re able to actually hack it, and suggestions on how to remediate, which includes parameters, settings, or even code changes.”

According to the panelists, asking for examples of the provider’s pentest reports and soliciting referrals from trusted partners in similar industries can also help you determine whether they’ll produce quality work. “Your network is everything,” Kinkaid said.

In addition, Anderson said high-quality pentests should be based on a standard, such as OWASP’s Application Security Verification Standard (ASVS), which includes “17 domains of testing criteria and upwards of 200+ tests that could be in-scope for the application.”

“Ultimately, we recommend that the pentester uses this ASVS not only for guiding their testing methodology, but also documenting,” Anderson said. 

According to Kinkaid, leveraging a standard like ASVS improves credibility and helps organizations better communicate the results of their pentest with auditors and partners. “You look at the same thing with your SOC 2, your ISO [certifications], your HITRUST,” he noted. “When you establish that there is…a credible mechanism or process, now we can have the same kind of talk—OK, you’re using ASVS, now I know what my expectations are. We can have a very direct conversation around the other aspects of the pentest, because I already have an idea of what that would look like.”

ASVS “establishes a baseline to have the conversations later, as with any framework—compliance or not,” Kinkaid said.

Another point to consider is scoping. “You can get a pretty good idea of the quality of your pentesting firm on how they actually scope out the work for you,” Padgham said. “If they’re just asking for the number of endpoints and parameters…and they’re not actually seeing your application, that should be a red flag for you.” 

Padgham went on: “A lot of applications have workflows, business logic, and [other] stuff that you have to work through to get from one place to the next in your application. Simply asking for the number of endpoints and parameters on those endpoints isn’t going to [provide the pentester] any kind of concept or idea of what that business logic and that workflow looks like.”

Padgham added, “Your pentesting partner company should really be trying to understand what the application does, why it’s doing it, and how to get to all the points in the application from a business logic perspective.”

According to Anderson, the scoping conversation is important not just for determining the pentest provider’s quality, but also to ensure accurate pricing. “They may overcharge for it because they don’t really know how much effort it’s going to take.”

Finally, Anderson urged SaaS providers to vet the penetration testers themselves, including asking about their education and qualifications. 

“You want a pentesting team that’s going to augment your team or support your team, and guide them through the process as needed,” Anderson said. “The onus is on you to answer for your security program and to do it well.”

Alongside the webinar, Anderson, Padgham, and Kinkaid also teamed up to publish a whitepaper that breaks down the process of choosing a high-quality pentest provider step-by-step.

Download the whitepaper for free to learn more, or watch the webinar now on-demand.

Is a pentest on your security roadmap? Contact us today.