Comparing Security Testing Options

Vulnerability Assessment
Vulnerability Assessment tools and technologies are leveraged to uncover all possible weaknesses in network infrastructures and associated services. Many of these tools and technologies pose an ongoing problem with false positive reporting. False positives occur when vulnerability is raised that in reality does not exist. Vulnerabilities must be verified and the impact of these vulnerabilities must be rated. Industry surveys have shown that typically there can be tens of thousands of vulnerabilities found by vulnerability scanners on large enterprise networks; however, only a small percentage represent critical business exposures.

Penetration Testing
Penetration testing actively exploits vulnerabilities in specific network resources and infrastructures to identify tangible threats through real-world attacks. The results from a penetration test enable IT staff to delineate critical security risks and issues and implement effective remediation tactics.

The Myth: Vulnerability scanning is penetration testing
Penetration testing can only be achieved through the effective vulnerability mapping and potential exploitation of identified vulnerabilities. A simple vulnerability scan is not a penetration test.

Comparing Testing Options

Action	Vulnerability Assessment	Penetration Testing
Scope of testing	Scan for all known potential network vulnerabilities.	Exploits vulnerabilities identified from vulnerability scanning
Vulnerability weight	Vulnerabilities categorized on standardized and theoretical information and their potential impact to specific network resources or infrastructures	Vulnerabilities are tested and potentially exploited and categorized according to impact to specific network resources and infrastructures
False Positives	Provides false positives. All documented vulnerabilities must be manually verified	Vulnerabilities identified are exploited. False positives can still occur if testing team does not maintain an up-to-date exploit database
Network Connectivity	Does not attempt to leverage attacks on network trust relationships	Exploits trust relationships in network resources and infrastructures
Testing of Security Technologies	Does not simulate attacks to test Firewalls, IDS, IPS or other security controls and technologies	Real world attacks leveraged to bypass Firewalls, IDS, IPS or other security controls and technologies
Depth of Testing	Shallow: typically only known vulnerabilities are identified	Deep : testing teams can go as deep as the client scope indicates
Security and Risk Assessment	Only identifies standardized and theoretical information on threats making it difficult to effectively assess security risks.	Provides tangible evidence of network threats
Remediation	Delivers lists of potential vulnerabilities that can limit remediation	Assesses potential risks of specific vulnerabilities, allowing a focused approach to mitigation strategies

Typically vulnerability assessments and penetration testing occur together with results from the vulnerability assessment mapped to the corresponding exploits from the penetration test. As noted in the above table, penetration testing can produce false positives due to the fact that testing firms must be able to produce and maintain an effective database of known exploits that can be directly mapped to vulnerabilities.

Another factor that is often overlooked are the skills of the testing firm. Any organization that actively promotes security testing must possess the technical capability to identify and write code to exploit services deemed vulnerable.

"Through 2008, insiders, working alone or with outsiders, will account for the majority of financial losses from the unauthorized use of computers and networks."
Gartner Group