Penetration testing is an important part of the software security process. In this article, we will discuss why this is so from various perspectives.

Penetration testing meaning

A penetration test, also known as a pen test, is a simulated cyber-attack on a computer system in order to verify whether exploitable vulnerabilities are present. The goal of this simulated attack is to identify weaknesses in a system’s defenses that could be exploited by attackers.

In the context of web application security, penetration testing is commonly used to augment the web application firewall (WAF).

Penetration testing can involve attempting to breach any number of application systems (e.g., application protocol interfaces (APIs), frontend/backend servers) to uncover vulnerabilities such as untreated inputs that are susceptible to code injection attacks.

The insights gained from the penetration test can be used to fine-tune WAF security policies and patch identified vulnerabilities.

It’s similar to a bank hiring someone to disguise themselves as a thief and try to break into their building and gain access to their vault. If the “thief” succeeds and gets into the bank or the vault, the bank gains valuable information about how it should tighten its security measures.

Penetration testing is important because it’s one of the best ways to find and fix system security vulnerabilities before an attacker can exploit them. By conducting penetration testing, organizations can prevent or mitigate the damage that an attacker could cause if a security vulnerability is successfully exploited.

To protect yourself, your company should regularly perform penetration tests and:

• identify security weaknesses so that you can address them or put in place appropriate control mechanisms,
• make sure your existing security controls are effective,
• identify new bugs in existing software,
• test new software and systems for bugs,
• support your organization’s compliance with the GDPR (General Data Protection Regulation) and DPA (Data Protection Act) and other relevant data protection and security laws and regulations.

Have a look at our article 7 common penetration testing mistakes.

Penetration testing benefits

Ideally, software and systems have been designed from the start to eliminate dangerous security vulnerabilities. The pen test provides insight into how well this goal has been achieved.

Penetration testing can help an organization:

• find weaknesses in systems,
• determine the reliability of the control mechanisms,
• promote compliance with data protection and cyber security regulations (PCI DSS, HIPAA, GDPR, etc),
• provide management with qualitative and quantitative examples of current security status and budget priorities.

What is teaming?

The number of attacks is increasing, and the amount of research and experience required to outrun these attacks with your team is widening the gap between attack time and detection time. This is where teaming comes in. Teaming exercises simulate real attack scenarios – with one team attacking and the other defending.

Red teams

The red team is on the offensive. The red team is formed with the intent to identify and assess vulnerabilities, test assumptions, review alternative attack options, and uncover limitations, security risks to the organization.

Blue teams

The blue team is tasked with defending the organization. Blue teams are responsible for building the organization’s defences and taking action when necessary.

Purple teams

Recently, the purple team concept has become more popular in team exercises. This is a way of thinking in which red and blue teams are perceived and treated as symbiotic. It’s not red teams versus blue teams, but rather one big team focused on one main goal: improving safety. The key to becoming a purple team lies in the communication between individuals and their teams.

Pen testing phases

Penetration testers (abbreviated as pen testers) simulate attacks by motivated adversaries. To do this, they usually follow a plan that includes the following steps:

1. Planning and exploration

The first phase includes:

• Defining the scope and objectives of the test, including the systems to be addressed and the test methods to be used.
• Intelligence gathering (e.g., network and domain names, mail server) to better understand how the target works and what its potential vulnerabilities are.

2. Scan

The next step is to understand how the target application will react to various intrusion attempts. This is usually done by:

• Static analysis – inspecting the application code to estimate how it behaves at runtime. These tools can scan the entire code in a single pass.
• Dynamic analysis – checking the application code on the fly. This is a more practical way of scanning because it provides a real-time view of application performance.

3. Gaining access

In this phase, web application attacks such as cross-site scripting, SQL injection and backdoors are used to expose target vulnerabilities. Testers then attempt to exploit these vulnerabilities, typically by escalating privileges, stealing data, intercepting traffic, etc. to understand what damage they can cause.

4. Maintaining access

The goal of this phase is to determine if the vulnerability can be exploited to achieve a persistent presence in the exploited system – long enough for a malicious actor to gain deep access. The goal is to mimic advanced persistent threats, which often remain on a system for months in order to steal an organization’s most sensitive data.

5. Analysis

The results of the penetration test are then compiled into a report with detailed information:

• specific vulnerabilities that have been exploited
• sensitive data accessed
• the time during which the tester was able to remain undetected in the system

This information is analyzed by security personnel to help configure WAF settings and other application security solutions in the enterprise to patch vulnerabilities and protect against future attacks.

What to do with the penetration test results

Once the penetration test results are available, it is essential to go through them, discuss further plans and reassess the overall security posture of the organisation.

Penetration testers will provide thorough reports with information consisting of several elements – accurate detailed information about each phase of the test. After discussing the results, a good approach is to develop a remediation plan, validate the implementation with a retest, and incorporate the findings into a long-term security strategy.

Penetration testing methods

External testing

External penetration tests focus on company assets that are visible on the Internet, such as the web application itself, the company’s website, email servers, and domain name servers (DNS). The goal is to gain access and retrieve valuable data.

Internal testing

In internal testing, a tester with access to the application behind its firewall simulates a malicious insider attack. This is not necessarily a simulation of a rogue employee. A common baseline scenario might be an employee whose credentials have been stolen as a result of a phishing attack.

Blind testing

In blind testing, the tester receives only the name of the business that is the target of the attack. This gives security personnel a real-time glimpse into how a real attack on the application would play out.

Double blind testing

In double blind testing, security personnel have no prior knowledge of the simulated attack. As in the real world, they will have no time to strengthen their defenses before an attempted breach.

Targeted testing

In this scenario, both testers and security personnel work together and keep each other informed of their movements. This is a valuable training exercise that provides the security team with real-time feedback from the hacker’s perspective.

Penetration testing types

A comprehensive approach to pen testing is essential for optimal risk management. This includes testing all areas of your environment.

Web applications

Testers examine the effectiveness of security controls and look for hidden vulnerabilities, attack patterns, and any other potential security gaps that could lead to a web application compromise.

Mobile applications

Using both automated and extended manual testing, testers look for vulnerabilities in the application binaries running on the mobile device and in the corresponding server-side functions. Server-side vulnerabilities include session management, cryptographic issues, authentication and authorization issues, and other common web service vulnerabilities.

Networks

This testing identifies common to critical security vulnerabilities in the external network and systems. Experts use a checklist that includes test cases for encrypted transport protocols, SSL certificate scoping issues, use of administrative services, and more.

Cloud

Cloud environments differ significantly from traditional on-premises environments. The responsibility for security is usually shared between the organization using the environment and the cloud service provider. For this reason, cloud pen testing requires a set of specialized skills and experience to thoroughly examine various aspects of the cloud, such as configurations, APIs, various databases, encryption, storage, and security controls.

Containers

Containers obtained from Docker often contain vulnerabilities that can be exploited on a large scale. A common risk associated with containers and their environments is misconfiguration. Both of these risks can be detected through expert pen testing.

Embedded devices (IoT)

Embedded/Internet of Things (IoT) devices such as medical devices, automobiles, home appliances, oil rig equipment, and watches have unique software testing requirements due to their longer lifecycle, remote locations, power constraints, regulatory requirements, and more. Experts perform a thorough communications analysis along with client/server analysis to identify the vulnerabilities that are most relevant to a given use case.

Mobile devices

Pen testers use both automated and manual analysis to find vulnerabilities in application binaries running on the mobile device and in relevant server-side functions.

Vulnerabilities in application binaries can include authentication and authorization issues, client-side trust issues, misconfigured security controls, and cross-platform development framework issues. Server-side vulnerabilities may include session management, cryptographic issues, authentication and authorization issues, and other common Web services vulnerabilities.

APIs

Both automated and manual testing techniques are used to cover the OWASP API Security Top 10 list. The security risks and vulnerabilities that testers look for include broken object-level authorization, user authentication, excessive data exposure, resource scarcity/speed limitations, and more.

CI/CD pipeline

Modern DevSecOps practices integrate automated and intelligent code scanning tools into the CI CD pipeline. In addition to static tools that scan for known vulnerabilities, automated pen testing tools can be integrated into the CI/CD pipeline to mimic what a hacker might do to breach application security. Automated CI/CD pen testing can reveal hidden vulnerabilities and attack patterns that are not detected by static code scanning.

Types of penetration testing tools

There is no universal tool for pen testing. Instead, different targets require different sets of tools for port scanning, application scanning, Wi-Fi penetrations, or direct network penetrations. In general, the types of pen testing tools can be classified into five categories.

1. Survey tools for discovering network hosts and open ports.
2. Vulnerability scanners to detect problems in network services, web applications and APIs.
3. Proxy tools such as dedicated web proxies or generic man-in-the-middle proxies.
4. Tools to gain and abuse system footholds or access to resources.
5. Downstream tools to interact with systems, maintain and expand access, and achieve attack objectives.

Several different types of tools can be used in a penetration test, each for a different phase.

Information gathering and misuse tools

• Zmap: this simple network scanner can scan everything from your home network to the entire internet. It’s free, and pen testers often use it to gather basic network information.
• Xray: Xray is a network mapping tool that uses the OSINT framework to guide its tactics.
• SimplyEmail: this is an email research tool that is used to collect related information found on the internet based on someone’s email. Pen testers use it during the survey phase.
• PowerShell-Suite: the PowerShell-suite is a set of PowerShell scripts that can retrieve information about processes, DLL, and other aspects of Windows computers. Using this tool, pen testers can quickly check which systems on a network are vulnerable to exploits.

Vulnerability scanning tools

• NMAP/ZenMap: this network security mapping tool provides pen testers with a view of open ports on any network and allows testers to dive into the possibility of specific vulnerabilities at the network level.
• sqlmap: this is an open source penetration tool that provides validation of possible SQL injection errors that can affect database servers. It is best used in database abuse tests.
• MobSF: a great tool for detecting vulnerabilities in mobile platforms. It is a comprehensive platform for pen testing and vulnerability detection through static and dynamic analysis.
• Linux-Exploit-Suggester: this tool, as the name suggests, is best used to test security on Linux systems without having to deal with other robust vulnerability scanners.

Penetration testing benefits

• It exposes insufficiencies in high-level security assurance practices such as automated tools, configuration and coding standards, architecture analysis, and other lighter vulnerability assessment activities.
• It locates known and unknown software flaws and security vulnerabilities, including small ones that are not of great concern on their own, but could cause material damage as part of a complex attack pattern.
• It can attack any system, mimicking how most malicious hackers would behave, thus simulating a real adversary as closely as possible.

Penetration testing disadvantages

• It is labour intensive and expensive.
• It will not comprehensively prevent bugs and flaws from making it into production.

Penetration testing best practices

Here are some best practices you can use to increase the effectiveness of penetration testing.

Research and planning are key

Penetration testing should start with vulnerability scanning and open exploration of security gaps. Just like a real attacker, a penetration tester should conduct a reconnaissance of the target organization, gather information from available sources, and plan the most effective exploits.

This phase should be carefully recorded, including vulnerabilities that were discovered and not exploited in the actual test. This allows developers to reproduce and fix bugs in the future.

Creating an attacker profile

A penetration tester should think and act like a striker. He should consider the motivations, goals, and skills of cyber attackers. Motivation is an important factor in understanding hacker behavior. For example, a hacker who wants to commit financial fraud will act differently than a hacker who wants to exfiltrate sensitive data or a hacktivist who wants to cause damage.

Before conducting penetration tests, an organization should identify the characteristics and personalities of the most likely attackers, rank them, and focus the tests on the most appropriate personality.

Freezing development in a penetration testing environment

Successful penetration testing requires a known, stable state of the system under test. Adding a new patch or software package, changing a hardware component, or changing the configuration will invalidate the penetration test because the vulnerabilities discovered may not exist after the update.

It’s not always possible to predict the positive or negative security implications of an update – which is the reason for penetration testing in the first place. If there is no choice and systems must be modified during the test, the attacker should be informed of this and it should be reflected in the penetration test report.

Define scope and budget

It may make sense to want to test the whole environment, but the cost may convince you otherwise. So consider your high and low priority areas that need penetration testing. High-priority areas are those where the company’s greatest vulnerabilities exist. Pentesters routinely identify operating systems, application code, and configuration files as the highest risk areas, especially in software development projects. Lower priority areas include applications with little or no code for internal business operations.

Include sources of financial and customer attacks

The organization’s data is its biggest asset, particularly in retail, finance, government and healthcare. Organizations in these industries typically have vast amounts of transactional, customer and financial data. If your organization has this type of data, conduct comprehensive, enterprise-wide penetration testing of your data sources, especially to meet industry and security regulations. But don’t just stop at the data sources; test the software that connects to them and their supporting infrastructure as well.

Follow penetration testing methodology

Penetration test results can vary significantly depending on which methodology you use. Common methodologies and testing standards include:

• Penetration Testing Execution Standard (PTES)
• Payment Card Industry Data Security Standard (PCI-DSS)
• Open Source Security Testing Methodology Manual (OSSTMM)
• OWASP Web Security Testing Guide
• National Institute of Standards and Technology (NIST) Special Publication 800-115
• Information Systems Security Assessment Framework (ISAFF)
• The choice of method is important when performing the actual penetration testing. However, when looking for a penetration testing service, consider what methodologies they follow and how they compare to your goals.

Examples of pen testing scenarios

Example: stolen laptop scenario

A great penetration test scenario is to demonstrate the consequences of a stolen or lost laptop. The systems have permissions and credentials on them that attackers could use to gain entry into the target organization.

The system may be password protected, but there are many techniques that can allow attackers to bypass this protection. Examples include:

• The system’s hard drive does not need to be fully encrypted, allowing an attacker to attach the hard drive to their own system and obtain data and credentials. These credentials could then be cracked and reused on many of the organization’s login pages.
• The user may have locked the system, but the user is still logged in. This user has applications and processes running in the background even when locked. Attackers could try to add a malicious network card to the system, for example via USB. This network card tries to become the preferred way for the system to access the Internet. If the system uses this network card, attackers can now see the network traffic and try to find sensitive data, even change the data.

As soon as attackers gain access to the system, they can start searching it for information that can be used to further control the attackers’ targets.

Example: social engineering scenario: being helpful

People usually want to be helpful to each other. We like to do nice things for others.

Let’s imagine a scenario in which Eve runs into the reception of a large corporate office with coffee-soaked papers. The receptionist clearly sees Eva’s distress and wonders what’s going on. Eva explains to her that she has a job interview in 5 minutes and badly needs to print out her interview papers.

Eve had prepared a malicious USB key in advance with documents designed to compromise the computers to which it was connected. She hands the receptionist the malicious USB key and asks, smiling, if the receptionist can print the documents for her. This may be all the attackers need to infect a system on the internal network, allowing them to compromise other systems.

Example: social engineering scenario: exploiting fear

People are often afraid of failing or not doing what they have been told. Attackers often use fear to try to get victims to do what the attackers need them to do. For example, they may try to pretend to be a company director and ask for information. Perhaps an update on social media reveals that the director is on holiday and this can be used to stage an attack.

The victim probably does not want to call the principal and since the principal is on vacation, it may be more difficult to verify the information.

Example: social engineering scenario: the reciprocity game

Reciprocity is doing something in return, such as responding to someone doing you a favour.

Considering that someone will hold the door for you to let you into the office building entrance, for this reason, you are likely to want to hold another door for that person to reciprocate. This door may be behind the access control, requiring employees to present their IDs, but to offer the same courtesy in return, you hold the door open. This practice is called tailgating.

Example: social engineering scenario: exploiting curiosity

People are naturally curious. What would you do if you found a USB stick lying on the ground outside an office building? Would you plug it in? What if the USB stick contained a document called “Salary information – current updates”?

An attacker could deliberately dump many malicious USB sticks in the vicinity where employees are staying and hope someone plugs them in.

Documents may contain malicious macros or exploits, or they may simply trick users into taking certain actions to compromise themselves.

If you speak German and are an IT tester or IT automation tester, take a look at our employee benefits and respond to our job offers!