Tosca tester
Negative testing: types, advantages and practical examples
While it is certainly important to verify that the software performs its basic functions as intended, it is equally important to verify that the software is capable of dealing effectively with unusual situations. Obviously, most bugs occur when testers use their creativity to create such situations.
Most of us are already familiar with different types of testing, such as functional testing, integration testing, regression testing, alpha and beta testing, accessibility testing, and so on. However, everyone will agree that no matter what category of testing you do, the entire testing effort can basically be generalized into two categories: positive testing paths (happy paths) and negative testing paths (negative paths).
Negative testing
Negative testing focuses on finding potential bugs, errors or vulnerabilities that positive testing (using valid input) may not have detected. Negative testing is commonly referred to as faulty path testing or failure testing. It is usually performed to ensure application stability.
Negative testing is a process where as much creativity as possible is applied and the application is validated against invalid inputs. That is, its purpose is to check whether errors are displayed to the user where they should be, or whether the wrong value is handled more efficiently. It is important to understand why negative testing is necessary.
The functional reliability of an application or software can only be quantified using effectively designed negative scenarios. The aim of negative testing is not only to uncover potential bugs that could have a serious impact on the use of the product as a whole, but it can also be instrumental in determining the conditions under which an application may crash. Finally, it ensures that there is sufficient validation of bugs in the software.
Purpose of negative testing
The purpose of negative testing is to evaluate the behaviour of a software system when exposed to invalid or unexpected inputs.
Here are some of the main objectives that negative testing can help software developers and testers achieve:
- It acts as a critical component of quality assurance, helping to verify that the system behaves as expected and meets customer requirements.
- It ensures that the system behaves gracefully when faced with invalid inputs, providing a better user experience and reducing the risk of user frustration or system crashes.
- It helps to identify and fix stability problems that may not have occurred during positive testing.
- You can use negative testing to check the robustness of the software by testing how it responds to invalid or unexpected input.
Negative testing types
There are several types of negative testing including:
- Boundary value testing: An effective approach to testing software applications is to expose them to inputs at the extreme limits of the input values. Evaluating these outliers can easily reveal potential underlying problems in the application, especially at the boundaries of acceptable inputs. This involves testing the system with inputs that are at the minimum or maximum limits of the input range.
- Input validation testing: Another form of negative testing involves testing the ability of a software application to handle invalid inputs, including invalid data types, out-of-range values, and even the presence of invalid characters. This form of testing then allows for a comprehensive assessment of the application’s ability to reject and identify input.
- Exception testing: Exception testing is critical to the evaluation of software applications. This type of negative testing focuses on examining an application’s ability to effectively handle exceptional conditions, including scenarios such as missing data, unexpected device/application shutdowns, and other exceptional events.
- Load testing: Software applications are subjected to load testing to ensure smooth functionality. The goal of these tests is to evaluate the performance of the application under heavy use and load conditions, such as high traffic, heavy data processing, and other resource-intensive scenarios. By performing stress testing, potential system failures or performance issues can be quickly identified and addressed.
- Compatibility testing: Compatibility testing is at the heart of complex testing. This special form of negative testing thoroughly examines the compatibility of a software application with different hardware, network configurations and software settings. Through this testing, developers can ensure that the application will work seamlessly in a variety of environments, thereby protecting themselves from potential compatibility issues.
Negative test case design
- Negative test case design involves identifying scenarios in which a system or application should not perform as expected, and testing it with invalid, unexpected or incorrect input to verify that it handles these scenarios correctly.
- Examine system requirements, use cases and features to identify potential scenarios where the system may not perform as expected.
- For each negative scenario, determine what the expected outcome should be. For example, an error message or a system shutdown.
- Based on the identified negative scenarios, select the inputs that trigger the negative behaviour. These could be invalid data, incorrect input values or unexpected inputs.
- Create test cases that describe the steps to be taken to perform the negative tests. Test cases should include inputs, expected results and any other relevant information.
Negative testing step by step
The following are the basic steps in performing a negative test:
- Identify the specific feature or functionality to be tested, such as data validation, error handling or security checks.
- Define the criteria for a negative test, which deliberately causes the software to crash or produce unexpected results. This could include entering invalid data, exceeding limits or performing activities out of sequence.
- Create a test case that contains the steps necessary to reproduce a negative test, including the inputs or actions that trigger the negative scenario and the expected results.
- Set up a test environment to ensure that the system is in a state where a negative scenario can be executed. This may require specific data or configurations.
- Perform the negative test following the steps outlined in the test case to reproduce the negative scenario.
- Observe and document the test results and compare the actual result with the expected result given in the negative test case.
- Report any errors or problems encountered during the test, providing a detailed description of the problem, the steps required to reproduce it, and any relevant logs or error messages.
- If necessary, repeat the test with different input data or test case variations to further explore possible failure scenarios.
Negative testing examples
Take the case of the lift. When you press the floor number, the lift will take you to a particular floor and the doors will open automatically when the lift reaches that floor.
- What happens if the number of people or weight exceeds the limit?
- What happens if someone starts a fire or smokes in the lift?
- What happens if there is a power failure?
All of these cases fall under negative testing. You cannot ensure that all of them will not happen, so you need to limit them.
In addition to the lift example, we have several examples of negative testing related to software:
- Test the ecommerce site’s payment gateway: enter invalid credit card information or try to exceed the maximum order value to verify that the site handles these scenarios correctly and prevents fraudulent transactions.
- Test the GPS navigation system: when the tester enters an incorrect address or attempts to use the system in an area with limited or no GPS coverage, to verify that the system handles these scenarios correctly and provides clear error messages.
- Test the website login page: by entering incorrect username and password combinations, to check that the website displays the appropriate error message and prevents unauthorized access.
- Test the cash machine: Try to withdraw more money than you have in your account or insert an invalid card to verify that the cash dispenser handles these scenarios correctly and prevents unauthorized transactions.
- Test the camera function of the mobile application: take a photo in low light or without enough memory on your device to check that the application handles these scenarios correctly.
Negative test cases can also be divided into frontend and backend cases:
Frontend negative test scenarios
- Create a user with an email address that already exists in the system/db.
- Login with an incorrect password.
- Check if the First Name and Last Name fields accept special characters, numbers, and emoticons.
- Check if the Telephone Number field accepts letters.
- Check if the user can complete the order with an expired credit card.
Negative backend test scenarios
- Login without entering a user token.
- Login with an expired token.
- Login with User 1 using User 2’s token.
. - Complete an order with a user who has no payment methods saved.
A practical example of automating negative testing
The user is expected to enter a name as shown below:
Let’s set some ground rules to make sure we design good negative scenarios.
Requirements:
- The Name text box is a required parameter.
- The description is optional.
- The name field can only contain the characters a-z and A-Z. No numbers or special characters are allowed.
- The name can be a maximum of 10 characters.
Example of positive test cases:
- ABCDEFGH (uppercase validation within character limit),
- abcdefgh lowercase validation within the character limit),
- aabbccddmn (character limit validation),
- aDBcefz (uppercase validation combined with lowercase validation within the character limit).
Negative test cases:
- ABCDEFGHJKIOOOOOKIsns (name longer than 10 characters),
- abcd1234 (name with numeric values),
- no name is specified,
- sndddwwww_ ( a name containing special characters).
Example of automating negative testing in Selenium using Java:
**import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions; import org.openqa.selenium.support.ui.WebDriverWait;
public class NegativeIntegrationTest {
public static void main(String[] args) throws InterruptedException {
System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver");
WebDriver driver = new ChromeDriver();
driver.get("[https://example.com/](https://example.com/): [https://example.com/](https://example.com/)");
// Test povinného poľa s názvom
WebElement nameInput = driver.findElement(By.id("Name"));
nameInput.clear();
driver.findElement(By.id("submit")).click();
WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("error-name")));
String errorText = driver.findElement(By.id("error-name")).getText();
assertEquals(errorText, "Pole názov je povinné a nesmie byť prázdne.");
// Test nepovolených špeciálnych znakov v poli s názvom
nameInput.sendKeys("Meno obsahujúce diakritiku – á,é,í,ó,ú,č,š,ľ,ž..");
driver.findElement(By.id("submit")).click();
errorText = driver.findElement(By.id("error-name")).getText();
assertEquals(errorText, "Pole názov môže obsahovať len znaky a-z a A-Z.");
// Test prekročenia maximálnej dĺžky názvu
nameInput.clear();
nameInput.sendKeys("ABCDEFGHJKIOOOOOKIsns");
driver.findElement(By.id("submit")).click();
errorText = driver.findElement(By.id("error-name")).getText();
assertEquals(errorText, "Pole názov môže mať maximálne 10 znakov.");
// Test názvu s číselnými hodnotami
nameInput.clear();
nameInput.sendKeys("abcd1234");
driver.findElement(By.id("submit")).click();
errorText = driver.findElement(By.id("error-name")).getText();
assertEquals(errorText, "Pole názov môže obsahovať len znaky a-z a A-Z, nie číselné znaky.");
}
private static void assertEquals(String expected, String actual) {
if (!expected.equals(actual)) {
throw new AssertionError("Expected: " + expected + ", Actual: " + actual);
}
}
}Negative testing advantages
- Negative testing helps to identify if the system behaves in an unexpected way, for example, if it breaks down or produces incorrect results when it receives invalid or unexpected input.
- By exposing weaknesses in the system, negative testing can help improve the overall quality of the software and increase its resilience.
- It helps to identify security vulnerabilities in the system, for example, by exposing weaknesses in the input authentication process.
- By testing the system under adverse conditions, negative testing increases confidence in the ability of the system to function properly in real-world scenarios.
- Negative testing gives the customer more confidence before going live.
- Testing with invalid data can help distinguish between valid and invalid data and determine which ones are valid. This means that invalid data can be isolated, updated accordingly, and possibly largely deleted from the project database.
Negative testing disadvantages
- Negative testing can be time-consuming, as it often requires extensive testing of edge cases and invalid inputs.
- Automating negative testing can be challenging because it requires the creation of test cases that accurately represent invalid inputs.
- Negative testing can only cover a limited set of potential negative scenarios, and it may not be possible to test for all possible invalid inputs.
- It can also be resource intensive, as it may require additional hardware or software to generate and manage invalid inputs.
- Negative testing also causes unnecessary release delays and increases costs for the customer.
- There is a chance that testers will spend a lot of time and energy on negative testing, resulting in less focus on positive testing.
Negative testing best practices
Here are some of the best practices for negative testing:
- Carefully plan your negative testing strategy and determine which scenarios and inputs will be tested.
- Clearly define the aims and objectives of your negative testing and what you hope to achieve by carrying out negative testing.
- Test the edge cases and boundary conditions of the system where invalid inputs are most likely to occur.
- Automate the negative testing process wherever possible to increase efficiency and reduce the risk of human error.
- Where possible, use real-world data to generate invalid inputs to better represent the types of inputs users are likely to encounter.
- Create a separate folder within the project for negative test scenarios. Although sometimes negative test scenarios may be part of the acceptance criteria from the user story, creating test cases in a separate folder within the same project/directory is a good practice as it makes them easier and quicker to access.
Conclusion
Negative testing is an important part of software quality assurance, helping to find bugs, vulnerabilities and security risks. Although it is time and resource consuming, it provides significant improvements in software stability and resilience, thereby increasing user confidence. Proper planning and a creative approach to negative testing are essential to achieving reliable and robust software.
