Exploratory testing

Exploratory testing definition

Exploratory testing is a form of software testing in which the tester can choose any possible software testing methodology. This is an unwritten approach to software testing. In exploratory testing, software testers use their insights, knowledge, skills and abilities.

Exploratory testing verifies the functionality and operation of the software as well as identifies functional and technical bugs in the software. The goal of exploratory testing is to optimize and improve the software in every possible way. It is often performed as a black-box testing technique.

Why exploratory testing?

In scripted testing, you first design the test cases and then proceed to execute them. In contrast, exploratory testing is a simultaneous process of designing and executing tests that are performed simultaneously. Unscripted test execution is usually an unplanned activity in which testers execute test steps and compare the actual results with the expected ones.

When should exploratory testing be performed ?

• If a new tester joins the team and needs to quickly get familiar with the application and provide quick feedback.
• It is appropriate when products need to be reviewed from the user’s perspective.
• Exploratory testing is useful in scenarios where early iteration is needed because teams don’t have a lot of time to structure test cases.
• Exploratory testing ensures that the tester does not miss edge cases that can lead to critical quality failures.
• Exploratory testing can be used to help with unit tests, document test cases and use test cases for extensive testing during later sprints.
• Critical test cases have been tested and exploratory testing is not performed as a substitute for legal or safety requirements.
• Exploratory testing seems to be more effective in finding bugs than other types of tests.
• You don’t have written and prepared test cases and want to test your product without them.

Source: guru99.com

Differences between scripted and exploratory testing

Exploratory testing types

1. Active Exploratory Testing

Active exploratory testing involves a tester designing an automated script for checking the tests to be executed by another tester. These scripts take into account previous tests where it is appropriate. The two testers usually swap tasks during the control procedure to double-check the reliability of these scripts and processes.

Active tests have wider coverage. These scripts also provide better documentation, making it easier to reproduce any issues that testers find.

2. Passive Exploratory Testing

Passive exploratory testing requires only one tester, although working in pairs could make the process even more efficient. This approach involves specific software that records the activities of the tester – providing them with simple steps to repeat any problem they detect. Usually this is in the form of a video with a tester’s commentary explaining operation step by step.

Recording the testing process also provides insight into the performance of the application, including how quickly it responds to input requests. Passive testing provides both testers and the development team with a wealth of detailed information about how the software works.

Exploratory testing techniques

Exploratory testing usually takes the form of a “walkthrough” – the tester examines the software in the most efficient way possible. There is a variety of walkthroughs from which the team can choose, including:

Guided tours (guidebook tours)

This approach favours main application functionality, replicating exactly the way a normal user interacts with the software and uncovering problems that they would naturally find.

History tours

This checks the oldest features of the app to make sure they are still functional, this is especially important if developers have added new features that conflict with it.

Money tour

This exploratory test checks critical application features, specifically those that customers and clients pay money to access – these are usually the highest priority for the test team.

Crime spree tour

Testers sometimes actively work to break the application or trigger negative scenarios, for example, by entering invalid information and seeing how the application responds.

Back alley tour

This process involves features that are likely to be used by fewer customers, these are equally important to any approach to testing, especially as they will interact with other features.

Intellectual tour

This walkthrough pushes the application further, testing the most complex functions with higher (sometimes maximum) values to determine the processing speed of the software.

Exploratory testing approach

1. Freestyle exploratory testing

Freestyle testing involves an ad hoc approach to control the application. It has few rules to consider, so its effectiveness can vary; some software and components require a more robust methodology.

2. Scenario-based exploratory testing

Scenario-based testing uses real-world situations as the basis for each test, for example by validating inputs that users are likely to make during typical operation of the software. Testers strive to ensure that each scenario they design matches how the user interacts with the application.

3. Strategy-based exploratory testing

Strategy-based testing encompasses a wide range of specific methods, including threshold testing, equivalence techniques, risk-based techniques and others. This generally favors testers who are already familiar with the application because they can create tailored strategies that incorporate these individual methods.

The strategy-based approach focuses primarily on the functionality of the software (and its inner functionality) without addressing the possible scenarios that could lead the user to the problems that occur.

How to perform exploratory test

The following is a step-by-step procedure on how to perform exploratory testing, which is also called session-based test management:

1) Creating a bug classification

• Categorise common types of errors found in past projects.
• Analyze root cause analysis of problems or errors.
• Find risks and create ideas for testing the app.

2) Exploratory testing charter

Testing charter should suggest:

1. what is to be tested,
2. how it can be tested,
3. what needs to be investigated.

Ideas for testing are the starting point for exploring testing. The test charter helps determine how the end user might use the system.

Template for testing charter can be found here – exploratory testing charter template

3) Time range

• This method involves a pair of testers working together for at least 90 minutes.
• There should be no interruptions during these 90 minutes.
• The time box can be extended or shortened by 45 minutes.
• This session encourages testers to react to the system’s response and prepare for the correct outcome.

4) Review of results

• Error evaluation,
• lessons from testing,
• analysis of coverage areas.

5) Debriefing

• Compilation of output results,
• comparison of results with the chart,
• checking whether further testing is needed.

Exploratory testing best practices

During the exploratory implementation, the following steps need to be taken:

• The mission of testing should be very clear.
• Keep notes on what needs to be tested, why it needs to be tested and an assessment of the quality of the product.
• Follow up on questions and issues raised during exploratory testing.
• It is better to create pairs of testers for efficient testing.
• The more we test, the more likely we are to execute the correct test cases for the desired scenarios.
• Splitting the software into modules for easier testing and more efficient test coverage.
• Create a checklist of all the features to be tested so you don’t miss any.
• Start with testing basic user scenarios. As testing progresses, expand it with additional features to be tested.
• Check the graphical user interface against the standards in the test charter.
• Record the problems that occurred during the tests.

Benefits of exploratory testing

• This testing is useful when requirements documents are unavailable or partially available.
• It reveals bugs that are commonly ignored by other testing techniques.
• It helps to expand the imagination of testers by executing more and more test cases, which ultimately increases productivity as well.
• This testing penetrates into the smallest part of the application and covers all requirements.
• This testing covers all types of testing and includes a variety of scenarios and cases.
• Generating new ideas while executing tests.
• In exploratory testing, testers can annotate bugs, add assertions and voice annotations to turn a user story into a test case.
• It helps to formalize findings and document them automatically. Everyone can participate in exploratory testing with visual feedback, allowing the team to adapt quickly to changes and facilitating an agile workflow.
• Integrating exploratory testing with tools like Jira allows recorded documentation to be directly exported to test cases.

Disadvantages of exploratory testing

• This testing depends solely on the skills and knowledge of the tester.
• Due to the ad-hoc nature of testing in exploratory testing, the tests are executed randomly and are therefore not suitable for prolonged implementation and it is not possible to repeat the same testing methodology.

Exploratory testing example

ConnectX, a (fictitious) social media platform, decided to perform an exploratory test to ensure the quality and reliability of its post features. ConnectX scheduled an exploratory test during the development phase of a major application update to ensure comprehensive test case coverage for regression testing and new feature testing.

The assigned tester performed the exploratory test by logging into their test account and navigating to the home page. He explored the options available for creating a new post and tested different combinations of text and media content. He paid attention to how the app handles different scenarios, such as uploading images, videos or links. In addition, he tested the privacy settings and verified that they were being used correctly.

The tester reported several errors, including:

• cases where uploaded images were not displayed on Android phones that were held horizontally,
• cases where privacy settings were not applied correctly because you could see other users private emails when you hit the “check” button on one of their feed posts.

ConnectX was able to resolve the issues before releasing the app update to its users. The result was an improved user experience. The tests found valuable bugs that did not exist inside the test cases and would not have appeared in a test case-only approach.

Exploratory testing tools

Debugging tools: debugging tools can help testers more quickly and efficiently identify and isolate bugs during exploratory testing.

Record and playback: these tools allow testers to record actions or interactions with the app and then play them back to repeat the same behaviour. This can be useful in automated regression testing or in creating reproducible test cases.

1) Xray Exploratory App (XEA)

The Xray Exploratory App (XEA) simplifies the exploratory testing process by giving you the tools to gather evidence and easily share your findings. XEA makes the documentation process easier so you can cover more untested code, discover new testing scenarios and find hard-to-discover bugs.

2) Bug Magnet

Bug Magnet is a session tester for Chrome and Firefox. The tool will help you include common problematic values and edge cases. It is one of the best exploratory tools that provides easy access during exploratory testing sessions.

3) Exploratory Testing Chrome Extension

This is a Chrome extension used for exploratory web testing. This extension reports bugs, ideas, comments. It can take screenshots, trace URLs automatically, export testing sessions in JSON, CSV or HTML format.

4) qTest Explorer

qTest is an easy-to-use, scalable test management solution that enables testers to centralize and accelerate test management. The qTest exploratory testing tool simplifies and streamlines every step of the QA process.

5) Azure Test Plans

Azure Test Plans is a key component of Microsoft’s Azure platform and allows testers to collect rich data in many scenarios. This option is suitable for both desktop and web applications while providing end-to-end traceability that has a clear record of software development.

However, this approach often requires deeper integration with Azure and therefore comes at the expense of flexibility.

Conclusion

Exploratory testing has a lot to offer software development companies of all kinds, as it complements scripted testing and many other controls of software.

With exploratory testing, QA teams can test applications at a higher level, improving the quality of the resulting software and helping developers to fix any bugs, so we don’t recommend ignoring it.

If you speak German and are looking for a job as an IT tester, take a look at our employee benefits and respond to the latest job offers.

About the author

Michaela Kojnoková

Agile Test Engineer

Po štúdiu informatiky na ŽU a TUKE som sa najviac ponorila do oblasti automatizácie testovania. Okrem toho sa venujem tvorbe webov, databázam, dátovej analytike, umelej inteligencii a strojovému učeniu. Mám rada cestovanie, šport a najviac si užívam čas strávený v prírode s mojimi blízkymi. LinkedIn