Software Testing 101
Software testing is a crucial discipline in the broader subject of Software Engineering. It is the systematic process of finding errors during software product development. Surprisingly, software testing is often overlooked, especially in startups, but sometimes in big companies.
Testing is not only about finding bugs. It also helps diagnose gaps in requirements, discover new insights about the user experience and even detect security vulnerabilities. Software testers, or QA Engineers, use both manual and automated types of Testing to ensure the best possible testing coverage. Usually, test results are reported back to the development team in an automated way so defects can be corrected in the next development cycle.
Software testing is a critical activity in delivering high-quality products to clients. This article will summarize the essential points about Software Testing and why it is necessary.
Impact of Software Testing
Organizations that develop software use it to add value to their customers, when the software malfunctions, this value-added chain gets compromised.
Software defects can dramatically affect users, forever impacting a business's reputation. Companies should make cost-effective efforts to detect software defects before affecting actual users.
Software defects can have different severities. Commonly referred to as a "bug," a flaw could be as trivial as a typo in a text or as critical as sensitive information disclosure.
A correctly implemented Software Testing strategy helps companies mitigate the risks above and obtain benefits in the following areas, to name a few:
- Improvements in CX/UX: a properly tested software will, in turn, help the company maintain and improve customer and user experience. A product with fewer nuances will enhance user engagement and lower incident reports.
- Costs: detecting defects as early as possible can dramatically impact overall development costs. While catching bugs in a development cycle can impact scope and development expenses in the short term, the long-term benefits of not moving customers in production are higher.
- Cybersecurity: risks in cybersecurity are the companies-killer of the XXI century. A data breach, a massive information leak, or a critical vulnerability can damage a company's reputation beyond repair. Including security best practices such as penetration testing and software audits as part of the software, testing practices can help mitigate these risks.
Different classifications of Software Testing
After decades of Software Testing practice, the industry has produced many different testing paradigms, frameworks, and methodologies.
Software Testing can be categorized by various criteria; manual or automated, black or white box, functional or non-functional.
Often, companies take a hybrid approach, pragmatically selecting elements of each type of Testing to build their in-house Software Testing practices.
Types of Functional Testing
As the name suggests, a "functional" test focuses on how the software functions, usually but not exclusively from an end-user standpoint.
In turn, there are several subtypes of functional tests. These are among the most popular ones:
- Unit testing: this type of Testing focuses on individual "units" of code and usually takes the form of "expected versus actual output," provided a known set of inputs. The size of a unit varies with each project. In object-oriented software, a unit could be a single class or a cohesive group of classes. In a functional programming environment, it could be a single function or a module that exports a set of functions with a common purpose.
- Integration testing: Integration Tests could be thought of as the next natural step following Unit Testing. After unit tests assert the correct behavior of each unit, an integration test checks if a group of units plays well together. While in Unit Tests, external dependencies are usually mocked, an Integration Test will check the actual interconnection of units.
- Smoke testing: taking its name from electronics hardware testing, a smoke test only verifies the basic behavior. It tests that nothing crashes or blocks a user in the most common functional paths.
- API testing: in the past few years, systems have become increasingly distributed. The most popular form of connection nowadays is REST APIs, although there are many others like web services and even database interfaces. This type of test focuses on testing an interface for expected behavior.
- Regression testing: a regression test, as the name implies, goes back to running every test available before adding new functionality or to a critical deployment. It is a costly test, so it should be automated whenever possible.
- User Acceptance testing: sometimes considered the final type of test, in User Acceptance or UA, final users test the existing system in a production-like environment, like a beta or A/B. If the UA Testing is successful, and previous types of Testing have also passed without failures, then the software is ready for deployment.
Types of Non-Functional Testing
Unlike functional testing, Non-functional tests focus on software attributes that are traversal to all functionality.
Examples of non-functional features are response time, scalability, overall performance, and security.
One of the most famous types of non-functional tests is "stress-testing." The system is tested beyond regular parameters in a Stress Test to find its current limit.
Black-Box vs. White-Box Testing
The terms white and black box refer to the amount of information a test has about its testing system. In a black-box test, nothing is assumed about the system under test. Aspects such as source code and internal structure are ignored.
Usually, black Testing is used to simulate the behavior of a potential user, but this is not to be confused with User Acceptance tests (where users could be, in fact, experts).
On the contrary, in white-box Testing, the internals of the software are under scrutiny. Source code can be audited with a code quality tool like Sonarqube, or security vulnerabilities could be detected by npm, to name a few.
This type of test can help improve code quality and adherence to best practices.
Software Testing processes and methodology
As part of Software Engineering, Software Testing is usually organized as a succession of well-defined tasks.
While this can vary according to the general software development framework adopted by a team (Scrum, Kanban, Waterfall, etc.), it is helpful to keep in mind that critical aspects of software testing should be present in each project.
Test Planning
To write or execute practical tests, it is advisable to gather the required information about the target to be tested (whether it is a unit, a non-functional aspect, or a complete regression).
Correct planning will help execute tests that are useful for the business.
Setup
Depending on the project's scope, architecture, and infrastructure, different testing environments should be set up. It could be anything from a production replica on the cloud to a tester running individual docker containers in their machines.
The important thing in the setup phase is to gather all resources and tools to ensure the execution phase will run as smoothly as possible.
This phase should include decisions about integrating test execution with the available CI/CD pipeline, if available.
Construction
After gathering all required information in the planning phase and setting up necessary environments in the setup phase, professional software testers will now build the needed test cases.
If tests are automated, they will code them in the required scripting tool. If tests are manual, they will be designed and documented so any testing team member can adequately run them and report the results.
Execution
After planning, setup, and construction, it is time to run the tests. Test Cases are executed, and results are gathered for the next phase.
Test execution usually takes assertions about the expected behavior than the actual output when presented with a defined, well-known input.
Reporting and Publishing
The last phase of a software testing pipeline is to gather all test results from the previous step, build reports and publish them to the rest of the team in an appropriate tool.
Developers should be able to easily access test execution information to improve the code as quickly as possible. If a CI/CD automated tool is in use, there should be a step where test reports are made public to the rest of the team.
A note on Agile Methodologies and Software Testing
In the Agile approach (Scrum and Kanban being the most popular), Testing is not a phase that happens "after development." In Agile, the software is developed in cycles called "iterations," Usually, testing is a vital part of each iteration.
Agile methodologies help shorten development cycles, but this should not translate into less Testing.
On the contrary, the correct amount of different types of tests conducted as soon as the first iteration helps lower the overall development cost of almost any given project.
Software Companies that adopt different testing environments should be set up depending on the project's scope, architecture, and infrastructure.
The agile approach does so because it need to meet its stakeholders' and customers' high expectations of their stakeholders and customers.
In this scenario, Software Testing becomes crucial in achieving short-term goals while maintaining the necessary software quality required for the long term.
While not mandatory, an Agile environment usually benefits from automated tools. In these scenarios, software testing should strive to automate as much of their work as possible while maintaining manual operations where human critical thinking is most needed.
Wrap up
Software testing is a crucial aspect of Software Engineering. It consists of the practice of detecting and reporting software defects. Testing is critical in delivering high-quality products to customers that are also secure, scalable, and robust.
Software Tests can be classified using different criteria. A test could be functional or non-functional, white or black-box, automated or manual.
As with any part of Software Engineering, Software Testing is also organized as well-defined activities that include Planning, Design, Environment setup, CI/CD integration, and Reporting.
At Devlane, we host an expert Software Development practice. Does your team need different testing environments that should be set up depending on the project's scope, architecture, and infrastructure to help set up an Agile Software Testing strategy? Contact us here.