We have Black box and white box testing but.. What about Gray Box testing?
We already know the White Box Testing, also known by some of it’s synonyms like Clear/Open/Glass/Transparent Box Testing or Code Based/Structural Testing which is a software testing method where the internal structure and design of the component to be tested is known or at sight of the tester.
Also we know the Black Box Testing, also named Behavioral Testing, which is a software testing method in which the internal structure, design or implementation of the component to be tested is not known or at sight of the tester. (These tests can be functional or non-functional, though usually functional.)
But what about the scenario where the tester can access the component’s code in a limited way or has limited information about the internal functionality of the component?
So the tester knows what methods are used in certain parts of the component when it is executed.
This, by definition is the Grey Box Testing.
So Grey Box Testing is a software testing method performed with limited information about the internal functionality of the components.
Grey Box Testers usually have some access to the detailed documentation along with information about requirements.
The purpose of this testing is to search for defects due to improper code structure or improper functioning usage of an application.
One of the main characteristics of the Gray Box Testing is that it increases the testing coverage by concentrating on all of the layers of any complex system.
As we can see Gray Box Testing gathers White Box Testing and Black Box Testing properties and gives the ability to test both sides of an application: the presentation layer and some of the code implementation. This is useful in Integration and Penetration Testing.
A quick example of this testing technique is the webpage link section testing. When the tester is trying to access the published links, if some inaccessible links were found then some quick changes can be made within the HTML code that allow a real time check.
Which testing method is the best: White box, Black box or Grey box?
As the reader can surely answer, it depends on the situation and resources are at hand. This means code accessibility, developing knowledge/skills, UI focus, etc.
In a close to perfect world the tester has full access to the code and understands it perfectly well, so if we want to approach this we realize that Gray Box Testing is a combination of White Box and Black Box, the best approach.
So digging deeper we find that Gray Box Testing has also this characteristics:
- The testing process is given from the user’s side more than from the designers or the developers side.
- Gives more agility reducing the time of doing functional and non-functional testing.
- It provides more feedback information to developers about found issues .
- It is a combination of testers and developers input and aims to improve the final product quality.
Strategies
For performing Gray Box Tests the tester does not need the full access to the source code, just enough to know the algorithms and how they work, which are their architectures and/or other high-level descriptions of the program behavior.
The the test cases are designed with a Black Box Testing technique and then preset all the conditions to do a test by assertion methods.
Some Grey Box testing methods:
- Regression testing: re-running functional and non-functional tests to ensure that previously developed and tested components still perform after a change.
This type of testing is carried out after executing a functional development or repair to the program. To verify whether the modification in any of the previous version of the software has regressed or caused any unintended or adverse side effect in other aspects of the program in the new version, the following testing strategies can be pursued:
- Retesting within a firewall where dependencies are analyzed for choosing baseline tests.
- Retesting risky use cases where the risk factor is considered.
- Retesting all existing test cases.
- Retesting by profile where time is allocated in proportion to the operational profile.
- Retesting changed segment where code changes are compared for choosing baseline tests.
At some stage in confirmation testing, if any defect got rectified, and that part of the software started functioning as intended, there might be a possibility that the rectified defect may have initiated a different defect somewhere else in the software.
Here, regression testing takes care of these types of defects by utilizing the above-mentioned testing strategies.
The tester, as a reference, may use 80% of the allowed time to run existing test cases and 20% of the allowed time to execute exploratory testing.
- Matrix Testing: this is a technique that defines all the used variables of a particular program. In any program, variables are the essential elements through which values can move through the program.
It should be on par with the requirement without which the readability of the program and speed of the software will be reduced.
Matrix technique is a way to eliminate uninitialized and unused variables by identifying used variables from the program.
Examination of inherent risks like technical risks and business risks that are associated with the variables with different frequencies labeled by the software developer is carried out under this type of testing.
- Orthogonal Array Testing or OAT: is a black box testing technique that is a systematic, statistical way of software testing. It is used when the number of inputs to the system is relatively small, but too large to allow for exhaustive testing of every possible input to the systems providing a maximum code coverage with a minimum test cases.
For example complex applications and e-comm products can be tested with this technique. Orthogonal Array Testing is composed of an array of values in which a variable is represented in each column of a table and a test case is represented in each row. It is a really simple technique but helps in maximizing the required testing coverage.
- Pattern Testing: based on the historical data of the previous system defects but unlike black box testing, gray box testing goes deeper within the code and tries to determine why the failure occurred
These analyses may contain specific reasons for the defect or bug with information on the problem that has been addressed, applicable situation, generic test cases, etc.
Unlike black box testing, grey box testing plows within the code to determine the reason for the failure so that they can be fixed in the next version.
It is noteworthy that pattern testing is applicable only to such type of software that has been developed by following the same pattern of previous software as the possibility of similar defects occurs in this software only.
Generally, Grey box methodology uses automation tools to conduct the testing. Module drivers and stubs are created to relieve the tester from manually generating the code.
How to Start using the Gray Box Testing?
Here are the 10 tasks required to do a Grey Box test:
1.- Identify inputs
2.- Identify the outputs
3.- Identify the major paths
4.- Identify Subfunctions
5.- Develop inputs for Subfunctions
6.- Develop outputs for Subfunctions
7.- Execute test case for Subfunctions
8.- Verify the correct result for Subfunctions
9.- Repeat steps 4 & 8 for other Subfunctions
10.- Repeat steps 7 & 8 for other Subfunctions
The test cases for grey box testing may include, GUI, Security, Database, Browser, Operational system related, etc.
Pros & Cons of the Method
Pros
- It is born directly from functional specifications or UML diagrams.
- Testers can design improved testing environments.
- Gray Box testing keeps a boundary between testers and developers.
Cons
- White box testing is not possible because of non having full accessibility to the source code.
- Grey-box testing is not suitable for a distributed system for being difficult to associate the found issues.
- If a component under test has issues they can lead to abortion of the ongoing operation
- Tests can be executed completely but with incorrect results.
Where you can use the Grey Box Testing?
- Gray-box testing is well suited for web applications. Web applications have distributed networks or systems; due to absence of source code or binaries it is not possible to use white-box testing. Black-box testing is also not used due to just contract between customer and developer, so it is more efficient to use gray-box testing as significant information is available in Web Services Description Language WSDL)
- Gray-box testing is suited for functional or business domain testing. Functional testing is basically a test of user interactions with external systems. Gray-box testing is well-suited for functional testing due to its characteristics; it also helps to confirm that software meets the requirements defined for the software
What’s in the Future?
The distributed nature of Web Services, allows gray-box testing to detect defects within a Service Oriented Architecture or SOA. As we know, white-box testing is not suitable for Web services as it deals directly with the internal structures.
White-box testing can be used for state art methods; for example, message change which generates the automatic tests for large arrays to help exception handling states, flow without source code or binaries.
Such a strategy is useful to push gray-box testing nearer to the outcomes of white-box testing.
Need someone to test your mobile or web apps?
Get to know everything our company has to offer to your organization over at our website.