Orthogonal array testing

Orthogonal array testing is a systematic and statistically-driven black-box testing technique employed in the field of software testing.^[1]^[2] This method is particularly valuable in scenarios where the number of inputs to a system is substantial enough to make exhaustive testing impractical.

Overview

Orthogonal array testing works on the premise of selecting a subset of test cases from a large pool of potential inputs. This selection is based on statistical methods to ensure that the chosen subset represents the whole input space. As a result, serious bugs can be identified while the number of tests necessary to do so is greatly reduced.

Benefits

Reduced testing cycle time: By strategically selecting test cases, the testing process becomes more efficient, leading to time savings.
Simplified analysis: The structured nature of orthogonal array testing makes analysis straightforward and less complex.
Balanced test cases: This technique ensures that test cases are well-balanced, making it easier to isolate defects and assess performance.
Cost savings: It offers a significant cost advantage over pair-wise testing, making it an economical choice for testing large-scale software systems.

Cons

Limited applicability: This technique is most effective when the number of inputs is relatively small. In cases with an extremely large number of inputs, it may not be as efficient.
Complex implementation: Properly designing orthogonal arrays requires a good understanding of statistical principles, which may pose a challenge for some testing teams.
May miss specific edge cases: While orthogonal arrays are designed to cover a broad range of scenarios, they may not capture very specific edge cases that could be crucial in certain applications.

Applications

User Interface Testing: Orthogonal array testing is employed to evaluate the user interface of software applications. It aids in identifying interface-related anomalies and inconsistencies.
System Testing: It is used to validate the functionality of entire systems, ensuring that they perform as specified in their requirements.
Regression Testing: Orthogonal array testing is effective in the detection of regressions, ensuring that new updates or modifications do not introduce unintended consequences.
Configuration Testing: This technique is valuable in assessing different configurations of software, ensuring compatibility across diverse environments.
Performance Testing: It can be applied to evaluate the performance characteristics of software systems, helping to identify potential bottlenecks or performance issues.

Principle of Orthogonality

Orthogonal array testing works based on something called orthogonal arrays.^[3] These are organized lists of different factors. When we use them, we make sure that the results we get from each factor are not connected or related. This means each test gives us new and unique information. This way of organizing inputs helps us avoid repeating tests and get the same info with the least number of experiments.

Orthogonal vector

The concept of orthogonal vectors in orthogonal arrays is fundamental to understanding orthogonal array testing. Orthogonal vectors possess key properties:

Unique Information: Each vector imparts information distinct from any other vector in the sequence, thereby avoiding redundancy.
Separability: Through linear addition, the signals can be easily separated.
Statistical Independence: Each vector is statistically independent of the others, signifying a lack of correlation between them.
Resultant Summation: When subjected to linear addition, the resultant is the arithmetic sum of the individual components

References

^ Pressman, Roger S. (2005). Software engineering: a practitioner's approach. McGraw-Hill series in computer science (6th ed.). Boston: McGraw-Hill. ISBN 978-0-07-285318-6.
^ Phadke, Madhav S.; Phadke, Kedar M. (January 2014). "Utilizing design of experiments to reduce IT system testing cost". 2014 Reliability and Maintainability Symposium. IEEE. pp. 1–6. doi:10.1109/rams.2014.6798451. ISBN 978-1-4799-2848-4. S2CID 8081137.
^ "What is Orthogonal Array Testing? (Example)". www.guru99.com. 2023-09-30. Retrieved 2023-10-05.

External links

Rao, Calyampudi Radhakrishna (2009). "Orthogonal arrays". Scholarpedia. 4 (7): 9076. Bibcode:2009SchpJ...4.9076R. doi:10.4249/scholarpedia.9076.
Delius, Gustav W (May 2004). "Orthogonal Arrays (Taguchi Designs)". University of York.
Kuhfeld, Warren F. "Orthogonal Arrays". SAS Institute Inc. SAS provides a catalog of over 117,000 orthogonal arrays.
Phadke, Madhav S. "Planning Efficient Software Tests". Phadke Associates, Inc. Archived from the original on 2012-04-25. Retrieved 2011-10-07. Numerous articles on utilizing Orthogonal Arrays for Software and System Testing.
"rdExpert Software for Orthogonal Array Testing". Phadke Associates, Inc. Archived from the original on 2011-01-14. Commercial toolset for Orthogonal Array Testing.

[1] Pressman, Roger S. (2005). Software engineering: a practitioner's approach. McGraw-Hill series in computer science (6th ed.). Boston: McGraw-Hill. ISBN 978-0-07-285318-6.

[2] Phadke, Madhav S.; Phadke, Kedar M. (January 2014). "Utilizing design of experiments to reduce IT system testing cost". 2014 Reliability and Maintainability Symposium. IEEE. pp. 1–6. doi:10.1109/rams.2014.6798451. ISBN 978-1-4799-2848-4. S2CID 8081137.

[3] "What is Orthogonal Array Testing? (Example)". www.guru99.com. 2023-09-30. Retrieved 2023-10-05.

[1]

[2]

[3]

v t e Software testing
The "box" approach	Black-box testing All-pairs testing Exploratory testing Fuzz testing Model-based testing Scenario testing Grey-box testing White-box testing API testing Mutation testing Static testing
Testing levels	Acceptance testing Integration testing System testing Unit testing
Testing types, techniques, and tactics	A/B testing Benchmark Compatibility testing Concolic testing Concurrent testing Conformance testing Continuous testing Destructive testing Development testing Differential testing Dynamic program analysis Installation testing Negative testing Random testing Regression testing Security testing Smoke testing (software) Software performance testing Stress testing Symbolic execution Test automation Usability testing
See also	Graphical user interface testing Manual testing Orthogonal array testing Pair testing Soak testing Software reliability testing Stress testing Web testing