What is the Underlying Logic of the Software Testing? （Part 1）

What is the underlying logic?

According to teacher Liu Run's explanation:
“The common ground between things is the underlying logic.

The underlying logic is something that has only the similarities among the differences and remains unchanged behind the changes.
......
Underlying Logic + Environment Variable = Methodology”

He also said: "Only the underlying logic is alive."

So, let's explore: what is the underlying logic of software testing?

1. Basic Understanding of Software Testing

The basic understanding of software testing makes us reach a consensus. Based on this consensus, it is easier to discuss the underlying logic of software testing. The basic understanding of software testing is described in one sentence, that is, grasping the essence of software testing and describing the correct software testing values in a concise way. Instead of an individual’s software testing values, it’s the values that can be accepted by most people.

In Chapter 1 of the Full Software Testing (Version 3), we discussed in depth the cognition of software testing,

Software testing is to verify whether the functional characteristics of software meet the requirements;
Software testing is to find the defects in the software;
Software testing includes static testing – requirement, design, and code review
Software testing is to systematically and completely evaluate the quality of software products and provide quality information;
Software testing is to expose and reveal product quality risks;
Software testing is not only a technical activity but also a comprehensive social and psychological activity;
Software testing is to greatly reduce the cost of poor quality by investing in quality assurance costs.

According to the listed above, the basic cognition of software testing is explained in one sentence, that is: based on the understanding of the real needs of users, to obtain the true and comprehensive quality information of software products through various means. Whether it is to verify whether the software features meet the requirements, evaluate the quality of the product or reveal the quality risk of the product, the judgment is made based on the real quality information obtained about the product, and the defect can be regarded as a by-product in the process of this activity.

It emphasizes the understanding of the real user requirements here. On the one hand, it reflects that "there is no quality without users, and quality exists relative to users." We must complete the test from the perspective of users. On the other hand, it is the real user requirements, not false and wrong requirements. The requirements of the business should eventually be decomposed into the requirements of user roles, and the functional/non-functional requirements of the system are also to meet the user requirements;
The "software product" mentioned here is not limited to programs but also includes data, requirements documents, design documents, codes, user manuals, technical manuals, etc.

After understanding "what is software testing", we can discuss the underlying logic of software testing.

Source: FreePik

2. The Underlying Logic of the Software Testing

The underlying logic of software testing can be summarized as the answers to three questions:

Why？
What？
How？

Moreover, in the process of answering these three questions, we should be able to adapt to different test objects (such as Windows / MacOS native applications, web software, mobile app, and embedded software), different test types (such as function test, performance test, security test, compatibility test, etc.), different test levels (such as unit test, integration test, system test, etc.), and different teams and different products. So, it becomes a universal answer. Although there are different test methods, technologies, and practices in different contexts, we can abstract their common ground.

Based on this consideration, let's answer these three basic questions:

Why? As long as the work is done by humans, there is no guarantee of infallibility and there will be problems. If the software goes out with problems, it is likely to bring losses to customers or make customers dissatisfied, eventually leading to damage to the interests of the enterprise. Numerous quality accidents in the past have proved this. Before delivering to customers, the software needs to be fully tested. Otherwise, there are serious consequences.
What? It depends on the quality objectives of delivery, that is, starting from the quality objectives, the objectives are decomposed, and then the quality data to be obtained about the test object is determined for each special sub-objective, so as to determine the scope of the test or the test items. If we go further, we determine the priority of the test item according to the user's different feelings about the quality characteristics and functional features. This section belongs to the work of test analysis and involves testing risks and testing strategies.
How? We find ways, methods, or means to obtain the quality data of the tested object, including test scheme design, scenario design, test case, or test data design.

That is, For Quality, from Quality objectives and by getting Quality data.

Source: FreePik

I have written the article With Three Questions About the Soul of Software Testing and how to Answer them? Does the answer to the "Three Soul Questions" also reflect the underlying logic of software testing?

Q1: Why can't the bug be detected?
Q2: How to test? Are the testers professional enough?
Q3: Quickly! Why do the testers always hinder the whole team and report bugs at the last?

In fact, it also reflects another layer of the logic of "software testing", which is:

The answer to Q1 presents the underlying logic: There is no testing without an end, testing is always full of risks, and the more bugs the developer writes, the more bugs the test misses. We can only prove that the defects that have been found are defects, instead of proving that there are no defects in the software. Because the test is a sample experiment.
The answer to Q2 presents the underlying logic: The test work (including the formulation of test objectives, the process of test analysis and the corresponding test design method) can be explained clearly, and the test is not an isolated work. Due to the influence of requirements (such as fuzzy requirements), system design (such as coupling and complexity), and programming (such as secretly modifying code), the test should cooperate closely with products and development.
The answer to Q3 presents the underlying logic: We can complete test analysis, test planning, and test design before developers finish the code writing. However, to really run the system test we need to wait until the developers finish the builds. Test execution is a later-stage work, and the test time is easy to be squeezed out by the early-stage work of development. Therefore, the delay of the project is easy to cause the illusion that testing is lagging behind.

The underlying logic of the test (probabilistic thinking): A test is a sample experiment that requires careful analysis and design to reveal quality risks at the earliest possible opportunity with minimal effort. Since it is a sample experiment, the distribution of defects is normally distributed, and the quality can be improved from 3sigma to 6sigma, but it will never reach 100%.