When is it available for automated testing?

What is Test Automation?

Historically, automation arose to reduce the human effort required for activities of programmable systems and mechanical operations. The purpose is to simplify the heavy, repetitive, and complex work and make it more effective and efficient. In this way, it is possible to save energy, time, and cost while freeing up time to focus on other tasks.

In software development, the same method can be used to automate the work that should have been completed manually. The steps followed by humans are translated into repeatable scripts to focus on other specific tasks that provide more excellent value and reduce execution times. In some cases, automation enables us to perform tests that cannot be performed manually. Especially considering the limitation of execution times in a specific period, automation can more easily complete large-scale and engineering tests.

When testers think about automation, one of the most common questions is, "when is something automatable?"

Knowing whether it should be automated, including evaluating potential investments, methods, benefits, and most importantly, evaluating information about current manual processes.

First of all, we should fully understand the manual process and know it like the palm of our hand. Only in this way can we realize automation. Complete knowledge of the manual process is the pillar of knowing when something is automatable, which means manual testing cannot be completely replaced. Before you can automate, you must be very proficient in manual testing. Learn how to walk, then run.

Automation misunderstandings

Automation has advantages and disadvantages, depending on the project, time, cost, quality, and methodology. Based on the above, another essential point is that in addition to automation or non-automation, we must also understand the specific situation. Everything we do is based on selecting and applying appropriate methods, tools, and skills to achieve our goals in the best way. Avoid the following common misunderstandings about test automation:

      -       Any software can be automated
      -       Automated software quality is better
      -       Automated testing is better than manual testing
      -       Automation delivers a faster return on investment

Whether it's manual or automatic, you might as well understand the objectives of the test according to the following seven principles:

      -       The value of any practice depends on its background;

      -       There is no "good practice," but good practice will be found in the context;

      -       Human cooperation is the most crucial part of all project environments;

      -       Projects are not static and often take unpredictable paths;

      -       The product is a solution. If the problem is not solved, the product will not work;

      -       Good software testing is a challenging intellectual process;

      -       Only through the judgment and skills of collaborative practice in the whole project can we do the right thing at the right time and test products effectively.

These principles were put forward by Cem Kaner, James Bach and Brett Pettichord in their book, Lessons Learned in Software Testing: A Context-Driven Approach, which helps us recognize the importance of the ability to adapt according to the current project situation.

Manual and automatic

At first, we might want to automate everything, but the cost of developing and maintaining automated test scripts is not easy.

When a project bet on automation, ideally, it should have a solid foundation starting with the unit test cases, preventing as many bugs as possible through immediate feedback, and then continuing to different levels. In this way, manual and exploratory testing are most valuable at the UI level, focusing on tests that cannot be automated.

Michael Cohen's test automation pyramid explains this concept:

                                                                                           _{Source: text automation pyramid}

On the left, you can see how automation is usually completed. On the right is the ideal automation test, in which unit test accounts for the most significant proportion in the pyramid.

Although there are differences between automatic and manual testing, they are not mutually exclusive but are regarded as complementary tasks to seek better software quality.

If you consider the return on investment of testing, manually testing new functions can quickly learn more about the application at a lower cost. With the acquisition of knowledge and the increase of test inventory, the cost of manual testing will also increase. On the other hand, automation has a higher initial cost, which decreases as it progresses. This trend is shown in the figure below:

                                                                                       ^{Source：graph}

Analyzing this, we see that automation has a lot of initial investment until the "breakthrough point." Compared with manual testing, we begin to see its positive impact on long-term costs. It can also be clear that the two testing activities are fully compatible and produce short-term and long-term benefits.

By the above argument, the importance and advantages of automation can be realized, and the use cases that can be automated must be identified. Therefore, the goal must be considered pursued and what level this goal is in the test pyramid.

Try to answer the following questions:

What is the goal?

The first thing we need to confirm is always targeted higher levels of software quality and analyze whether automation is suitable for the project. To answer this question, a feasibility analysis of the target is recommended.

Here are some of the scenarios most likely for automation:

      -       There is a technical debt that needs to be eliminated.

      -       Regression testing is very time-consuming.

      -       The project is very complex and long-term.

What use should cases be automated?

Not everything can be automated in context, which is the relevant reason to understand which cases fit our purpose. Unit testing is the easiest to script from the code level and developers. We are generally committed to automating regression cases at the UI and API levels, first considering the most critical and most complex processes on the tester's side.
The following are the automated test cases:

Regression testing

Owing to we already have a test suite that has to be performed regularly after each product release, the work of manually running these kits becomes repeated, plus taking time from other unautomated tasks to gain more value. These regression test cases are highly automated and convenient for integration into the CI / CD models. This increases the cost and time required to perform other tasks, as the scripts can be performed unattended when performing other activities.

High - risk test

Stakeholders often agree upon these cases, focusing on examining high priority and critical functions that, if they fail, will significantly impact business models. That is why this approach is called "risk-based tests."

Cases that automate testing these features help detect risk events immediately after each release that may prevent the release or be handled quickly.

Complex or time - consuming tests

There may be some complicated situations to copy manually in a project, so if we convert it to scripts, it would be easier to execute them in an automated way. If it is a form with large amounts of data, testers can be more error-prone, especially when they have to test the same form with multiple data variants. This is possible to reduce the probability of error through automation.

Repeat the test case

Just as regression testing becomes a repetitive task, automation is easily automated in some exceptional cases. For example, manually testing lots of data from the same process takes a lot of time, and repeating tests makes the process more tedious. However, by automating this process, we can parameterize the data without manually testing each value. These are also known as data-driven tests, where automated tests are parameterized and obtain data from a data source (such as a file or database).

Tool selection

Since we know what to automate, we can continue to choose the tools to use. The activity may be one of the most complex analyses given the number of available tools, and the decision will have to consider the items, budget, knowledge, and experience involved.

Several open-source, commercial, and custom tools vary in their limitations and availability. To choose the right tool, you must know what requirements must be met to continue a cost-benefit analysis of its use. Here is a brief overview of some test automation tools:

                                               ^{Source: Dzone.com}

 
      -       Selenium： An open-source tool, widely accepted worldwide, for testing Web applications on different browser peace platforms.

      -       Appium： Another open-source framework (based on Selenium) for testing automation, mainly on mobile devices for iOS and Android.

      -       Cucumber： This tool is part of the BDD (Behaviour-Driven Development) approach. The main advantage of Cucumber is its ease of use because it is very intuitive, offers a wide range of features, and is still open source.

      -      ZTF： Zentao Testing Framework is an open-source automated test management framework that focuses on the management function of automated testing and can solve problems such as use-case information management, test script execution, test results alignment, defect Bug submission, etc.

      -      ZenData： ZenData is convenient to use an available generation tool, which has two main functions: data generation and data analysis. Support one clicks to generate massive performance and pressure testing data suitable for large-scale, engineered automated testing.

It should be noted that there is no best tool for all cases. Depending on the tested application and decision-making standards, you can be more flexible in choosing between different software.