The New Age of Software Testing

Enterprise IT software is extremely complex, quickly developed by highly skilled people across the world through collaboration and must run in many ecosystems and on many devices, uninterrupted. In the past, software might be released a couple of times per year, but now, new releases may go out once each minute, seamlessly. In many cases the software development process started with a waterfall methodology, and has now moved to agile, lean, and continuous integration (CI). In this new age, cloud service consumers (software users) live in a multi-tenant world in which they no longer have control over when updates occur. Customers are scattered across many time zones and each customer must access the software, so there is never a good time for maintenance, or timeouts, or failure.

Image Source: Quora

Continuous operations manage software and hardware changes in a way that is non-disruptive to software users. Even though software and servers may be taken off-line during planned maintenance or updates, this is managed so users continue to be serviced by existing versions of the application, and are automatically switched over to newer versions once they have been deployed and successfully smoke-tested.

Ensuring the quality of these highly sophisticated and dynamic software products is becoming extremely difficult at the speed of development, delivery, deployment and operations. In the New Age, software testing is a multi-dimensional approach, and a quality/test engineer always evaluates the following dimensions every minute.

Dimension:

1. Validating technical deliverables (Front-end, Middleware, Back-end, Platform).
2. Evaluating in the customer ecosystem (Web Browsers, Mobiles, Desktops, Tablets, Native, Hybrid, Cross Platform, Wearables, Operating Systems, Various Versions).
3. Increase speed of testing to match the speed of delivery through continuous integration/ delivery/ production through maximum automation.
4. Achieve full coverage by doing various types of testing (User Behavior, Functional, Performance, Security, Usability, etc.)
5. Evaluate as per the needs of the end user, product owner, product manager, developers, architects, business analysts, DevOps manager.
6. Optimize test design through high end test Design techniques (Risk based, requirements based, Combinatorial, Scenario based, Model based, Pairwise , Combinatorial, Orthogonal , Boundary values, etc.)
7. Reuse maximum assets of other phases of the test life cycle and reuse maximum legacy assets especially from business analysts/product owners/developers/architects/developers/other testers.

In the New Age, test engineers wear many hats in software delivery. They need to change their software testing thinking, and integration of test automation with various solutions/technologies to support their multi-dimensional way of working.

In the New Age, testing is no longer limited to automation of test execution, automation of test design, or automation of a single part of the testing process, but：

• About how the tests are derived and designed in the first place, as well as how tests are managed and executed;
• About transparency, visibility and control;
• About speed and turnaround time;
• About seamless integration, such as bringing tools and processes together.

The following sections emphasize the real-world needs from software testing in the New Age:

Increased productivity:

• Automate functional testing with minimal manual involvement.

• Leverage existing test assets – test cases, requirements, prototypes.

• Eliminate test automation backlog.

Increased and optimized coverage:

• Thoroughly test complex systems with a minimum of test cases; optimize regression test suites.

• See generated tests, what they cover, and why they’re needed; gain visibility into what has been tested and what has not.

Integrated test flow:

• Tightly integrate with all SDLC tools and processes.

• Directly link to automated execution frameworks.

Test automation:

• Shorten the path to test automation.

• Simplify advanced test design techniques.

• Automated execution of tests on many platforms and ecosystems.

• Parallelize testing on premise, or in the cloud.

Reusability:

• Reuse existing assets including models, manual test cases, and record and playback tests.

• Reuse every stakeholder's assets to quicken the delivery.

Test Automation

Testing at the speed of development is becoming extremely crucial to release software. Automated tests are the only way to achieve sufficiently high test coverage in each sprint and thus provide the high quality and rapid feedback that testers seek for their projects.

The benefits of automation are various. The time saved when executing tests automatically instead of manually is an obvious advantage. Another is that test automation can lead to more exhaustive testing because with automation, nothing stops testers from executing the same test several times with more varied test data and perhaps even test different environments. However, the main benefit of test automation is that confidence in the system and its quality are increased when more comprehensive tests are performed. This trust makes it possible to improve and customize the system to project requirements.

Additionally, resources become available for other tasks. Instead of retesting existing functionality, testers can put their energy into testing areas of new functionality where human insights are really needed. Developers also become more confident with the help of automation. There will be fewer errors left as developers quickly see the consequences of a code change, thus saving the team’s (and their organization’s) time and money!

Test Automation Framework

A test automation framework is scaffolding that is laid to provide an execution environment for the automation test scripts. A framework is a constructive blend of various guidelines, coding standards, concepts, processes, practices, project hierarchies, modularity, reporting mechanism, test data injections etc. to pillar automation testing. Thus, users can follow these guidelines while automating applications to take advantage of various productive results.

At a high level, an automation framework uses drivers and data sets to execute tests in a variety of test environments.

The best framework is an amalgam of many test automation techniques, taking advantage of their strong points and intelligently tackling their weaknesses to satisfy the needs of various kinds of project stakeholders.

Highly-used test automation framework types:

1. Module-based testing framework.
2. Library architecture testing framework.
3. Data-driven testing framework.
4. Keyword-driven testing framework.
5. Hybrid testing framework.
6. Behavior-driven development framework (BDD).
7. Business process testing framework.

Advantages of test automation frameworks:

1. Reusability of code.
2. Maximum coverage.
3. Recovery scenario.
4. Low cost maintenance.
5. Minimal manual intervention.
6. Easy reporting.