#ATAGTR2023 Speaker

Reduce Reuse Recycle (RRR) of Test Automation Maintenance

Functional test automation once built catches regression bugs and becomes the lifeline of release cycle. As the application under test evolves, tests and frameworks also evolve. External factors like environments, browsers, stack changes, attrition, skill gap and fast paced release cycles can bring in wear and tear of regression inventories. Very quickly such an inventory if not maintained well can give large scale failures. Failures may be genuine or false alarm; sheer volume of failures adds to lot of frustration and derailment of release cycles. Hence it is utmost important to form wholistic maintenance plan. A well-oiled machinery of regression test helps development of a fast feedback continuous testing ecosystem. And that in turn speeds up the release cycle, CI/CD process.

“Test Automation” is assumed for lot of things we do for “script level coding”. But that’s not all it is umbrella for many other activities. You can correlate it to same way we deal with AUT. isn’t coding just the 50% of all the peripheral work which is needed to successfully deliver a feature or a story to production. That may include, design, architecture, UI/UX, functionality, security, performance, usage pattern and timing. In same way, it is essential to think Holistically what all constitute “test automation” and if we put diligence to each aspect, overall maintenance will get automatically curtailed, or roped into mainstream

1. Strategy – “What to automate, where to automate, what tools can be used, who does it”
2. Automation framework – It is a software that is reused to develop/code tests. And it has all characteristics of a successful s/w reliability, evolved, perf.
3. Script development – routine task where you develop card level, story level automation.
4. Playback Environment – Carefully selecting Configuration settings for optimal playback performance.

Here are few of the areas that I wish to elaborate and explain

1. Optimizing test automation maintenance can be achieved by following best practices such as the test pyramid. Key is to reduce no of UI/Functional TEST. And the Test Pyramid helps you achieve that. The test pyramid approach helps reduce test automation maintenance by minimizing the number of tests requiring maintenance, identifying issues earlier in the development cycle, reducing the risk of flaky tests, and providing faster feedback on changes.

2. Using a layered approach in the test automation framework. Tests must be separated from automation code. The test layer provide insight into business acceptance criteria. BDD can mandate this layer. In absence of BDD, the test layer has to be present. Step definition should eventually call custom DSLs. DSL bring in reusability and modularity aspect of test automation at a page/component level. Implementing custom DSLs can simplify and streamline the test automation code, and combining this with logging and exception handling at each layer helps to effectively debug test automation failures. Lowest layer interacts with browser interaction (web driver/selenium/tool API). BOT implementation pattern. Where it wraps the actions like click and type, with enhanced exception handling, logging and wait strategies.

3. Use of clean coding practices provides much required discipline to seal the skill gap. Use of tools like IDE plugin for coding conventions, centralized keyword library of automation APIs in Nexus, use of logging and exception handling with Prometheus-grafana, maven archetypes for templatizing workspace setup

4. Collaborating between development and QA. Agile testing involves continuous feedback and collaboration between the testing team and the development team. Test cases when developed in real time and executed in parallel along with the development progress, then any issues or defects also get detected and fixed in real-time. When dealing with large and disruptive user stories, work with developers to take a step-by-step approach and identify which aspects of the microservices are undergoing changes. We have upstream and downstream value streams that help us correctly gain insights into how much of porting effort will be incurred to realistically plan for in-sprint automation. I wish to discuss the same.

5. Environmental sanitization is a critical aspect that reduces the effort needed for test automation maintenance. When transitioning from in-house machines to cloud or containerized playback, it is essential to ensure that the test environment is correctly set up.

6. The advent of AI/ML in test automation has revolutionised how tool vendors integrates various usecases and optimise test automation workflows. Few of the tools that we have investigated we found, they bring in optimization in capturing recognition strings, NLP in writing automation scenarios, traceability between code and tests, playback auto heal and others. I wish to share some of these findings towards the end.

In summary, maintain the test automation by adopting clean coding practices – REUSE, considering upstream test automation impacts by collaborating with development – RECYCLE Test Pyramid – helps REDUCE.

Key Takeaways for audience

1. Simple practical tips of clean coding practices
2. Tools and Techniques at grass root level
3. Agile upstream and downstream value streams that help in-sprint automation.
4. Practical and reusable checklist for environment sanitization in a microservices kind of environment
5. Insights into AI/ML enabled test automation tools and specific techniques that these optimize.