In this era of speed and agility, organizations are faced with the task, to reduce time to market of their business applications and products, with increased quality. Test automation becomes an imperative. Faster test automation, with high test coverage is the need of the hour.
On the other hand, adoption of test automation has its own challenges. Clients are expecting quicker ROI and CX, in terms of more business value from the spend on test automation. Broadly we have experienced challenges/roadblocks for adoption of testing automation across five operating dimensions:
When to automate? When to automate and when not to is the key question to answer before starting test automation. Organizations still use legacy systems for their business operations. Should we automate legacy applications? The answer depends on the organization strategy. Legacy system on transformation path to modernization should not be focused on from automation perspective.
Highly customized COTS systems are a major consideration in organization IT strategy. However, COTS systems must undergo configuration, customization and/or extension before they will meet the business needs of the organization. These highly customized applications pose different challenges for test automation. Automation adoption challenges related to COTS application are:
• Low return on investment
• The large scope of testing
• Changing technologies between releases and versions
• Test tool incompatibility with the product
Not selecting the right toolset is a major reason for test automation adoption failures. Alignment to development technologies and client business are important criteria for selection, which are generally overlooked. While some companies may focus on generalized capabilities like automation of web and mobile, others may prioritize specific capabilities, such as automation for Angular or React applications.
Sometimes selecting the best test automation tool could be challenging, with plethora of tools available in market. Meeting the client requirements and improvement of the overall test cycle time should be the focus. But most organizations fail to pick the best tool to suit their needs. Compatibility with existing ecosystem is of prime importance that gets ignored and leads to implementation challenges.
Success of automation testing is, highly dependent on the selected test automation tool. Every tool has specific capabilities. But teams lack the level of expertise required to get the best out of these capabilities.
Clients get caught up in the hype of a tool, without realizing that it does not provide what they are aspiring for. Plus, sometimes the cost of the tool exceeds the allocated budget. The overall impact is delay in decision making.
Design, configuration, and implementation of test automation needs specific skill set. A tester carrying out automation should have a clear understanding of what the development team is building, and the technologies used for the same. They should be able to understand the needs and articulate the same to stakeholders clearly. Clients generally have a thinking that any developer or tester with coding skills can carry out test automation.
To start automation projects, it is essential to have testers with extensive technical knowledge. Even though the investment in hiring technically sound test automation resource is high, the return is worthwhile.
Changing people’s mindset might be the biggest challenge. Automation first mindset is a hallmark of a good test automation resource. Test automation should be done early in the lifecycle and everything possible should be automated, including test processes and non-functional testing. Traditional mindset, of seeing testing as a phase and test automation as a specific activity, constraints adoption of test automation.
With the growing adoption of Agile and DevOps, organizations aspire for quicker deliveries with a continuous stream of updates delivered to production frequently. This puts a lot of pressure on QA and testing teams, to deliver more, at a quicker rate and at reduced costs. In-sprint test automation is to be enabled, to ensure maximum coverage within short release cycles. For in-sprint automation the upstream phases (requirement & development) play a significant role, any delay in these phases has cascading effect on automation.
• User stories not covering requirements properly
• Ambiguous requirement Development
• Development not delivering right quality of code
• Delay in deliverables to test resulting in lesser time for Quality Engineering team
The major contributors to increasing costs and testing and QA are, the adoption new models requiring faster and shorter development cycles, with the need to test more and test often. Spending on the next generation test automation tools also cause a spike in the testing budget. Though in the long run, it would help to cut IT spends on testing provided adequate control on ROI is maintained.
Key challenges from budget aspect include
• Funding not available right now
• Automation is low priority for the stakeholders