Harnessing AI and ML in automated testing to improve the speed and quality of development for a Leading Travel Technology Company

Challenges.

Explain what challenges affected the client’s operations or goals, emphasizing any constraints that exacerbated the difficulties. Also, list the key business challenges encountered by the client in bullet points.

Customer’s existing test automation using selenium was flaky when executed. Test automation script creation and maintenance took team away from feature testing, resulting in defects. The customer was looking for:

• Stable and reliable test automation.
• Needed to democratize testing across non-technical team to increase test coverage and accelerate CI/CD pipeline.
• Significantly reduced test maintenance across releases and rapid UI changes.

Solution.

Describe the implemented solution in detail, including practical steps for its execution. Be sure to mention all specific tools or technologies utilized as part of the solution. List the key highlights of the Coforge’s solution in bullet points and consider adding relevant images if available.

For the customer, Coforge invested in analysing different test automation tools. We proposed an innovative test automation solution, based on cognitive technologies. It leveraged SaaS based test automation platform, mabl. The platform had machine learning (ML) abilities with some unique features:

• Auto-healing: Tests adapt to UI changes automatically and stay up to date even after several successive UI changes. 
• Performance regression testing: Machine learning models help differentiate between anomalies and significant slowdowns of test execution and page load times.
• Visual anomalies detection: Detect important visual anomalies in the application.
• End-to-end testing: Enabling end-to-end testing using test automation for UI, APIs and PDF/email documents testing.

For implementing Continuous Testing, Coforge implemented four-pronged approach:

• Plan: In this phase, the current regression suite is analysed for, identifying gaps, increasing coverage and creating end-to-end test scenarios. Customer had ~450 regression test cases, which were analysed, and 220 end-to-end regression test cases were finalized, for complete coverage.
• Create: The next step was to create journeys for the 220 end-to-end scenarios.
• Test: In this phase the created journeys are executed. In addition to the functional defects identified by journey execution, the insights provided by tool’s machine learning model were analysed for auto-healing, performance anomalies, client-side JavaScript errors and visual defects.
• User Training: We created a training plan for the customer’s technical and non-technical staff and had specific tools workshop on mabl.

The impact.

The ‘Impact’ section summarizes the achieved results and improvements resulting from the solution in a tabular form. It includes both qualitative benefits experienced by the client and the relevant quantitative metrics (e.g., percentage improvements, time saved). Example:

Solution enabled automating application features in parallel once a scheduled sprint is complete. This increased the velocity of the development team by allowing them to identify complex defects and fix them within the sprint cycle. It enabled 2x faster test automation script creation and 4x faster test script maintenance.