Harnessing AI and ML in automated testing to improve the speed and quality of development for a Leading Travel Technology Company

Challenges.

Customer’s existing test automation using selenium was flaky when executed. Test automation script creation and maintenance took team away from feature testing, resulting in defects. The customer was looking for:

• Stable and reliable test automation.
• Needed to democratize testing across non-technical team to increase test coverage and accelerate CI/CD pipeline.
• Significantly reduced test maintenance across releases and rapid UI changes.

Solution.

For the customer, Coforge invested in analysing different test automation tools. We proposed an innovative test automation solution, based on cognitive technologies. It leveraged SaaS based test automation platform, mabl. The platform had machine learning (ML) abilities with some unique features:

• Auto-healing: Tests adapt to UI changes automatically and stay up to date even after several successive UI changes. 
• Performance regression testing: Machine learning models help differentiate between anomalies and significant slowdowns of test execution and page load times.
• Visual anomalies detection: Detect important visual anomalies in the application.
• End-to-end testing: Enabling end-to-end testing using test automation for UI, APIs and PDF/email documents testing.

For implementing Continuous Testing, Coforge implemented four-pronged approach:

• Plan: In this phase, the current regression suite is analysed for, identifying gaps, increasing coverage and creating end-to-end test scenarios. Customer had ~450 regression test cases, which were analysed, and 220 end-to-end regression test cases were finalized, for complete coverage.
• Create: The next step was to create journeys for the 220 end-to-end scenarios.
• Test: In this phase the created journeys are executed. In addition to the functional defects identified by journey execution, the insights provided by tool’s machine learning model were analysed for auto-healing, performance anomalies, client-side JavaScript errors and visual defects.
• User Training: We created a training plan for the customer’s technical and non-technical staff and had specific tools workshop on mabl.

The impact.

Solution enabled automating application features in parallel once a scheduled sprint is complete. This increased the velocity of the development team by allowing them to identify complex defects and fix them within the sprint cycle. It enabled 2x faster test automation script creation and 4x faster test script maintenance.