Quality Engineering - Tech Mahindra

© NelsonHall 2020 1 March 2020

April 2022 Licensed for distribution © NelsonHall 2022

Licensed for distribution

1

NEAT EVALUATION FOR TECH MAHINDRA:

Quality Engineering

Market Segment: Overall

Introduction

This is a custom report for Tech Mahindra presenting the findings of the NelsonHall NEAT vendor evaluation for Quality Engineering in the Overall market segment. It contains the NEAT graph of vendor performance, a summary vendor analysis of Tech Mahindra for quality engineering services, and the latest market analysis summary.

This NelsonHall Vendor Evaluation & Assessment Tool (NEAT) analyzes the performance of vendors offering quality engineering services (formerly referred to as software testing services). The NEAT tool allows strategic sourcing managers to assess the capability of vendors across a range of criteria and business situations and identify the best performing vendors overall, and with a specific capability in application security testing, continuous testing, AI & cognitive testing, UX testing, and cloud migration testing.

Evaluating vendors on both their ‘ability to deliver immediate benefit’ and their ‘ability to meet client future requirements’, vendors are identified in one of four categories: Leaders, High Achievers, Innovators, and Major Players.

Vendors evaluated for this NEAT are: Amdocs, Atos, Capgemini, Cigniti, eInfochips, Expleo, Infosys, LTI, TCS, Tech Mahindra, TestingXperts, Trigent, ValueMomentum, and Wipro.

Further explanation of the NEAT methodology is included at the end of the report.

NEAT Evaluation for Tech Mahindra: Quality Engineering


2

NEAT Evaluation: Quality Engineering (Overall)

NelsonHall has identified Tech Mahindra as a Leader in the Overall market segment, as shown in the NEAT graph. This market segment reflects Tech Mahindra’s overall ability to meet future client requirements as well as delivering immediate benefits to its quality engineering clients.

Leaders are vendors that exhibit both a high ability relative to their peers to deliver immediate benefit and a high capability relative to their peers to meet future client requirements.

Buy-side organizations can access the Quality Engineering NEAT tool (Overall) here.

https://research.nelson-hall.com/sourcing-expertise/neat-reports/?avpage-views=neat&avpage-views=neat&NEATaction=viewProject&project=292



3

Vendor Analysis Summary for Tech Mahindra

Overview

Tech Mahindra was established in 1986 and is a 35.9% subsidiary of the Mahindra Group, with its headquarters in Pune, The company had a headcount of 126k at the end of Q1 FY21.

Due to its background as a JV between BT and Mahindra & Mahindra, Tech Mahindra, at the corporate level, continues to have a strong presence in the communication, media & entertainment sector (representing 40% of its revenues in Q2 FY22). Other important industries are manufacturing (16%); BFSI (16%); technology (9%); retail, transport, and logistics (8%), and other (11%).

For the same reason, Tech Mahindra has a more geographically balanced client presence than many of its Indian peers; the Americas represented 48% of its revenues in Q1 FY22, with Europe representing 26%, and RoW 26%.

Tech Mahindra's Digital Assurance Services (DAS) is a TCoE and oversees pre-sales consulting; is responsible for creating service offerings; builds and maintains IPs, accelerators, and platforms; and undertakes delivery of specific projects. The unit has dotted line responsibility for testing delivery.

NelsonHall estimates that Tech Mahindra had ~12.4k career testers across units at the end of 2021, and that 62% of career testers work on agile projects while the remaining 38% work on traditional waterfall projects. The company has ~300 testing clients and 30 IPs.

The key strategic focus for DAS relies on automation and the creation of accelerators and platforms. The company has aggregated most of its automation IP and accelerators around its LitmusT brand. LitmusT has a wide range of capabilities, including:

• Functional test execution in the context of agile/continuous testing and support services (e.g., test environment and test data management, and code quality analysis)

• Reports, analytics, and cognitive

• Model-based testing

• Non-functional, e.g., security and performance testing.

DAS has seven automation priorities (which are part of LitmusT):

• AI-based use cases for increasing test automation

• Intelligent test automation, based on the MAGiX IP

• BI and enterprise data warehouse automation

• Test data management solution

• Intelligent test design automation, i.e., model-based testing with test execution in MAGiX

• Test environment management

• Reporting dashboards.

Tech Mahindra highlights that most of its testing contracts are multi-tower, involving ADM in the context of agile projects.



4

Tech Mahindra’s two main delivery centers are Hyderabad and Pune. The Pune center has historically serviced telecom clients, while the Hyderabad facility serves financial services, manufacturing, and retail. There are also centers in Chennai, Bangalore, Noida, Vizag, Kolkata, Bhubaneshwar, and Mumbai. Currently, all centers in India are client-dedicated, with little sharing of personnel across clients. Tech Mahindra's delivery organization in India has been certified at TMMi level 5.

Tech Mahindra also has 2k career testers in the U.S., working onsite and in factories, and 1k in the U.K. and Europe. Also, the company has opened a center in Hamburg (which provides performance testing and ADM services) and also has centers in Budapest (Hungary), Gothenburg (Sweden), and Wiesbaden (close to Frankfurt), which all offer testing services. In total, Tech Mahindra has 0.5k career testers in Continental Europe, onshore and nearshore.

Examples of recent very significant testing contracts include:

• $350m contract with a Canadian bank in FY21

• $400m multi-tower deal with a European telecom

• $100m contract with a tier-one U.S. technology firm.

Financials

NelsonHall estimates that Tech Mahindra’s 2021 testing services revenues were ~$780m.

Strengths

• Continuous testing: with MAGiX, Tech Mahindra has developed a core platform that is the central point for aggregating all accelerators and IP. It has a systematic approach to test automation beyond test execution with model-based testing, test data management, test environment management and reporting, performance testing, and security testing. The company is, therefore, making MAGiX comprehensive and is putting all functionality in the DevOps pipeline

• Model-based testing offering is comprehensive

• AI-based analytics: the AI use cases offering is growing and well-developed with its dual approach based on the Cha.AI and eAnalytics IP. However, we would like Tech Mahindra to standardize on one or the other, as the two IPs are increasingly overlapping

• AI-based automation: Tech Mahindra is one of the vendors that has invested in AI-based automation, with a next-gen record-and-playback tool

• Explainable AI: Tech Mahindra is investing in developing this offering

• Application security testing: the offering is comprehensive and has technology accelerators supporting it.

Challenges

• Testing of RPA bots: the company has not structured its RPA testing capabilities

• Testing of chat- and voice-bots/virtual assistants: Tech Mahindra says it has one testing framework. However, NelsonHall was not in a position to evaluate it



5

• Testing of AI models: the AI testing approach is only starting, with Tech Mahindra having developed a methodology; the offering is not comprehensive and lacks IP and accelerators to automate it further. The Visual Testing offering is a good move in the right direction.

Outlook

Tech Mahindra continues to expand the capabilities of LitmusT and is taking a comprehensive approach. It is well advanced in AI in the context of testing both in terms of AI-based analytics and AI-based automation. It is one of the few vendors that has started the Explainable AI journey.

We would like Tech Mahindra to accelerate its effort in UX testing and RPA. The potential for UX testing automation is immense, and the growth prospects for UX testing are high. Also, other digital technologies such as business process RPA and voice assistants offer great potential for adoption, and we would like Tech Mahindra to structure its offerings.



6

Quality Engineering Market Summary

Buy-Side Dynamics

The three major client segments for QE services are:

• ‘Agile Mainstream’: organizations that are transitioning to hybrid agile (with digital projects adopting agile and non-digital remaining on waterfall methodologies). They are currently implementing DevOps tools (i.e., continuous testing) to increase their level of automation

• ‘Advanced Automation’: organizations that are engaged in an agile and continuous testing transformation like Agile Mainstreams. However, they look at emerging automation opportunities (e.g., AI-based automated test script creation, RPA tools) to reach new levels of automation, initially in functional testing

• ‘Digital Matures’: organizations that have several digital programs and look to automate digital technologies (e.g., Salesforce, chatbots, IoT) and UX research and testing to overcome challenges such as volume (IoT).

‘Agile Mainstream’ clients select their QA vendors based on their past performance in similar projects, including internally and externally (with other clients); vendors must also demonstrate their ability to:

• Deploy continuous testing technologies to drive automation, to serve agile projects

• Expand automation outside of functional execution and experiment with new functionality such as test support services (e.g., test data and environment management) and AI use cases

• Reskill manual testers towards technical services.

‘Advanced Automation’ organizations select their QA vendors based on their ability to demonstrate:

• Their investment in AI use cases, initially around AI-based analytics and expanding to automation

• Best practices and sharing a clear view of the art of the possible

• Change management capabilities to drive tester buy-in.

For ‘Digital Matures’, vendors must demonstrate:

• They have either a specialization in testing the digital technology (e.g., Salesforce, chatbots, IoT) or have both build and test capabilities. If the digital technology comes from an ISV, vendors need to demonstrate they have formalized their partnership with the technology vendor. They also need to articulate their status level and what that level means

• Their QA capabilities can effectively play the role of a quality gate and must be independent of the implementation/development team

• They bring automation capabilities, rather than manual functional expertise.



7

Market Size & Growth

The current global software testing (QE) services market size stands at ~$36bn. NelsonHall is basing its forecast on the prediction of the IMF, as the testing market has become a cyclical market.

NelsonHall expects a rebound in 2021 (+4%), led by a catch-up, after a moderate deceleration in 2020 (+2.5%) in testing services spending. This deceleration was much softer than expected with clients pushing external testing expenses, cloud, and digital projects.

Growth in 2022 will slow down somewhat to +3%, impacted by the end of the catch-up effect.

Spending will reach $38bn in 2024, representing a +4% CAGR between 2020 and 2025.

Outlook

Functional testing (including manual activities, automation, COTS, and digital testing) represents ~84% of spending. Looking ahead, functional testing will accelerate its decline to a pace of 4-5% annually. The decline is fueled by investment in functional automation (+9-10% annually). Digital testing spending continues to grow by 7-9%, led by digital projects. COTS testing is growing nicely (4-6%), driven by SaaS application adoption.

Specialized services account for the remaining 16%, with non-functional representing 9%. They benefit from the trend toward more technical services. The growth is across service lines with AI, security, and test support (test data, test environment, and service virtualization) leading growth.

Approximately ~71% of delivery lies in India. This percentage has slightly declined due to the adoption of digital and major vendors increasing their presence onshore. The economic crisis will drive the adoption of ‘distributed agile’ and relocate further work to India.

The role of the client TCoE is evolving from a centralized testing delivery location to a shared service center providing specialized and technical services.



8

NEAT Methodology for Quality Engineering

NelsonHall’s (vendor) Evaluation & Assessment Tool (NEAT) is a method by which strategic sourcing managers can evaluate outsourcing vendors and is part of NelsonHall's Speed-to-Source initiative. The NEAT tool sits at the front-end of the vendor screening process and consists of a two-axis model: assessing vendors against their ‘ability to deliver immediate benefit’ to buy-side organizations and their ‘ability to meet client future requirements’. The latter axis is a pragmatic assessment of the vendor's ability to take clients on an innovation journey over the lifetime of their next contract.

The ‘ability to deliver immediate benefit’ assessment is based on the criteria shown in Exhibit 1, typically reflecting the current maturity of the vendor’s offerings, delivery capability, benefits achievement on behalf of clients, and customer presence.

The ‘ability to meet client future requirements’ assessment is based on the criteria shown in Exhibit 2, and provides a measure of the extent to which the supplier is well-positioned to support the customer journey over the life of a contract. This includes criteria such as the level of partnership established with clients, the mechanisms in place to drive innovation, the level of investment in the service, and the financial stability of the vendor.

The vendors covered in NelsonHall NEAT projects are typically the leaders in their fields. However, within this context, the categorization of vendors within NelsonHall NEAT projects is as follows:

• Leaders: vendors that exhibit both a high ability relative to their peers to deliver immediate benefit and a high capability relative to their peers to meet client future requirements

• High Achievers: vendors that exhibit a high ability relative to their peers to deliver immediate benefit but have scope to enhance their ability to meet client future requirements

• Innovators: vendors that exhibit a high capability relative to their peers to meet client future requirements but have scope to enhance their ability to deliver immediate benefit

• Major Players: other significant vendors for this service type.

The scoring of the vendors is based on a combination of analyst assessment, principally around measurements of the ability to deliver immediate benefit; and feedback from interviewing of vendor clients, principally in support of measurements of levels of partnership and ability to meet future client requirements.

Note that, to ensure maximum value to buy-side users (typically strategic sourcing managers), vendor participation in NelsonHall NEAT evaluations is free of charge and all key vendors are invited to participate at the outset of the project.



9

Exhibit 1

‘Ability to deliver immediate benefit’: Assessment criteria

Assessment Category Assessment Criteria

Offerings

Advisory and consulting services

Continuous testing

Application migration to the cloud QE

IAC testing

MBT

AI-based analytics

AI-based automation

Testing of business process automation RPA tools

Testing of virtual assistants

Testing of AI models

Explainable AI

UX research and testing: Usability

UX research and testing: Accessibility

Application security testing

Delivery

Indian delivery capability

US onshore capability

EMEA onshore capability

Offshore leverage

Presence

Customer presence globally

Customer presence in NA

Customer presence in EMEA

Customer presence in APAC

Customer presence In LATAM

Benefits Achieved

Level of cost savings achieved

Increased application quality/reduced production downtime

Increased speed-to-market for digital initiatives

Increased end-user/business satisfaction/UX

Other benefits achieved

Pricing approach



10

Exhibit 2

‘Ability to meet client future requirements’: Assessment criteria

Assessment Category Assessment Criteria

Levels of Investment

Continuous testing

Application migration to the cloud QE

IAC testing

MBT

AI-based analytics

AI-based automation

Testing of business process automation RPA tools

Testing of virtual assistants

Testing of AI models

Explainable AI

Usability testing

Accessibility testing

Application security testing

Ability to Innovate

Mechanisms in place to deliver client automation innovation

Extent to which client perceives that automation innovation has been delivered

Suitability of vendor to meet future continuous testing needs of clients

Suitability of vendor to meet future cognitive testing needs of clients

Suitability of vendor to meet future UX testing needs of clients

Perception of suitability to meet future needs for other technologies

Other Market momentum

Financial security

For more information on other NelsonHall NEAT evaluations, please contact the NelsonHall relationship manager listed below.

research.nelson-hall.com

Sales Inquiries NelsonHall will be pleased to discuss how we can bring benefit to your organization. You can contact us via the following relationship manager:

Guy Saunders at [email protected]

Important Notice

Copyright © 2022 by NelsonHall. All rights reserved. NelsonHall exercises its best efforts in preparation of the information provided in this report and believes the information contained herein to be accurate. However, NelsonHall shall have no liability for any loss or expense that may result from incompleteness or inaccuracy of the information provided.

http://www.nelson-hall.com/

Quality Engineering - Tech Mahindra

Documents

Transcript of Quality Engineering - Tech Mahindra