Top five KPIs for a Testing Center of Excellence (2023)

Top five KPIs for a Testing Center of Excellence (1) Andrei Mikhailau
SW Testing Director @ ScienceSoft

TESTING

To consolidate QA resources, best practices and automation tools dispersed around the enterprise, and to optimize testing performance, many companies turn to Testing Centers of Excellence (TCoEs). At the same time, TCoE isn't a heal-all concept for every company, as its setup requires a lot of organizational efforts. TCoE performance results can't always outweigh them.

(Video) KPI Best Practices

How can we know for sure whether a TCoE is a successful QA hub or the fifth wheel for the enterprise? Here you can find five main TCoE performance criteria to monitor and be realistic about your TCoE's effectiveness.

The formula of TCoE KPIs

You can evaluate TCoE performance with the help of KPIs; each KPI is based on a particular TCoE goal resulting from the QA challenges TCoE is supposed to face. For example, TCoE managers are to advise a test lead on the most appropriate line-up of a test team for a project (challenge) to meet the budget limits (goal). So, from here you get the KPI for testing costs.

You need to prioritize the whole set of goals and transform only the most acute ones into KPIs. For the evaluation to be multidimensional, you need to choose, from your shortlisted TCoE goals, the ones corresponding to different TCoE focus areas, for example, testing quality, testing time and budget reduction, QA maturity nurture and the promotion of test automation.

A tricky point with TCoE performance measurement is that very good results for all KPIs simultaneously are not an adequate aim to be set. To get the right picture of TCoE performance, you need to acknowledge that impressive KPIs for all TCoE goals are utopian. One striking KPI is prone to negatively influence another, and only moderately good and balanced results are a key to performance success. For example, if TCoE managers have encouraged the test team to complete all the project's tests in 3 days, which sounds quite stunning, and the KPI of testing time proves it, this impressive result is likely to be bound to low testing quality (a lot of missed bugs) and the bloated number of test engineers involved (excessive testing costs).

Top five TCoE KPIs and a balanced approach to their evaluation

Your KPIs are meant to highlight the TCoE performance results that are of utmost importance for your enterprise with regard to its specific business objectives and priorities. That's why there is no set of universal KPIs. Here's the top list of the most common KPIs, which can be used as a springboard for your tailor-made one.

(Video) Putting Your Data To Work: Five Frameworks to Develop Your Top KPIs

The missed bugs rate

This KPI is based on such a TCoE goal as testing quality enhancement. To find out whether the testing results of a specific project live up to the established quality standards and truly contribute to the quality of an application under test (AUT), it's handy to use the percentage of defect leakage.

The excessive number of missed bugs (more than 5% of the total bugs' number) may be caused by the following problems: the test team's misunderstanding of software requirements, an inefficient test plan not corresponding to the changing project requirements, the lack of coherent testing quality standards and methodologies.

However, if you see that a missed-bug rate equals zero, you need to check whether the test team managed to meet the iteration and project deadlines, and whether the number of found bugs results out of the iteration requirements' scope.

Testing time

Test engineers are often pressed to complete testing faster to ensure that bug-less software reaches its users sooner. The average testing duration in comparison with the results observed without TCoE indicates its effectiveness.

If average testing time remains unchanged, it may indicate that your TCoE isn't successful in the development of a balanced and efficient test strategy (involving automated testing and the coherent allocation of the test team's roles and duties). It's likely that a testing methodology promoted by TCoE needs further improvement.

(Video) Putting Excellence In Your Testing CoE!

On the other hand, if you find out that the testing duration got a serious and rapid reduction after TCoE's setup (more than 30-40% in comparison with the "before-TCoE" rate), you need to check the missed-bug rate, as too little time may have been devoted to testing and the testing quality could have suffered.

Testing costs

The goal promoted by this KPI is to optimize the QA and testing budget. To understand whether your TCoE is successful in meeting this objective, you need to compute the average cost of testing before and after establishing TCoE.

TCoE's typical cost savings rate is about 35% in a 3-year time frame. A low cost reduction rate may be the result of unbalanced resource allocation or insufficient control over a test team's efficiency. Still, you shouldn't be too optimistic about an unexpectedly high cost reduction rate. TCoE managers may try to economize regardless of missed bugs and poor testing quality, for example, by giving their testing teams insufficient quantity and qualifications.

QA maturity level

The achievement of higher QA maturity levels and promotion of QA culture around the whole enterprise are fundamental TCoE goals. QA maturity can be evaluated using one of the widely recognized models: Test Maturity Model Integration (TMMi) or Test Process Improvement (TPI). Both models describe what QA and testing activities are characteristic to each of 5 (in TMMi) and 4 (in TPI) maturity levels.

The gist of the TCoE presence within an enterprise is to ensure that QA activities (performance measurement and regular reporting, knowledge transfer, smooth communication between the departments, etc.) are promoted through the enterprise. Therefore, low QA maturity levels are not normally observed if there is TCoE, except for the case when it's still at the beginning of its formation.

(Video) 5 Focus Areas for a European Commerce Center of Excellence

In case this KPI shows the highest maturity level, you should pay special attention to TCoE managers not becoming oversensitive or perfectionistic about QA. The high quality of processes and output is just one of the business goals. The highest QA maturity level should not interrupt other business needs and priorities, as TCoE managers being overprotective about quality may impede meeting project deadlines and budgets.

The rate of test automation

Test automation is one the most booming trends for the near QA future, and its promotion is one of the main TCoE goals. The major reasons why test automation is so popular include: the reduction of testing time and costs, enhanced testing quality, and vast test coverage (unachievable with manual efforts). The test-automation rate is counted by dividing the number of automatically covered test cases by their total number.

However, you should mind that this KPI may be tricky or even misleading, if you are not fully aware of some test automation hidden pitfalls. Deciding on every testing project's automation rate demands deep analysis of the software requirements and testing scope. For some testing projects, the percentage of test automation can catch up with 70-80%, while for others it may remain 0. It's neither a good, nor a bad result, as, by far, not every testing project should presuppose test automation.

There is no point to perform automated testing for small-scale projects, as it won't pay off. Besides, it's senseless to insist on test automation in projects with constantly changing requirements. The automated test scripts will get outdated after each iteration or source code change. Some test types like: usability, ad-hoc and exploratory testing - can't be automated. Thus, TCoE should strike a balance between automated and manual testing efforts. So, if you find out that this KPI results in a generally high percentage through all of the testing projects, you should check whether the TCoE managers promote a reasonable approach to test automation.

The outcomes

To ensure that your TCoE performance results meet expectations, you should come up with an effective and coherent set of KPIs TCoE is to live up to. Your KPIs need to evaluate TCoE performance from different perspectives to provide a balanced view of how TCoE meets its various goals. Though no set of KPIs is universal, our top five KPIs may come to be handy as they reflect such crucial TCoE objectives as: shorter testing time, reduced testing costs, testing quality improvement, and higher levels of test automation and QA maturity. Still, don't forget to carefully analyze the assessment results, as one too impressive KPI may be the sign of some missing TCoE goal.

(Video) KPIs for ACCA APM

FAQs

What are KPIs in testing? ›

KPIs or Key Performance Indicators in the software testing industry are some measurable values that are computed to gauge the efficiency and effectiveness of the testing process as a whole.

What are the KPIs quality metrics to measure the success for a Test Manager? ›

  • 12 Key Performance Indicators for QA & Test Managers. By: Matt Angerer. ...
  • 1 – Active Defects. Tracking active defects is a pretty simple KPI that you should be monitoring regardless. ...
  • 2 – Authored Tests. ...
  • 3 – Automated Tests. ...
  • 4 – Covered Requirements. ...
  • 5 – Defects Fixed Per Day. ...
  • 6 – Passed Requirements. ...
  • 7 – Passed Tests.
21 Dec 2015

What are the 4 main KPIs? ›

Anyway, the four KPIs that always come out of these workshops are:
  • Customer Satisfaction,
  • Internal Process Quality,
  • Employee Satisfaction, and.
  • Financial Performance Index.
25 Sept 2013

What is the most important KPI? ›

The 5 Most Important Financial KPIs That Drive Business Strategy
  • Sales Growth Rate. Performance Indicators.
  • Revenue Concentration. Performance Indicators.
  • Net Profit Margin. Performance Indicators.
  • Accounts Receivable Turnover. Performance Indicators.
  • Working Capital.

How do you measure test effectiveness? ›

Answer: Test effectiveness can be calculated using the below formula: Test Effectiveness = Number of valid bugs fixed/( Bugs injected+ number of bugs escaped)*100.

How do you set KPI for quality assurance? ›

Recommended KPIs for quality assurance
  1. Right first time. This KPI measures how many products you produce correctly from the first time without any modifications before sales.
  2. Complaint rate. A KPI that measures your number of complaints.
  3. Internal audit plan adherence. ...
  4. Overdue corrective actions.
5 Mar 2020

What are the 3 types of KPIs? ›

Types of KPIs include: Quantitative indicators that can be presented with a number. Qualitative indicators that can't be presented as a number. Leading indicators that can predict the outcome of a process.

How do you write a clear KPI? ›

Steps to follow to write effective KPIs
  1. Step 1 – Identify your organization's strategic objectives. ...
  2. Step 2 – Define the criteria for success. ...
  3. Step 3: Develop key performance questions. ...
  4. Step 4- Collect supporting data. ...
  5. Step 5: Determine what to measure and how frequently you should measure. ...
  6. Step 5: Develop the KPIs.
24 Jun 2021

How do you set goals for KPIs? ›

Setting SMART KPIs

Specific: be clear about what each KPI will measure, and why it's important. Measurable: the KPI must be measurable to a defined standard. Achievable: you must be able to deliver on the KPI. Relevant: your KPI must measure something that matters and improves performance.

What are the KPI of quality? ›

Quality indicators are tools used to measure and monitor a company's performance and are among the principal types of process performance indicators, or the famous KPI's (Key Performance Indicators).

What is SLA in QA? ›

A Service-Level Agreement for quality assurance, or QA SLA, is the part of a written contract between you and a software QA company that specifies what you expect from the service provider and the process for conflict resolution. Usually, these are made to ensure availability of resources.

What are quality assurance best practices? ›

The Quality assurance best practices to keep in mind:
  • develop a clear-cut plan;
  • set SMART QA goals and objectives;
  • employ a multi-directional testing strategy;
  • conduct external quality reviews;
  • maintain thorough documentation;
  • maintain a healthy atmosphere in the team;
  • don't underestimate errors;
  • keep learning.
16 Jun 2022

What is smart KPI? ›

Many companies recognise the value of KPIs (key performance indicators), but not everyone has implemented SMART (Specific, Measurable, Achievable, Relevant and Time-Bound) KPIs for their employees. SMART KPIs provide clarity in terms of performance expectations and progress.

How do you build a strong KPI? ›

Steps to Success of a KPI
  1. Step 1: Create Objectives. As said before, KPIs require objectives. ...
  2. Step 2: Describe Results. ...
  3. Step 3: Identify KPIs. ...
  4. Step 4: Define Thresholds. ...
  5. Step 5: Measure. ...
  6. Step 6: Interpret Results. ...
  7. Step 7: Take Action.
18 Jan 2021

How many KPIs should you have? ›

As already mentioned, the aim is to have two to four KPIs per goal. Some goals will need only one KPI; others will have four. However, exceeding four KPIs is not recommended.

Which can be most important 10 KPI for an event? ›

23 Important KPIs for Measuring Event Success
  • Event Check-ins. This key metric directly indicates the number of attendees who have checked in at the event. ...
  • Event Surveys. ...
  • Net Promoter Score (NPS) ...
  • % promoters – % detractors = NPS. ...
  • Active Community Members. ...
  • Messages Sent. ...
  • Speaker Engagement. ...
  • Session Analytics.
3 Jun 2022

How can testing efficiency be improved? ›

How to Increase Test Efficiency with TestQuality
  1. Step 1: Examine the test failure data. ...
  2. Step 2: Repair or remove flaky tests. ...
  3. Step 3: Comprehend your tests. ...
  4. Step 4: Remove outdated tests. ...
  5. Step 5: Arrange the tests in order of importance. ...
  6. Step 6: Run tests in descending order of priority.
15 Mar 2022

What is the best metric for ensuring quality QA? ›

Some of these QA metrics examples are:
  • Number of tests in a certain time period = Number of tests run/Total time.
  • Test design efficiency = Number of tests designed/Total time.
  • Test review efficiency = Number of tests reviewed/Total time.
  • Number of bugs per test = Total number of defects/Total number of tests.
2 Sept 2020

What is efficiency testing? ›

Definition: Efficiency testing tests the amount of resources required by a program to perform a specific function. In software companies, this term is used to show the effort put in to develop the application and to quantify its user-satisfaction.

What is the KPI of QA call center? ›

Three KPIs to measure in your QA program include average handle time (AHT), Net Promoter Score (NPS), and first call resolution (FCR).

What is the most important test metric? ›

Bug Find Rate: One of the most important metrics used during the test effort percentage is bug find rate. It measures the number of defects/bugs found by the team during the process of testing.

Which tool is best for performance testing? ›

Top Performance Testing Tools
  • WebLOAD.
  • LoadNinja.
  • HeadSpin.
  • ReadyAPI Performance.
  • LoadView.
  • Keysight's Eggplant.
  • Apache JMeter.
  • LoadRunner.
24 Sept 2022

What makes a good test case? ›

Typically, test cases should be small, isolated and atomic. Test cases should be easy to understand and steps should be executed fast. They should independent with each other & fail/pass independently from one another. Fairly, each good test should have defined its expected result.

What is a strategic KPI? ›

Key Performance Indicators (KPIs) are the elements of your plan that express what you want to achieve by when. They are the quantifiable, outcome-based statements you'll use to measure if you're on track to meet your goals or objectives. Good plans use 5-7 KPIs to manage and track the progress of their plan.

How do I write my own KPI? ›

How to write and develop key performance indicators
  1. Write a clear objective for each one.
  2. Share them with all stakeholders.
  3. Review them on a weekly or monthly basis.
  4. Make sure they are actionable.
  5. Evolve them to fit the changing needs of the business.
  6. Check to see that they are attainable (but add a stretch goal)

What is a KPI report example? ›

Examples of KPI Reporting Dashboards and Templates

Leads (number of leads, the evolution of the number of leads, etc.) Website traffic and sessions. Top-performing post (engagement, clicks, cost per click) Cost per click or cost per acquisitions.

What is an actionable KPI? ›

So an actionable KPI is a revenue-centric metric (or metrics) with a clearly defined target.”

What are 4 types of quality control? ›

The four types of quality control are process control, control charts, acceptance sampling, and product quality control.

How can quality KPI be improved? ›

Plan, think and manage your way to a better KPI process in the new year.
  1. Spend time planning for KPI implementation. ...
  2. Think about unintentional consequences. ...
  3. Limit your focus to 10 indicators or metrics or less at each level of the organization. ...
  4. Manage behaviors, measure results. ...
  5. Cascade indicators down from corporate goals.
5 Jan 2015

What are the 4 aspects of SLA? ›

The main elements of a good SLA.
  • Overall objectives. The SLA should set out the overall objectives for the services to be provided. ...
  • Description of the Services. The SLA should include a detailed description of the services. ...
  • Performance Standards. ...
  • Compensation/Service Credits. ...
  • Critical Failure.

Is SLA a KPI? ›

The difference between SLAs and KPIs

An SLA is an agreement between you and your customer that defines how your relationship will work in the future. Key performance indicators (KPIs) are the metrics chosen to gauge how well a team performed against agreed standards.

What is SLA P1 P2 P3? ›

P1 – Priority 1 incident tickets (Critical) P2 – Priority 2 incident tickets (High) P3 – Priority 3 incident tickets (Moderate) P4 – Priority 4 incident tickets (Low) SLA success rate is given as percentage. 'Red' colour indicates that we have failed to. achieve SLA cut-off in that particular period. '

What matters most while testing? ›

Asking questions is the most important part of Software Testing. If you fail at it, you are going to lose an important bunch of information. Questions can be asked: To understand requirement.

What are the 3 types of KPIs? ›

Types of KPIs include: Quantitative indicators that can be presented with a number. Qualitative indicators that can't be presented as a number. Leading indicators that can predict the outcome of a process.

What is KPI and KRA? ›

When it comes to measuring success, then KPI and KRA are two key measurable values that help business owners to gauge their success and progress. KRA is Key Result Area or also known as the Key Responsibility Area and KPI is Key Performance Indicators.

What are KPIs for employees? ›

Key Performance Indicators (KPIs) are metrics that can assist in tracking the ability of your employees to meet your expectations as well as their impact on the business objectives.

What are the 12 type of Key Performance Indicators? ›

12 Types of Key Performance Indicators
  • Quantitative indicators. ...
  • Qualitative indicators. ...
  • Leading indicators. ...
  • Lagging indicators. ...
  • Input indicators. ...
  • Output indicators. ...
  • Process indicators. ...
  • Practical indicators.
18 Aug 2022

What is a strategic KPI? ›

Key Performance Indicators (KPIs) are the elements of your plan that express what you want to achieve by when. They are the quantifiable, outcome-based statements you'll use to measure if you're on track to meet your goals or objectives. Good plans use 5-7 KPIs to manage and track the progress of their plan.

How do you write KPI and KRA? ›

How to WRITE SMART Goals (KRAs)?
  1. Identify the appropriate Organization KPIs where the employee can contribute meaningfully. ...
  2. Document the responsibilities of the employee in relation to the work they are doing. ...
  3. The unique responsibilities found in step 2 above will help shortlist KRAs to be used for this employee.

What is KPI and KRA and SLA? ›

KPIs provide information on the efficiency and success in meeting organizational goals or expectations. While SLAs are used to ensure that service level metrics don't fall below certain metrics criteria, KPIs help ensure that specific performance improvements and results are met adequately or exceedingly.

What are key result areas? ›

Definition: Key result areas or KRAs refer to the general metrics or parameters which the organisation has fixed for a specific role. The term outlines the scope of the job profile, and captures almost 80%-8% of a work role.

What is smart KPI? ›

The acronym “SMART KPI” stands for “Key Performance Indicators” which are “Specific, Measurable, Attainable, Relevant, and Time-Bound.” SMART KPIs are measurable metrics used to assess employee and company performance. When companies talk about SMART KPIs, what they mean is that KPIs should be: Specific.

How many KPIs should you have? ›

As already mentioned, the aim is to have two to four KPIs per goal. Some goals will need only one KPI; others will have four. However, exceeding four KPIs is not recommended.

What is an achievement indicator? ›

They are variables that reveal the attributes, qualities, properties, conditions, requirements, to be checked for each result to determine if it has been achieved or not.

What are learning outcomes and indicators? ›

The primary difference between student outcomes and performance indicators is that student outcomes are intended to provide general information about the focus of student learning and are broadly stated of the outcome, not measurable, while performance indicators are concrete measurable performances students must meet ...

What does KPI stand for in education? ›

Higher education collects an exorbitant amount of information that can be used to measure the performance of students, faculty, facilities and the institution as a whole, but where do educators even begin? Proving value and analyzing performance starts by identifying your key performance indicators (KPIs).

Videos

1. Top CUSTOMER SUCCESS KPIs for Managing CSM Teams
(CSM Practice)
2. Building the Salesforce Center of Excellence – Different Operating Models 20201120 2059 1 1
(CRMCOE)
3. Zen Master: 50 Reasons Your Organization Should Have a Center of Excellence
(Tableau Software)
4. Webinar: Taking a Product From Good to Great by Centercode Product Director, Chris Rader
(Product School)
5. Ways to Measure and Improve KPI
(Progenesis Academy)
6. Putting Excellence In Your Testing CoE!
(Checkpoint Technologies)
Top Articles
Latest Posts
Article information

Author: Duane Harber

Last Updated: 02/03/2023

Views: 6218

Rating: 4 / 5 (71 voted)

Reviews: 86% of readers found this page helpful

Author information

Name: Duane Harber

Birthday: 1999-10-17

Address: Apt. 404 9899 Magnolia Roads, Port Royceville, ID 78186

Phone: +186911129794335

Job: Human Hospitality Planner

Hobby: Listening to music, Orienteering, Knapping, Dance, Mountain biking, Fishing, Pottery

Introduction: My name is Duane Harber, I am a modern, clever, handsome, fair, agreeable, inexpensive, beautiful person who loves writing and wants to share my knowledge and understanding with you.