Published on January 20, 2014
By Harmony Brenner January 21, 2014
“Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.” ― H. James Harrington
Agenda Goals Definitions Pre-Requisites Process Requirements Metrics and Exclusions Calculate DRE Metrics Calculate Rework Metrics Presenting Results Metrics Usage and Sharing
Goals Increase attention given to all phases of testing (Unit, System, User Acceptance) to improve quality. Decrease number of defects detected in later phases of testing to reduce rework and overall costs. Implement metrics as an objective method to measure outcomes and improvements over time.
Definitions DRE – Defect Removal Effectiveness. Metrics – Use of numerical ratings to determine quality and performance of item / phase being measured. Also define any Testing phases and describe who / which team(s) own accountability. Examples – SIT is usually done by IT Team; UAT is usually performed by Business Team
Pre-Requisites Ensure your Defect Tracking system contains discrete categories for the following: Statuses Reasons / outcomes for defect closure Impact Levels Root Causes Ability to track Cycles of Rework
Process Requirements Encourage consistent creation and updates of defects by IT and Business teams immediately upon discovery. Mentor teams how to create effective defect reports. Foster communication between teams to ensure that all release defects are worked through to completion. Defect Tracking Training Resolve other barriers to acceptance
Metrics and Exclusions Decide and describe whether any Defect Statuses should be excluded from measurement. Example – statuses indicate the defect should not have been created may merit exclusion. Cancelled, No Action Required, etc.
Metrics and Exclusions (cont’d) Decide and describe whether any Defect Root Causes should be excluded from measurement. Example – categories that indicate the defect should not have been created may merit exclusion. Change Request, User Error, etc.
Calculate DRE Metrics Calculate the DRE (Defect Removal Effectiveness) for each test phase in the project to be measured: Phase DRE = Phase Defects / (Phase Defects + All Defects) SIT Defects (10) / (SIT Defects (10) + UAT Defects (5) ) 10 / 15 = 67% is SIT DRE 5 / 15 = 33% is UAT DRE
Calculate Rework Metrics Decide how to track Rework instances for defects. Example – Audit trail may count changes between Retest vs. Rework statuses. Where possible, automate Rework counts per defect so these can be calculated vs. manually determined.
Presenting Results Share results in a non-judgmental framework. Anonymize and de-personalize outcomes wherever possible. This is team, not individual, performance. Give credit where due and be a change champion!
Metrics Usage and Sharing Always tell yourself the truth! Use metrics to raise awareness on overall quality – not as punishment! Share metrics at regular intervals – no surprises. Compare metrics over time on the same project or among different projects to measure performance.
Want to Learn More? I enjoy professional discourse on a variety of subjects pertaining to Software Quality and Testing. Harmony Brenner – firstname.lastname@example.org LinkedIn – www.linkedin.com/in/harmonybrenner
Creating Meaningful Defect Metrics by Harmony Brenner This is a presentation on how to create and implement meani...
Share Creating meaningful credentials. ... Creating Meaningful Defect Metrics by Harmony Brenner. ... Creating a Meaningful Security Awareness Program.
LEAN SIX SIGMA METRICS: HOW TO MEASURE IMPROVEMENTS ... on a metric that is meaningful to the ... metrics Did you succeed in creating ...
Slide 1 CREATING A MEANINGFUL CAREER CENTER METRICS DASHBOARD Joslyn DiRamio Bedell & Joe Testani Virginia Commonwealth University Slide 2 CHALLENGE: ...
Download Creating meaningful credentials. Transcript. Recommended. Creating Meaningful Brands Creating Meaningful Rubrics
View Harmony Brenner, ISTQB ... Responsible for creating and reviewing recommended ... Performed defect root cause analysis to trace defect source in ...
KEY PERFORMANCE INDICATORS Function ... relate to manufacturing metrics. Since performance measurements for maintenance must include both results
Quality Metrics Why are we going… Where are we going ... Currently have limited “practical” access to most meaningful data