advertisement

Iwsm2014 defect density measurements using cosmic (thomas fehlmann)

67 %
33 %
advertisement
Information about Iwsm2014 defect density measurements using cosmic (thomas fehlmann)
Software

Published on October 9, 2014

Author: COSMIC-FSM

Source: slideshare.net

Description

IWSM Presentation
advertisement

1. 1 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Defect Density Measurements Using COSMIC Thomas M. Fehlmann, Zürich Eberhard Kranich, Duisburg Euro Project Office AG E: info@e-p-o.com H: www.e-p-o.com

2. 2 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Dr. Thomas Fehlmann  1981: Dr. Math. ETHZ  1991: Six Sigma for Software Black Belt  1999: Euro Project Office AG, Zürich  2001: Akao Price 2001 for original contributions to QFD  2003: SwissICT Expert for Software Metrics, ICTscope.ch  2004: Member of the Board QFD Institute Deutschland – QFD Architect  2007: CMMI for Software – Level 4 & 5  2011: Net Promoter® Certified Associate  2012: Member of the DASMA Board  2013: Vice-President ISBSG  1981: Dr. Math. ETHZ  1991: Six Sigma for Software Black Belt  1999: Euro Project Office AG, Zürich  2001: Akao Price 2001 for original contributions to QFD  2003: SwissICT Expert for Software Metrics, ICTscope.ch  2004: Member of the Board QFD Institute Deutschland – QFD Architect  2007: CMMI for Software – Level 4 & 5  2011: Net Promoter® Certified Associate  2012: Member of the DASMA Board  2013: Vice-President ISBSG Dr. Thomas Fehlmann

3. 3 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions  Mathematics and Computer Science  Emphasis on Mathematical Statistics  Mathematical Optimization  Theory of Polynomial Complexity of Algorithms  Working at T-Systems International GmbH in Bonn, Germany  Six Sigma Black Belt for Software Development  Software Quality Assurance Manager  Member of the DASMA Board Eberhard KranichEberhard Kranich  Mathematics and Computer Science  Emphasis on Mathematical Statistics  Mathematical Optimization  Theory of Polynomial Complexity of Algorithms  Worked at T-Systems International GmbH in Bonn, Germany  Six Sigma Black Belt for Software Development  Software Quality Assurance Manager  Member of the DASMA Board

4. 4 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions What is a Defect?  Defect = Behavior impacting expected or required functionality of software  How many bugs?  By counting the size of defect repositories?  By number of entries???

5. 5 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Software Testing as a Game  Tester sees selected sequences in the UML sequence diagram  Tester can “walk” the data movements when planning or executing tests  Functionality becomes visible to the agile team  Defects impacting functionality become visible to testers Functional Process Other Application Some Device 8.// Move some data 9.// Move some data 10.// Move some data 11.// Move some data Other Device

6. 6 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Functionality, Defect Size, and Defect Density  What happens if data movements have defects?  Testers mark the data movement where a defect has been detected  Same Metric:  ISO/IEC 19761 COSMIC Functional Process Other Application Some Device 8.// Move some data Move some data 10.// Move some data 11.// Move some data Other Device  Functional Size  Number of Data Movements needed to implement all FUR  Test Size  Number of Data Movements executed in Tests  Test Story  Collection of Test Cases aiming at certain FURs  Defect Count  Number of Data Movements affected by some defect detected in a test story

7. 7 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Defects Density Prediction?  Now he counts the defects!  And counts and adjusts test size  By ISO/IEC 19761 COSMIC Functional Process Other Application Some Device 8.// Move some data 9.// Move some data 10.// Move some data 11.// Move some data Other Device How does he know that he found all the defects?

8. 8 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Research Questions  What is Defect Density?  Defects per KDLOC?  What is Test Coverage?  Code lines executed by some test case?

9. 9 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions SW Testing and SW Metrics  Counting practices for defect counting are undocumented  “Number of Defects Found” per Stages / with Tests / etc.  How do you count “Number of Defects”?  Is it simply the number of entries in a defect repository?  How can you avoid double reporting?  Or make sure two defects are reported twice and not in a single report?  A successor to the “Defect Measurement Manual” published by UKSMA in October 2000 is under review: “Defect Measurement and Analysis Handbook”  By European cooperation  Important enhancement for ISBSG’s Data Collection!

10. 10 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions SW Testing and SW Metrics  Counting practices for defect counting are undocumented  “Number of Defects Found” per Stages / with Tests / etc.  How do you count “Number of Defects”?  Is it simply the number of entries in a defect repository?  How can you avoid double reporting?  Or make sure two defects are reported twice and not in a single report?  A successor to the “Defect Measurement Manual” published by UKSMA in October 2000 is under review: “Defect Measurement and Analysis Handbook”  By European cooperation  Important enhancement for ISBSG’s Data Collection!

11. 11 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Learn & Remember Software Development the Six Sigma Way Use COSMIC ISO/IEC 19761 Benchmark! Count Defects based on Reference Model

12. 12 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions As a … [Functional User] I want to … [get something done] Such that …[quality characteristic] So that … [value or benefit] 1) Q001 Propose Standard Destinations User of public transportation be able to store my preferred destinations they are valid for the Ticket Shop I no longer have to pay fees when catched without tickets 2) Q002 Find Nearest Boarding Station User of public transportation locate nearest station with GPS that's being served right now I immediately can see whether it's right 3) Q003 Process Payment Provider of transportation services give user access to their preferred payment options all payments are traceable in Ticket Shop they can manage spending 4) Q004 Issue Ticket User of public transportation get a valid ticket with settings from Ticket Shop I no longer have to pay fees when catched without tickets 5) Q005 Show Ticket Ticket controller see the validated ticket I can check validity period and travel range I don't need to go into a dispute with a client Functional User Requirements Example: The Ticket Apps  Customer’s Voice:  Give me a ticket subito! Without much ado and questions if I simply want to go home

13. 13 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions The Ticket Apps 12 Entry (E) + 9 eXit (X) + 1 Read (R) + 2 Write (W) = 24 CFP App User Home Destinations Saved Stations Ticket Shop Ticket Purchase Phone's GPS GIS Application Timetable Service Loca 1.// Add New Destination Traveler 2.// Check whether Destination Exists 3.// Collect Matching Destinations 4.// Show Matching Destinations 5.// Select Exact Destination 6.// Record Selected Destinations 7.// Ask App for a Ticket 8.// Propose Destinations 9.// Proposed Destinations 10.// Select Destination Prepare 11.// GPS Coordinates 12.// Date & Time 13.// Ask GIS 14.// Nearest Boarding Station

14. 14 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions The Ticket Apps – Objects of Interest Description Type Wants to buy a ticket to ride Device User Store a list of destinations that are valid for "getting home" Functional Process List of standard stations eligible as travel destination Persistent Data Finds nearest origin station Other Application Buy the ticket needed for that route Functional Process External ticket shop able to identify the App’s user Other Application Finds connections between stations at a given time & day Other Application Provides GPS coordinates where phone is located Device User Synchronized time service located in smartphone Device User App User Home Destinations Ticket Purchase Ticket Shop Phone's GPS Timetable Service Saved Stations GIS Application Local Time

15. 15 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions The Ticket Apps – Prepare Destinations App User Ticket Shop 1.// Add New Destination Prepare 2.// Check whether Destination Exists 3.// Collect Matching Destinations 4.// Show Matching Destinations 5.// Select Exact Destination Home Destinations Saved Stations 6.// Record Selected Destination  Preparation:  Store standard destination stations on Smartphone  After checking with the Ticket Shop whether they exist or which stop to select  Involves Ticket Shop and local data ‘Standard Stations”

16. 16 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions The Ticket App in the IFPUG Model Boundary IFP=51 EI EO EQ ILF EIF EQ 1 / 2 Select Destination EI 1 / 3 Enter New Destination EI 2 / 2 Request Ticket EI 2 / 1 Confirm Purchase EIF 2 / 12 Ticket Shop EIF 1 / 3 GIS Application EIF 1 / 5 Timetable Service EIF 1 / 1 Local Time EO 2 / 2 Boarding Station EO 3 / 3 Mobile Ticket EO 1 / 2 Show Matching Destinations ILF 1 / 2 Home Destinations  Less complicated than UML Sequence Diagrams  Suits business better  Better boundary identification between different layers

17. 17 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Prepare Destination in the IFPUG Model  It is not obvious what happens  Two elementary process seem unrelated  Nevertheless, they update an ILF  [Uh, they forgot the elementary process for deleting obsolete home destinations!] Boundary IFP=19 EI EO EQ ILF EIF ILF 1 / 2 Home Destinations EO 1 / 2 Show Matching Destinations EI 1 / 3 Enter New Destination EIF 2 / 12 Ticket Shop

18. 18 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Buy Ticket in the IFPUG Model  Elementary processes stay  The ILF ‘Home Destinations’ becomes an EIF  Adding the two counts yields 19 + 42 > 51  IFPUG is useless for functional sizing when developing Apps or counting defects  IFPUG 4.3 is great for identifying system boundaries splitting apps! Boundary IFP=42 EI EO EQ ILF EIF EIF 2 / 12 Ticket Shop EIF 1 / 3 GIS Application EIF 1 / 5 Timetable Service EIF 1 / 1 Local Time EIF 1 / 2 Home Destinations EI 2 / 2 Request Ticket EI 1 / 1 Confirm Purchase EO 2 / 2 Boarding Station EO 2 / 3 Mobile Ticket EQ 1 / 2 Select Destination

19. 19 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Requirements for a Defect Measurement Reference Model  It must be additive  The total size must be the sum of the components’ sizes.  It must be understandable  Support UML Sequence Diagrams  Works well with UML Use Cases  Also with Business Process Model & Notation (BPMN) 2.0  It must fit into agile delivery  Buglione-Trudel Matrix for managing agile projects  Story Cards for Sprints Story Card for Prepare Destinations Test is Ready Draft is Ready Review Done Final- ized Appro- ved Func- tional R001-01: Prepare Destinations 6 x reworked Refactoring Count: 5 0 0 Business Impact: Functional Size: Story Points: App User Prepare Destinations Standard Stations Ticket Shop 1.// Add New Destination 2.// Check whether DestinationExists 3.// Collect Matching Destinations 4.// Show Matching Destinations 5.// Select Exact Destination 6.// Record Selected Destinations The user prepares his ticket app by entering standard destinations. The ticket shop is involved for resolving station names matches and avoid spelling errors

20. 20 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Learn & Remember Software Development the Six Sigma Way Use COSMIC ISO/IEC 19761 Benchmark! Count Defects based on Reference Model

21. 21 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions What is a Test?  A Software Test has  Several Test Stories • Weighted by Customer’s Priority for the Test Story, reflecting the value for the customer  Each Test Story has many Test Cases • For various kind of test data  A Test Size attribute • Number of data movements executed by test cases  A Test Coverage attribute • Percentage of data movements covered with test cases  An Outcome • Passed or Failed – Passed: All responses according expectations – Failed: at least one test case didn’t yield the expected response

22. 22 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Sample Test Stories  Eight Test Stories for the Ticket Apps  Each Test Story has several Test Cases  Each Test Case has defined Test Data  The Expected Response is known as per Test Case Test Story Case 1 Test Data Expected Response Case 2 Test Data Expected Response CT-A Prepare CT-A.1 Find Nearest Station CT-A.1.1 Enter GPS exactly Returns correct station CT-A.1.2 Enter GPS nearby Returns nearest station CT-A.2 Served Stations only CT-A.2.1 Select time of service Returns next available connection CT-A.2.2 Select time out of service Return next best served station CT-A.3 Enter New Destination CT-A.3.1 Enter valid station name Destination stored CT-A.3.2 Enter invalid station name Destination rejected CT-B Ticketing CT-B.1 Select Destination CT-B.1.1 Station of origin known Asks for destination CT-B.1.2 No destination selected Stops without contacting Ticket Shop CT-B.2 Get Ticket CT-B.2.1 Both stations known Asks to confirm price CT-B.2.2 Boarding station undefined Doesn't ask for destination CT-B.3 Price Calculation CT-B.3.1 Both stations known Presents price of next actual connection CT-B.3.2 Both stations known Display intermediate change stations CT-B.4 Issue Ticket CT-B.4.1 Issue successful, ticket shown All credentials visible CT-B.4.2 Ticket Shop blocked Explanation given CT-B.5 Payment Tests CT-B.5.1 Credit limit exceeded Returns exceed notice CT-B.5.2 Payment service out of order Returns warning

23. 23 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions What is a Test Case?  A Test Cases has  Entry Data (“Test Data”) • Explaining the environment for the test case • Typically valid, invalid, borderline data • Normal and disturbed communication services  A known Expected Response • The response of the system is known in advance  A known sequence of data movements executed • Defining Test Coverage • Each Test Case has a Size

24. 24 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions  The sequence of data movements can be visualized in the sequence diagram per test case  By double- clicking the respective data movements Find Route GIS Application Phone's GPS 8.// GPS Coordinates 9.// Date & Time 10.// Ask GIS 11.// Nearest Boarding Station Local Time What is a Test Case? Test Story No. 1 User Stories CT-A.1 Find Nearest Station Q001: Propose Standard Destinations Q002: Find Nearest Boarding Station Q003: Process Payment Q004: Issue Ticket Q005: Show Ticket Expected Response CT-A.1.1 Enter GPS exactly E012 E012,X009 Returns correct station CT-A.1.2 Enter GPS nearby E012,X009 Returns nearest station CT-A.1.3 Enter malformatted GPS E006 Returns no station CT-A.1.4 Enter far away GPS position E006,E007,X004,E008 E012,X009 Returns nearest station plus warning CT-A.1.5 GPS not working E005,E006 E006 Returns warning Test Story Contribution (CFP): 2 6 0 1 6 Test Size Test Case Measurements for Test Story CT-A.1

25. 25 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Defect Reporting  When a defect has been identified in a test case, it can be recorded by double-clicking the suspect sequence of data movements User Stories CT-B.1 Select Destination Q001: Propose Standard Destinations CT-B.1.1 Station of origin known R001,X001,E005 CT-B.1.2 No destination selected R001,X001 CT-B.1.3 Valid boarding & destination W001,E005,R001,X001 CT-B.1.4 List of destinations X003,R001,W001 Test Story Contribution (CFP): 12 Test Case Measurements for Test Story CT-B.1 Test Story No. 4 Expected Response CFP Name Label Description Name Label Asks for destination 6 Stops without contacting Ticket Shop 2 Returns next available connection 15 All readable and visible 3 #003 Multiple References If a station contains more than one transport mode, e.g., bus and train, both are recorded as separate destinations W001 Record Selected Destinations Test Size 26 1 Defects Observed Data Movements Affected Defect Count

26. 26 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions 12 Entry (E) + 9 eXit (X) + 1 Read (R) + 2 Write (W) = 24 CFP App User Home Destinations Saved Stations Ticket Shop Ticket Purchase Phone's GPS GIS Application Timetable Service 1.// Add New Destination Traveler 2.// Check whether Destination Exists 3.// Collect Matching Destinations 4.// Show Matching Destinations 5.// Select Exact Destination 6.// Record Selected Destinations 7.// Ask App for a Ticket 8.// Propose Destinations 9.// Proposed Destinations 10.// Select Destination Prepare 11.// GPS Coordinates 12.// Da 13.// Ask GIS 14.// Nearest Boarding Station Extract Report Select Data Movements for Test Case CT-B.1.4: List of destinations when executed in view of Q001: Propose Standard Destinations This Test Case identifies Fault #003: Multiple References affecting Data Movement 6.// Record Selected Destinations Finish Record Defects

27. 27 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Count Defects  You can count One Defect  Per Data Movement  Identified by One Test Story And never more… ©2012-2014 Rhafiel

28. 28 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Compare  This means  The more test stories you have, the more defects you can uncover  Test size is equally important as functional size  Defect density depends from test size  Test Benchmarking Test Status Summary Total CFP: 24 Defects Pending for Removal: 3 Test Size in CFP: 185 Defects Found in Total: 3 Test Intensity in CFP: 7.7 Defect Density: 13% Test Coverage: 79%

29. 29 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Test Stories GoalTestCoverage FindNearestStation ServedStationsonly EnterNewDestination SelectDestination GetTicket PriceCalculation IssueTicket PaymentTests AchievedCoverage CT-A.1 CT-A.2 CT-A.3 CT-B.1 CT-B.2 CT-B.3 CT-B.4 CT-B.5 Q001 Propose Standard Destinations 0.52 2 9 12 8 9 7 0.55 Q002 Find Nearest Boarding Station 0.45 6 9 7 10 3 5 4 0.45 Q003 Process Payment 0.62 3 4 4 4 10 7 16 0.59 Q004 Issue Ticket 0.24 1 3 1 3 6 2 3 0.23 Q005 Show Ticket 0.29 6 3 2 1 9 3 3 0.31 Ideal Profile for Test Stories 0.18 0.20 0.26 0.40 0.38 0.51 0.36 0.40 Convergence Gap 0.18 0.2 0.2 0.4 0.4 0.5 0.4 0.4 0.06 0.10 Convergence Range 0.20 Convergence Limit Test Stories Deployment Combinator User Stories Toggle Measured Controls Measuring Test Coverage Achieved Response

Add a comment

Related presentations

Speaker: Matt Stine Developing for the Cloud Track Marc Andressen has famou...

This presentation explains how to develop a Web API in Java using (JAX-RS or Restl...

1 App,

1 App,

November 10, 2014

How to bring innovation to your organization by streamlining the deployment proces...

Cisco Call-control solutions can handle voice, video and data

Nathan Sharp of Siemens Energy recently spoke at the SAP Project Management in Atl...

Related pages

Iwsm2014 defect density measurements using cosmic (thomas ...

1. 1 Customer Orientation Lean Six Sigma Agile Processes Project Estimations Transfer Functions Defect Density Measurements Using COSMIC Thomas M. Fehlmann ...
Read more

Iwsm2014 defect density measurements using cosmic (thomas ...

... Density Measurements Using ... Density Measurements Using COSMICThomas M. Fehlmann, ... defect density measurements using cosmic ...
Read more

dblp: IWSM/Mensura 2014

Bibliographic content of IWSM/Mensura 2014. ... Defect Density Measurements Using COSMIC ... Thomas Fehlmann, ...
Read more

Density Measurements | LinkedIn

High pressure density measurements and modeling of fluids with the vibrating tube thechnique at Instituto Politécnico Nacional Past
Read more

Measurements density - Documents

Download Measurements density. Transcript. 1. Measurement and Units Density. 2. Density .
Read more

Defect Density | LinkedIn

Current Manager/ Line Block Stability (Yield Defect Density) at Infineon Technologies Past Principal Engineer/ Yield Defect Density at GLOBALFOUNDRIES ...
Read more

Iwsm2014 cosmic approximate sizing using a fuzzy logic ...

Iwsm2014 cosmic approximate sizing using a fuzzy logic approach (alain abran) by nesma
Read more

COSMIC - Common Software Measurement International ...

Using cosmic in agile projects. 5 months ago. Software. ... Iwsm2014 measuring cosmic software size from functional execution traces of java business...
Read more