OWASP AppSec2006 Seattle Security Metrics

67 %
33 %
Information about OWASP AppSec2006 Seattle Security Metrics

Published on August 30, 2007

Author: Barbara

Source: authorstream.com

The OWASP Application Security Metrics Project:  The OWASP Application Security Metrics Project Bob Austin Application Security Metrics Project Lead KoreLogic, Inc. bob.austin@korelogic.com 804.379.4656 Presentation Objectives:  Presentation Objectives Drivers for Security Metrics Review the Project Plan. Work Accomplished To Date, Next Steps Provide Application Security Metrics Resources Solicit Feedback and Participation Slide3:  The Best Metrics….Can Answer Hard Questions How secure am I? Am I better than this time last year? Am I spending the right amount of money? How do I compare to my industry peers (senior management’s favorite question)? Source: Dr. Dan Geer Slide4:  Forrester Survey: 'What are your top three drivers for measuring information security?' Source: 'Measuring Information Security Through Metrics And Reporting', Forrester Research, Inc., May 2006' Report progress to business Better stewardship Base: 40 CISOs and senior security managers Slide5:  Forrester Survey: What do CISOs want to measure? 'As a CISO, if you have a choice of measuring and monitoring up to five areas in security, which ones would you measure?' Base: 34 CISOs and senior security managers. Source: 'Measuring Information Security Through Metrics And Reporting', Forrester Research, Inc., May 2006' Slide6:  Project Goal and Roadmap     Phase One Phase Two Project Goal: Address the current lack of effective application security metrics by identifying, sharing and evolving useful metrics and metric processes to benefit the OWASP community. Current Project Contributors: Jeff Williams (Aspect Security), Cliff Barlow (KoreLogic), Matt Burton (Mitre) http://www.owasp.org/index.php/Category:OWASP_Metrics_Project  Current Project Status  Slide7:  Phase One – Application Security Metrics Baseline Survey Plan Information Capture Analysis Survey Results    http://www.owasp.org/index.php/Metrics_Survey_Form Slide8:  Useful Resources from Research OWASP CLASP Project – 'Monitor Security Metrics' Dr. Dan Geer’s 'Measuring Security' Tutorial Other Initiatives: Securitymetrics.org, Metricon 1.0 Secure Software Development Life Cycle: 'The Security Development Lifecycle', Howard and Lipner, 'Security in the Software Lifecycle', DHS, Cybersecurity Div. Information Security Metrics Standard - ISO 27004 Dr. Larry Gordon, Cybersecurity Economics Research Projects Resources from NIST: Security Metrics Guide for Information Technology Systems, Guide for Developing Performance Metrics for Information Security NIST Software Assurance Metrics and Tool Evaluation (SAMATE) Organizing Metric Types:  Organizing Metric Types Process Metrics Information about the processes themselves. Evidence of maturity. Vulnerability Metrics Metrics about application vulnerabilities themselves Management Metrics specifically designed for senior management Examples Secure coding standards in use Avg. time to correct critical vulnerabilities Examples By vulnerability type By occurrence within a software development life cycle phase Examples % of applications that are currently security 'certified' and accepted by business partners Trending: critical unresolved, accepted risks Slide10:  Opportunities for Metrics - Secure Development Life Cycle (SDL) Secure questions during interviews Concept Designs Complete Test plans Complete Code Complete Deploy Post Deployment Threat analysis Security Review Team member training Data mutation andamp; Least Priv Tests Review old defects Check-ins checked Secure coding guidelines Use tools Learn andamp; Refine External review Source: Microsoft Were software assurance activities conducted at each lifecycle phase? Slide11:  Examples of Application Security Metrics Process Metrics Is a SDL Process used? Are security gates enforced? Secure application development standards and testing criteria? Security status of a new application at delivery (e.g., % compliance with organizational security standards and application system requirements). Existence of developer support website (FAQ's, Code Fixes, lessons learned, etc.)? % of developers trained, using organizational security best practice technology, architecture and processes Management Metrics % of applications rated 'business-critical' that have been tested. % of applications which business partners, clients, regulators require be 'certified'. Average time to correct vulnerabilities (trending). % of flaws by lifecycle phase. % of applications using centralized security services. Business impact of critical security incidents. Examples of Application Security Metrics:  Examples of Application Security Metrics Vulnerability Metrics Number and criticality of vulnerabilities found. Most commonly found vulnerabilities. Reported defect rates based on security testing (per developer/team, per application) Root cause of 'Vulnerability Recidivism'. % of code that is re-used from other products/projects* % of code that is third party (e.g., libraries)* Results of source code analysis**: Vulnerability severity by project, by organization Vulnerabilities by category by project, by organization Vulnerability +/- over time by project % of flaws by lifecycle phase (based on when testing occurs) Source: * WebMethods, ** Fortify Software The Path Forward:  The Path Forward Complete KoreLogic-sponsored surveys Encourage others to complete survey forms Create metrics taxonomy. Test drive it. Collaborate/share with other metrics initiatives 'Will Work for Metrics'. Volunteers needed! Solicit survey participants. Collect survey data. Help analyze survey data Donate useful application security metrics Help plan Phase Two Slide14:  Our Security Metrics Challenge 'A major difference between a 'well developed' science such as physics and some of the less 'well-developed' sciences such as psychology or sociology is the degree to which things are measured.' Source: Fred S. Roberts, ROBE79 'Give information risk management the quantitative rigor of financial information management.' Source: CRA/NSF, 10 Year Agenda for Information Security Research, cited by Dr. Dan Geer Supplemental Slides and Metrics Resources:  Supplemental Slides and Metrics Resources Resources – Security Metrics:  Resources – Security Metrics Security Metrics Standards: ISO 27004 - a new ISO standard on Information Security Management Measurements. Other metrics initiatives - Securitymetrics.org Metricon 1.0 presentations, http://www.securitymetrics.org/content/Wiki.jsp?page=Metricon1.0 Dan Geer’s measuringsecurity tutorial. Pdf, http://geer.tinho.net/usenix Developing metrics programs: Security Metrics Guide for Information Technology Systems, http://csrc.nist.gov/publications/nistpubs/800-55/sp800-55.pdf Guide for Developing Performance Metrics for Information Security, http://csrc.nist.gov/publications/drafts/draft-sp800-80-ipd.pdf Establishing an Enterprise Application Security Program, Tony Canike, OWASP 2005 Metrics-related Tools: NIST Software Assurance Metrics and Tool Evaluation (SAMATE), http://samate.nist.gov/index.php/Main_Page Metrics-related Models, Frameworks: http://www.sse-cmm.org/model/model.asp Current Articles on Metrics www.csoonline.com/metrics/index.htm Metric-related Financial and Econometric Resources: Economics and Security Resource Page, Ross Anderson), http://www.cl.cam.ac.uk/~rja14/econsec.html Dr. Larry Gordon, University of Maryland, Cybersecurity Economics Research Projects, http://www.rhsmith.umd.edu/faculty/lgordon/Cybersecurity%20Economics%20Research%20Projects.html Resources – Software Assurance:  Resources – Software Assurance 'A Clinic to Teach Good Programming Practices', Matt Bishop, http://nob.cs.ucdavis.edu/bishop/talks/2006-cisse-2/clinic.html Team Software Process for Secure Systems (TSP-Secure), http://www.sei.cmu.edu/tsp/tsp-security.html OMG’s Software Assurance Workshop 2007, http://www.omg.org/news/meetings/SWA2007/call.htm DHS Cybersecurity Division Software Assurance Initiatives: Software Assurance Measurement Workshop, Oct, 2006 Software Assurance Program, http://www.psmsc.com/UG2006/Presentations/11_DHS_SwA_Overview_for_PSM.pdf Software Assurance Forum, https://buildsecurityin.us-cert.gov/daisy/bsi/events/521.html CERT Secure Coding Standards, https://www.securecoding.cert.org/confluence/display/seccode/CERT+Secure+Coding+Standards CRA Conference on 'Grand Research Challenges in Information Security andamp; Assurance', http://www.cra.org/reports/trustworthy.computing.pdf Resources – General Software Measures & Metrics:  Resources – General Software Measures andamp; Metrics Measures and Metrics Web Sites, http://www.stsc.hill.af.mil/crosstalk/1999/06/measuresites.asp Software Process Metrics Organizations: http://www.totalmetrics.com/cms/servlet/main2?Subject=Listandamp;ID=3 http://www.swmetrics.org/ Software Metrics Symposium Capability Maturity Model Integration (CMMI) Tenth ANNUAL PSM USERS' GROUP CONFERENCE Performance and Decision Analysis, http://www.psmsc.com/UsersGroup2006.asp History of Software Measurement (Horst Zuse), http://irb.cs.tu-berlin.de/~zuse/metrics/History_00.html NASA Software Engineering Laboratory, Experience Factory: http://sel.gsfc.nasa.gov/website/exp-factory.htm ISO/IEC 15939, Software Engineering - Software Measurement Process Software Metrics Glossary, http://www.totalmetrics.com/cms/servlet/main2?Subject=Listandamp;ID=12 2006 State of Software Measurement Practice Survey, http://www.sei.cmu.edu/sema/presentations/stateof-survey.pdf Slide19:  Really Bad Metrics Advice According to my data, roughly 122.45 percent of this journal's 347,583,712 readers need some sharpening up on how to effectively collect and use metrics. There is less than a 0.0345 percent chance that this column will help. Q: I'm a manager who believes in keeping metrics simple, which is why I've limited the number we collect to 62. But I also want to simplify their collection—do you know where I can find timecard readers designed for bathroom stalls? A: Try voice print-activated stalls with timed door locks. But first, are you really trying to collect 62 metrics? 62? [snicker snort chortle] You're obviously clueless about the 'KISS' principle: Keep It Stupefyingly Strenuous. You can collect a lot more than 62 different metrics. The accepted rule of thumb for the number of metrics you can reasonably work with is this: 'Seven, plus or minus the square of the number of door knobs in your home.' Remember, if something can be measured, it must be measured, and all metrics are equally critical. Q: I feel vindicated. Now I can introduce additional metrics for every obscure area of our process improvement model. Naturally, I plan to drop the whole wad as an enforced edict and then make myself unavailable for a few weeks. A: Bravo! But be sure you don't overcomplicate things by defining every minute detail, such as data integrity standards or what you plan to do with the data. People learn nothing from constant handholding. Your job is to sit back and wait for those reliable numbers to start pouring in. Q: Great! What do you suggest I do with all that data? A: What should you do with the data? Do? That question implies that metrics are a means to some end. Don't waste resources—time spent analyzing metrics is time that could have been spent collecting even more metrics. Q: My boss keeps asking for data on stuff I don't think can be quantified—and it's often common sense stuff he could just ask us! Aren't metrics just a big sham? A: Shhh! You're right, metrics are actually an extensive conspiracy—but an extremely helpful one. When people want to make decisions based on 'facts' rather than 'opinions,' you need metrics to push your personal agenda under the guise of unassailable objectivity. Perception is everything: Politicized emotional drivel: 'Let's try my approach. Her plan isn't working.' Objective insight: 'A consumptive analysis of my plan projects a 84.67 percent increased density of pro-active rationals within six months. However, her key preambulatory vindicators are creating a 24.38 percent downward sloping polymorphic trend. Plus, she wears really cheesy business suits.' Source: http://www.stsc.hill.af.mil/crosstalk/1998/08/backtalk.asp

Add a comment

Related presentations

Related pages

File:OWASPAppSec2006Seattle Security Metrics.ppt - OWASP

Warning: This file type may contain malicious code. By executing it, your system may be compromised.
Read more

OWASP AppSec Seattle 2006/Agenda - OWASP

OWASP AppSec Seattle 2006/Agenda. From OWASP. Jump to: navigation, search. OWASP Training Courses. T1. ... OWASP Application Security Metrics
Read more

The OWASP Application Security Metrics Project | Many PPT

The OWASP Application Security Metrics ... Metrics Project 2 . OWASP AppSec Seattle ... on down to OWASP . Goals of Application Security Metrics .
Read more

OWASP Source Code Center / Mailing Lists

owasp-seattle; owasp-singapore; owasp-socal; owasp-spain; owasp-spanish; owasp-standards; ... owasp-metrics — The OWASP Metrics project mailing list
Read more

OWASP's Books and Publications Spotlight - Lulu - Online ...

... security architecture review, security testing, code review, and metrics. ... The Open Web Application Security Project (OWASP) is a 501(c)(3) ...
Read more

About OWASP! - Hack In The Box

OWASP Application Security Metrics Project OWASP Application Security Requirements Project OWASP CAL9000 Project OWASP CLASP Project
Read more

Security Metrics | Many PPT

Security Metrics Security Metrics . ... Security Metrics Project 2 . OWASP AppSec Seattle ... and Metrics Security Measures & Metrics ? 2004 ...
Read more

Mike - The OWASP Seattle Chapter (Seattle, WA) - Meetup

This is the Seattle Chapter of the Open Web Application Security Project http://www.owasp.org We have a ... The OWASP Seattle Chapter http://www.meetup.com ...
Read more