No More Fudge Factors and Made-up Shit: Performance Numbers That Mean Something

Presented at BSidesSF 2015, April 19, 2015, noon (60 minutes).

This talk isn't about security. It's about how any security team can measure and improve its overall performance, and also better satisfy your non-technical bosses and clients. Besides "are we secure?", bosses want to know "is the team performing well?" At some point, you'll probably be asked for a scorecard or dashboard - "and make it simple". Maybe you've already tried to create a scoring spreadsheet only to find that it's full of fudge factors, incomprehensible formulas, or made-up shit. There is a better way. This session presents a credible and powerful method - the Thomas Scoring System (TSS) -- to estimate an aggregate performance index from a grab bag of ground-truth metrics and evidence. TSS can help you present solid, defensible metrics to the bosses, and it can also help your team learn what really drives performance and how to improve. Several case studies will be demonstrated: Vendor Risk Assessment, Vulnerability Management, and Security Operations. TSS is Creative Commons and open source. Excel and R+Shiny tools will be released.


Presenters:

  • Russell C. Thomas - Data Scientist, PhD candidate - A financial firm
    Data Scientist at a large bank and PhD Candidate in Computational and Data Sciences at George Mason University. Focus: risk analysis, innovation ecosystem analysis, and Agent-based Modeling (ABM). Shameless crosser of boundaries and mixer of metaphors.

Links:

Similar Presentations: