Presented at
BSides Austin 2016,
March 31, 2016, 12:30 p.m.
(60 minutes).
Abstract
Many mature SOC's are struggling in 3 key areas when it comes to personnel: Providing continuous analyst training, measuring effective KPI's for analyst performance and extending employee retention. In this talk we introduce gamification and machine learning as a concept that can solve these challenges. Leveraging technology and qualitative assessment we believe we've gamified security operations.
Description
Gamification is a concept often used in marketing to engage users by applying game mechanics to non-game related contexts such as referring a new product to friends and family. The desired behavior is rewarded by handing out points, badges or other incentives. In this talk we explore the options to transfer this concept to the daily operations of a world-wide 24x7 Security Operation Center.
A common problem in security operations is the short retention period of the Level-1 Intrusion Analysts often caused by job frustration. The resulting high turnover rate requires a constant training program and creates teams with varying levels of experience and seniority which is difficult to manage.
By collecting various quantitative and qualitative metrics in real-time we try to provide an interesting and challenging work environment which offers meaningful feedback and instant gratification for good intrusion analysis. For the metrics analysis we apply machine learning algorithms to maximize the level of automation. As a side effect this creates a comprehensive key performance indicator measurement framework for evaluating Analyst performance. Even with an extended retention period turnover will eventually happen. That is why we also introduced an Analyst proficiency program for continuous Analyst training tracked by the progress mechanics known from computer games.
At the end we present the challenges we faced while introducing this program as a proof-of-concept into a large global security organization and the lessons we have learned during the implementation. We also look at potential downsides of this approach, e.g. from creating a highly competitive environment, the over-dependence on extrinsic motivators and even....analysts gaming the system.
Presenters:
-
Joshua Stevens
An innovator and subject matter expert in vulnerability management and SIEM integration, Josh has more than a decade of experience in security, with specialization in perimeter defense, event analysis and incident response. Stevens is a featured SANS presenter, with multiple patents pending for visualizing cyber-attack data. He co-created and open sourced an application in 2011 under the MIT license, offering IT Operations with visibility into scanning activity. In his current role at Hewlett Packard Enterprise, he serves as Chief Architect for Security Operations, responsible for providing strategic and technical direction for the Cyber Defense Center with a primary focus on product evaluations, applied research and HPE-on-HPE projects.
Links:
Similar Presentations: