The recent rapid growth of artificial intelligence (AI), and especially machine learning (ML), entices many cybersecurity professionals into believing these popular techniques are a silver bullet solution to our biggest cybersecurity challenges. Unfortunately, many cyber efforts focus more on using data-driven ML without a clear strategy to carefully measure results derived from the models. Furthermore, AI is an area of advanced computer science that requires experienced analysis, careful data preparation, and measured expectations of model performance and risks. However, it is oftentimes used as a synonym for automation.
Deloitte has performed enterprise-scale cyber analytics at the performance level of 1M events per second / 300B events per month for the last several years. In performing these analyses, Deloitte developed several advanced cyber analytic workflows that utilize ML, contextual enrichment, and data science to support mission-centric cyber hunt operations. Experience has shown analytical workflows are more impactful and relevant when they are defined not by the models used, but by the mission-oriented results they produce.
In this talk, we describe a results-driven perspective of AI and ML applicable to real-world challenges in cyber analytics with a particular focus on detection of malicious actors and behaviors as well as advanced network understanding. We will discuss how analytical workflows utilize AI models and other techniques to define normal and anomalous behaviors; manage the base rate fallacy; integrate cyber-domain knowledge into data-driven models to provide context and explainability; and provide advanced decision-support to human cyber operators and threat hunters.
We will conclude our talk with stories of where AI has effectively supported cybersecurity analysis, where AI has failed in cybersecurity and analytic applications, and recommendations for moving beyond current data-driven AI models focused on classification and toward results-driven AI approaches which are robust to contextualization, uncertainty, and ambiguity in the cyber security domain.