Improving Alert Recall: miss fewer attacks through customizable ML anomalies

Presented at Blue Team Con 2022, Aug. 28, 2022, 10 a.m. (30 minutes)

In the ongoing game of cat and mouse between attackers and defenders, attackers continually find new ways to evade detection. Whilst high fidelity security detections tend to have high precision, they can sometimes have low recall, therefore some new attack techniques can go undetected. Anomalies on the other hand are much noisier but can capture attacks that would otherwise be missed. Anomalies don’t necessarily indicate malicious behavior on their own. But when combined with other anomalies or alerts their cumulative effect is much stronger.

In this talk, we explore our approach at Microsoft Sentinel to provide the user with customizable anomaly rules. Our engineering methodology uses a PySpark backend to implement a variety of ML techniques including both supervised and unsupervised learning. We deep dive into the ML behind one of our customizable anomalies and then demonstrate the ease at which the rules can be configured by the user. Lastly, we demonstrate, via simulated attacks, how anomalies and alerts can be combined at various stages of the kill chain to produce high quality incidents.

Thus, we can see how customizable anomaly rules improve recall while reducing the noise of traditional anomalies via machine learning and customization.


  • Ed Gardner - Senior Product Manager for Cloud Security, Microsoft
    Ed Gardner is Senior Product Manager for Cloud Security at Microsoft. He has over 10 years of experience with endpoint, network, and cloud security. He helped launch Akamai’s Web Security services; worked on the service that secures all of the payment instruments ingested by; and co-founded, a Managed Detection and Response company. Currently, he works on several machine learning features for Microsoft, including customizable anomalies and build your own machine learning models for Microsoft Sentinel. When not working, Ed likes to spend time with his wife and two sons, reading philosophy, and running. .
  • Karishma Dixit - Senior Security Data Scientist, Microsoft Threat Intelligence Centre (MSTIC)
    Karishma Dixit is a Senior Security Data Scientist in the Microsoft Threat Intelligence Centre (MSTIC) team at Microsoft in the UK. She has 5+ years industry experience in Data Science. She has a first class Mathematics BSc from the University of Warwick and a distinction in her Applied Statistics MSc from the University of Oxford. Outside of work, Karishma is a self-confessed foodie and enjoys playing as “Goal Attack” in the local netball fun league.

Similar Presentations: