Presented at
Black Hat USA 2020 Virtual,
Aug. 5, 2020, 11 a.m.
(40 minutes)
Deep neural networks routinely achieve near human-level performances on a variety of tasks, but each new breakthrough demands massive volumes of quality data, access to expensive GPU clusters, and weeks or even months to train from scratch. AI researchers commonly release model checkpoints to avoid the wasteful duplication of these costly training runs, since fine-tuning pre-trained neural networks for custom tasks requires less data, time, and money compared to training them from scratch. While this emerging model sharing ecosystem beneficially lowers the barrier to entry for non-experts, it also gives a leg up to those seeking to leverage open source models for malicious purposes. <br />
<br />
Using open source pre-trained natural language processing, computer vision, and speech recognition neural networks, we demonstrate the relative ease with which fine tuning in the text, image, and audio domains can be adopted for generative impersonation. We quantify the effort involved in generating credible synthetic media, along with the challenges that time- and resource-limited investigators face in detecting generations produced by fine-tuned models. We wargame out these capabilities in the context of social media-driven information operations, and assess the challenges underlying detection, attribution, and response in scenarios where actors can anonymously generate and distribute credible fake content. Our resulting analysis suggests meaningful paths forward for a future where synthetically generated media increasingly looks, speaks, and writes like us.
Presenters:
-
Lee Foster
- Senior Manager, Information Operations Analysis, FireEye
Lee Foster is Senior Manager of Information Operations Analysis at FireEye Mandiant, having come to FireEye Mandiant via the iSIGHT Partners acquisition. Lee and his team specialize in identifying cyber-driven nation-state influence campaigns and non-state actor hacktivism. He holds Master's degrees in political science and intelligence and international security and is currently pursuing a Master's degree in data science. Prior to FireEye, Lee previously worked in the field of political risk intelligence.
-
Philip Tully / KingPhish3r
- Staff Data Scientist, FireEye
as Philip Tully
Philip Tully is a Data Scientist at FireEye who applies natural language processing and computer vision in order to detect fraud, platform abuse, and security threats. He earned his joint doctorate degree in computer science from the Royal Institute of Technology and the University of Edinburgh. His research concerning the intersection of artificial intelligence and cyber security has been presented at Black Hat, RSA, DEF CON, and a NeurIPS workshop, and it's been covered by The New York Times, BBC, TechCrunch, CNN, KrebsOnSecurity, and more. He's a hackademic that's interested in applying brain-inspired algorithms to both blue and red team operations.
Links:
Similar Presentations: