Hacking Humans with AI as a Service

Presented at DEF CON 29 (2021), Aug. 6, 2021, 2 p.m. (45 minutes).

As the proliferation of Artificial Intelligence as a Service (AIaaS) products such as OpenAI's GPT-3 API places advanced synthetic media generation capabilities in the hands of a global audience at a fraction of the cost, what does the future hold for AI-assisted social engineering attacks? In our talk, we will present the nuts and bolts of an AIaaS phishing pipeline that was successfully deployed in multiple authorized phishing campaigns. Using both paid and free services, we emulated the techniques that even low-skilled, limited resource actors could adopt to execute effective AI-assisted phishing campaigns at scale. By repurposing easily-accessible personality analysis AIaaS products, we generated persuasive phishing emails that were automatically personalized based on a target's public social media information and created by state-of-the-art natural language generators. We will also discuss how an AI-assisted phishing workflow would impact traditional social engineering teams and operations. Finally, we look at how AIaaS suppliers can mitigate the misuse of their products. REFERENCES 1. T. Karras, S. Laine, and T. Aila, "A Style-Based Generator Architecture for Generative Adversarial Networks," arXiv:1812.04948 [cs.NE], 2019. 2. S. Gehrmann, H. Strobelt, and A. M. Rush, "GLTR: Statistical Detection and Visualization of Generated Text," arXiv:1906.04043 [cs.CL], 2019. 3. G. Jawahar, M. Abdul-Mageed, and L. V. S. Lakshmanan, "Automatic Detection of Machine Generated Text: A Critical Survey," arXiv:2011.01314 [cs.CL], 2020. 4. J. Seymour and P. Tully, "Weaponizing Data Science for Social Engineering: Automated E2E Spear Phishing on Twitter," 2016. 5. P. Tully and F. Lee, "Repurposing Neural Networks to Generate Synthetic Media for Information Operations," 2020. 6. OpenAI, "OpenAI Charter," OpenAI, 09-Apr-2018. [Online]. Available: https://openai.com/charter/. 7. G. Brockman, M. Murati, and P. Welinder, "OpenAI API," OpenAI, 11-Jun-2020. [Online]. Available: https://openai.com/blog/openai-api/. 8. A. Pilipiszyn, "GPT-3 Powers the Next Generation of Apps," OpenAI, 25-Mar-2021. [Online]. Available: https://openai.com/blog/gpt-3-apps/. Would like to thank contributing author Timothy Lee Timothy is a security researcher who likes to break things and tries to understand how the system works during the process. In the past year, he is researching with iOS security and is starting his journey on iOS vulnerability research. Additionally, he has contributed to red team social engineering operations and security tooling, with practical experience in vishing and in-person social engineering. https://www.linkedin.com/in/timothylee0/

Presenters:

• Eugene Lim - Cybersecurity Specialist, Government Technology Agency of Singapore
  Eugene Lim, also known as spaceraccoon, is a security researcher and white hat hacker. He regularly participates in live-hacking events and was awarded the Most Valuable Hacker title in the h1-213 Live-Hacking Event by Hackerone. Besides white hat hacking, he enjoys building security tools, including a malicious npm package scanner and an open-source intelligence social engineering honeypot that were presented at Black Hat Asia Arsenal 2019 and Black Hat USA Arsenal 2020. His writeups on https://spaceraccoon.dev are regularly cited by other white hat hackers. @spaceraccoonsec https://www.linkedin.com/in/limzhiweieugene/
• Glenice Tan - Cybersecurity Specialist, Government Technology Agency of Singapore
  Glenice is a security researcher that enjoys exploring the quirks of different systems, applications, and processes. In the past year, she had the opportunity to conduct social engineering exercises, which includes phishing and vishing. Apart from applications and human hacking, she also experiments on ways to automate or improve red team operations. https://www.linkedin.com/in/glenicetan/
• Tan Kee Hock - Cybersecurity Specialist, Government Technology Agency of Singapore
  Tan Kee Hock is a Cybersecurity Specialist who simply likes to 'hack' things. He loves to play CTFs and is always keen to explore more! https://www.linkedin.com/in/tankeehock/

Links:

• https://defcon.org/html/defcon-29/dc-29-speakers.html#lim
• https://infocon.org/cons/DEF CON/DEF CON 29/DEF CON 29 presentations/Eugene Lim Glenice Tan Tan Kee Hock - Hacking Humans with AI as a Service.pptx
• https://infocon.org/cons/DEF CON/DEF CON 29/DEF CON 29 video and slides/DEF CON 29 - Eugene Lim, Glenice Tan, Tan Kee Hock, Timothy Lee - Hacking Humans with AI as a Service.mp4
• https://infocon.org/cons/DEF CON/DEF CON 29/DEF CON 29 presentations/Eugene Lim Glenice Tan Tan Kee Hock - Hacking Humans with AI as a Service.pdf (slides)
• https://infocon.org/cons/DEF CON/DEF CON 29/DEF CON 29 video and slides/DEF CON 29 - Eugene Lim, Glenice Tan, Tan Kee Hock, Timothy Lee - Hacking Humans with AI as a Service.srt (subtitles)
• https://www.youtube.com/watch?v=tWWhRbzhkrg

Similar Presentations:

• Black Hat USA 2021 / Turing in a Box: Applying Artificial Intelligence as a Service to Targeted Phishing and Defending Against AI Generated Attacks
• DEF CON 31 (2023) / Growing the Community of AI Hackers with the Generative Red Team
• DEF CON 32 (2024) / Social Engineering Like you’re Picard