Turing in a Box: Applying Artificial Intelligence as a Service to Targeted Phishing and Defending Against AI Generated Attacks

Presented at Black Hat USA 2021, Aug. 5, 2021, 10:20 a.m. (40 minutes)

With recent advances in next-generation language models such as OpenAI's GPT-3, AI generated text has reached a level of sophistication that matches or even exceeds human generated output. The proliferation of Artificial Intelligence as a Service (AIaaS) products places these capabilities in the hands of a global market, bypassing the need to independently train models or rely on open-source pre-trained models. By greatly reducing the barriers to entry, AIaaS gives consumers access to state-of-the-art AI capabilities at a fraction of the cost through user-friendly APIs.

In our research, we present a novel approach that uses AIaaS to improve the delivery of Red Team operations - in particular, the conduct of phishing campaigns. We developed a targeted phishing pipeline that uses OpenAI and Personality Analysis AIaaS products to generate persuasive phishing emails. Our pipeline automatically personalizes the content based on the target's background and personality. We observed that AI generated phishing content outperformed those that were manually created by Red Team operators. Furthermore, the pipeline freed up Red Team resources to focus on higher-value work such as context building and intelligence gathering.

In addition, we present an AIaaS-powered phishing defense framework to detect such attacks. Compared to traditional classification-based email filters, our framework adapts deep learning language models such as OpenAI's GPT-3 to accurately distinguish between AI and human generated text. This allows security teams to mount a credible defense against advanced AI text generators without requiring significant AI expertise or resources.

Our research provides actionable takeaways for both red and blue teams to prepare for the current reality of advanced AI proliferation. We discuss the long-term implications of this trend and recommend high-level strategies such as AI governance frameworks to safeguard against the abuse of AIaaS products.


Presenters:

  • Timothy Lee - Associate Cybersecurity Specialist, Government Technology Agency Singapore
    Timothy Lee is an Associate Cyber Security Specialist focusing on iOS security. As a certified OSWE, OSCP, and CRT, he has performed several security assessments, ranging from mobile and web application penetration tests to source code reviews. His main research area is on iOS kernel security.
  • Tan Kee Hock - Cybersecurity Specialist, Government Technology Agency Singapore
    Tan Kee Hock is a Cybersecurity Specialist who simply likes to 'hack' things. He loves to play CTFs and is always keen to explore more!
  • Eugene Lim - Associate Cybersecurity Specialist, Government Technology Agency Singapore
    Eugene Lim is an Associate Cybersecurity Specialist at GovTech Singapore. Also known by his white hat handle @spaceraccoon, Eugene enjoys researching application security and devsecops. His work has been featured in newsletters such as Security Week and The Daily Swig.
  • Glenice Tan - Associate Cybersecurity Specialist, Government Technology Agency Singapore
    Glenice Tan is an associate cybersecurity specialist at GovTech Singapore who enjoys exploring the quirks of different systems, applications and processes. Currently, she is focusing on web security, cloud technology, and social engineering practices.

Links:

Similar Presentations: