Presented at
DEF CON 31 (2023),
Aug. 11, 2023, 9 a.m.
(45 minutes)
We’re running the largest live AI hacking event ever in the AI village this year. Anthropic, Google, HuggingFace, Meta, NVIDIA, OpenAI, and Stability, have all provided models to attack and Scale AI have built the platform. This event is orders of magnitude bigger than any previous AI red team effort. There are observers from the White House, NIST, NSF, and the EU coming to learn from hackers. We built this event to grow the community that knows how to effectively evaluate Large Language Models as it is much more than prompt injections and jailbreaks.
AI works fundamentally differently to traditional software and only forms a part of a product. Trust and Security of AI in a system thus has to work fundamentally differently to traditional software. This is especially true for generative AI systems. The core difference is AI is a stochastic component of software and is allowed to make a small amount of mistakes. This changes bug hunting, reporting, and payouts.
Come to this talk to hear about how and why we organized this, and the history of algorithmic & bias bounties that led up to the largest one ever at DEFCON 31. We’ll also give you some tips to help you in the contest.
REFERENCES:
We Need Bug Bounties for Bad Algorithms - Amit Elazari - https://www.vice.com/en/article/8xkyj3/we-need-bug-bounties-for-bad-algorithms
Introducing Twitter’s first algorithmic bias bounty challenge - Rumman Chowdhury & Jutta Williams - https://blog.twitter.com/engineering/en_us/topics/insights/2021/algorithmic-bias-bounty-challenge
Sharing learnings from the first algorithmic bias bounty challenge - Kyra Yee & Irene Font Peradejordi - https://blog.twitter.com/engineering/en_us/topics/insights/2021/learnings-from-the-first-algorithmic-bias-bounty-challenge
Bias Buccaneers - Rumman Chowdhury, Jutta Williams, Subho Majumdar, Scott Steinhardt, Ben Colman - https://www.biasbuccaneers.org/
An Algorithmic Framework for Bias Bounties - Ira Globus-Harris, Michael Kearns, Aaron Roth - https://arxiv.org/abs/2201.10408
Machine Learning Security Evasion Competition - Hyrum Anderson, et al. - https://mlsec.io/ https://cujo.com/announcing-the-winners-of-the-2021-machine-learning-security-evasion-competition/
MITRE ATLAS - Ram Shankar Siva Kumar, et al - https://atlas.mitre.org/
The Spherical Cow of ML Security - Sven Cattell - http://aivillage.org/adversarial%20ml/spherical-cow/
The Case for a Hippocratic Oath for Connected Medical Devices: Viewpoint - Beau Woods, Andrea Coravos, and Joshua David Corman - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6444210/
Announcing OpenAI’s Bug Bounty Program - https://openai.com/blog/bug-bounty-program
Microsoft Malware Classification Challenge - Royi Ronen, Marian Radu, Corina Feuerstein, Elad Yom-Tov, Mansour Ahmadi - https://www.kaggle.com/c/malware-classification https://arxiv.org/abs/1802.10135
Presenters:
-
Rumman Chowdhury
- Co-Founder and CEO at Humane Intelligence
Dr. Rumman Chowdhury’s passion lies at the intersection of artificial intelligence and humanity. She is a pioneer in the field of applied algorithmic ethics, creating cutting-edge socio-technical solutions for ethical, explainable and transparent AI. Dr. Chowdhury currently runs the nonprofit Humane Intelligence and is a Responsible AI Fellow at the Berkman Klein Center for Internet & Society at Harvard University.
Previously, Dr. Chowdhury was the Director of META (ML Ethics, Transparency, and Accountability) team at Twitter, leading a team of applied researchers and engineers to identify and mitigate algorithmic harms on the platform. Prior to Twitter, she was CEO and founder of Parity, an enterprise algorithmic audit platform company. She formerly served as Global Lead for Responsible AI at Accenture Applied Intelligence. In her work as Accenture’s Responsible AI lead, she led the design of the Fairness Tool, a first-in-industry algorithmic tool to identify and mitigate bias in AI systems.
Dr. Chowdhury is dedicated to cultivating and growing the next wave of technology-forward companies enabling the responsible use of emerging technologies.
-
Austin Carson
- Founder & President at SeedAI
Austin Carson is the Founder and President of SeedAI, a nonprofit established to work with a diverse group of policymakers, academics, and private sector experts to help communities across the United States access the resources they need to engage with AI.
Previously, Austin established and led the DC government affairs operation for NVIDIA, translating NVIDIA’s expertise in artificial intelligence and high performance computing for policymakers. Prior to joining NVIDIA, Austin held a number of public sector and NGO positions, including serving as Legislative Director for Chairman Michael McCaul and Executive Director for the Technology Freedom Institute. Austin co-founded the Congressional Tech Staff Association, co-led the Congressional Cybersecurity Caucus and the Congressional High Tech Caucus, and is a founding fellow of the Internet Law and Policy Foundry.
-
Sven Cattell
- Founder at nbhd.ai & AI Village
Sven founded the AI Village in 2018 and has been running it ever since. Sven is also the founder of nbhd.ai, a startup focused on the security and integrity of datasets and the AI they build. He was previously a senior data scientist at Elastic where he built the malware model training pipeline. He has a PhD in Algebraic Topology, and a postdoc in geometric machine learning where he focused on anomaly and novelty detection.
Links:
Similar Presentations: