Presented at
Black Hat Asia 2019,
March 28, 2019, 5 p.m.
(30 minutes).
Deep Neural Networks (DNNs) have been found vulnerable to adversarial examples – inputs that an attacker has intentionally designed to cause the model to make mistakes. Fortunately, generating adversarial examples usually requires white-box access to the victim model, and adversarial attacks against black-box models are still imaginary without running unlimited brute-force search. Thus, keeping models in the cloud can usually give a (false) sense of security. Our goal of this talk is to shed light on a new hidden attack vector of DNNs, which allows adversarial examples to be efficiently generated against black-box models used in mission-critical tasks such as facial recognition, image classification, and autonomous driving.
We report an intriguing vulnerability that allows an attacker to effectively attack black-box object detection DNNs using adversarial examples generated from white-box open source models. This vulnerability comes from a prevailing strategy used in deep learning areas to alleviate the thirst for data, called transfer learning, where highly tuned and complex models that have been well-trained on huge datasets are used as pre-trained layers for other similar applications. It is also a recommended practice by major deep learning service providers, including Google Cloud ML and Microsoft Cognitive Toolkit. However, despite its appeal as a solution to the data scarcity problem, we show that the model similarity introduced by transfer learning also creates a more attractive and vulnerable target for attackers.
In the talk, we will first present the alarming results from our measurement study that most main-stream object detection models are adopting those winning image classification models in the ImageNet contest as their first few layers, to extract low-level features in the image. Then we will discuss the attack algorithms, as well as the techniques to identify which pre-trained feature extractor is used by target object detection model with limited queries. We will demo how the adversarial examples generated using our algorithms from YOLOV3, is able to attack other object detection DNNs, that are usually considered using totally different techniques. Finally, we wrap up the presentation with a demo on attacking models from commercial machine-learning-as-a-service provider to make audience aware that keeping models proprietary isn't a guarantee for security against adversarial examples.
Presenters:
-
Yunhan Jia
- Senior Security Scientist, Baidu X-Lab
Yunhan Jia is a senior security scientist at Baidu X-Lab. He obtained his PhD from University of Michigan with a research focus on smartphones, IoT, and autonomous vehicle security. His past research revealed the open port vulnerabilities in apps that exposed millions of Android devices to remote exploits. He is currently working on the memory safety and deep learning model security issues of autonomous vehicle platform.
-
Zhenyu Zhong
- Staff Security Scientist, Baidu X-Lab
Zhenyu Zhong's current research focuses on adversarial machine learning, particularly deep learning. He explores physical attack tactics against autonomous perception models, as well as defensive approaches to harden the deep learning model. Previously, Dr. Zhong worked for Microsoft and McAfee, mainly applying large scale machine learning solutions to security problems such as malware classification, intrusion detection, malicious URL detection, spam filtering, etc..
-
Yulong Zhang
- Principal Research Scientist, Baidu X-Lab
Yulong Zhang is a security researcher of Baidu USA and Baidu X-Lab. He is currently leading the research of mobile security and automobile security.
-
Qian Feng
- Research Scientist, Baidu USA LLC
<span>Qian Feng is a senior research scientist in Baidu X-Lab USA. She received her PhD degree in computer security from Syracuse University. Her research focuses on program analysis, memory forensics, machine learning, and reverse engineering. Her works have been published on mainstream security conferences such as CCS, ACSAC and ASIACCS. She has received the best paper candidates at ACM Asia Conference on Computer and Communications Security in 2016, and the student travel grant for the Annual Computer Security Applications Conference in 2014.</span>
-
Tao Wei
- Chief Security Scientist, Baidu X-Lab
Dr. Wei Tao is the Chief Security Scientist at Baidu Inc. and an Adjunct Professor at Peking University.
-
Yantao Lu
- PhD Student, Syracuse University
<span style="font-size: 10pt;">Yantao Lu received a B.S. degree in electrical engineering from Xian Jiaotong University, Shaanxi, China, in 2013, and the M.S. degree in electrical engineering from Syracuse University, Syracuse, NY, USA, in 2015, where he is currently pursuing the Ph.D. degree with the Department of Electrical Engineering and Computer Science. His research interests include activity recognition from wearable cameras, computer vision based adversarial examples, and smart and mobile camera systems. He is now an intern at Baidu X-Lab.</span>
Links:
Similar Presentations: