Presented at
Black Hat Asia 2021 Virtual,
May 7, 2021, 12:30 p.m.
(40 minutes).
In this study, we find that a correlation bias exists in major computer vision systems and exploit it to automatically craft adversarial images.
Objects commonly found together in nature have a strong correlation with each other. This leads computer vision systems to develop a bias for detecting these objects together. For example, almost any round shape next to a dog will be seen as a frisbee by computer vision systems trained on the COCO dataset, since dogs and frisbees appear together very often in both nature and the dataset. The same is true in reverse. Objects that have very weak correlation such as stop signs and pizza will be harder to detect when they appear together.
Using these correlation biases, we generate adversarial images using RetinaNet, YOLOv3 and TinyYOLOv3 trained on the COCO dataset as detectors. First, we determine the target object to hide. Second, we extract objects with a detection certainty above 95% from the COCO dataset and combine them with low correlation (less than 5%) backgrounds. Finally, we re-detect the target object and choose adversarial images in which the target object is now completely hidden.
In addition, we confirm that the crafted adversarial images can be used to attack arbitrary systems where access to the training dataset or knowledge of the network is not available. We evaluate 1,000 adversarial images on leading commercial computer vision systems and receive a 90% success rate on fooling these completely different systems. We also make sure that our crafted adversarial images work in physical environments by pasting the printouts of crafted adversarial backgrounds behind physical objects, taking photos from different angles and confirming that the objects are no longer detected.
Presenters:
-
Yin Minn Pa Pa
- Senior Researcher and Manager, Deloitte Tohmatsu Cyber LLC | Deloitte Japan
<p>Dr. Yin Minn Pa Pa is originally from Myanmar and currently works as a senior researcher and manager at Deloitte Japan. She has been in the cyber security field for more than 10 years. Her research interests include network security, malware analysis, IoT security, web security and AI security. She was the author of research papers presented and published in several well-known cybersecurity conferences and journals. She conducted several International research collaborations with universities in Myanmar, US, Netherland and Germany. She also spends her spare time on the development of cyber security research in Myanmar giving security talks and teaching Myanmar students online. More about her can be found at https://www.yinminnpapa.com.</p>
-
Paul Ziegler
- CEO, Reflare Ltd
Paul Ziegler specializes in assisting large corporate customers with information security research and training. Being completely self-taught, his speaking style tends to be unconventional, but easy to follow. Paul graduated high school in 2007 and gave his first speeches at DefCon and Black Hat immediately after. He then emigrated from Germany to Japan where he began working various information security jobs for corporate customers. He founded Reflare in 2010 to facilitate his work. His work has given him a comparatively shallow but in turn wide understanding of information security. Merging two different - and often seemingly unrelated - fields has led to many speeches across all major conferences.
-
Masaki Kamizono
- CTO and Partner , Deloitte Tohmatsu Cyber LLC | Deloitte Japan
<p>Masaki Kamizono, CTO and Partner of Deloitte Japan, engages in research and new service development, manages large-scale projects and designs socially supportive demonstrational experiments that utilize research results. He contributes to human resource development through the publication of papers, lectures, and government specialists, and through academic committee members and university lecturers.</p><p>Before he joined Deloitte, in 2015, he launched Cyber Security Research Laboratories as a consulting firm, developing new cyber security intelligence services, and responding to incidents and emergencies. In 2009, he joined a security company and consistently engaged in research and development that contributed to cyber security. He also established new core technologies, developed products and solutions, presented papers and gave lectures. At the same time, taking advantage of R&D results, he was in charge of principal researchers and project management in many large-scale national projects. He has participated in several joint research projects with the National Institute of Information and Communications Technology (NICT) since his university time.</p>
Links:
Similar Presentations: