How smart is Intelligent Fuzzing - or - How stupid is Dumb Fuzzing?

Presented at DEF CON 15 (2007), Aug. 3, 2007, 6 p.m. (20 minutes).

Dynamic analysis, or fuzzing, is a popular method of finding security vulnerabilities in software. Fuzzing may be used by a developer to find potential problems as part of the quality-assurance process or may be used to find potential exploits in an existing software application. Fuzzing has grown in popularity because it is much easier (and often more effective) to generate and run arbitrary inputs than it is to perform a manual code audit or use software reverse engineering. However, the quality of the fuzzing analysis depends heavily on the quality and quantity of the fuzzed inputs. These inputs, called test cases, are normally constructed in one of two ways: mutation-based or generation-based. In mutation-based fuzzing, known good data are collected and then modified; modifications may be random or heuristic. The advantage of mutation-based fuzzing is that little or no knowledge of the protocol or application under study is required, however it is likely that the collected test cases will only test the most common functionality. Generation-based fuzzing starts from a specification or RFC, which describes the file format or network protocol, and constructs test cases from these documents. Generation-based fuzzing is a much more complete method of fuzzing, but it requires a significant amount of up-front work to study the specification and manually generate test cases. In this talk we analyze the differences between mutation and generation-based fuzzing techniques for the Portable Network Graphics (PNG) format, and quantify the potential advantages gained by using a generation-based approach. Our results show that generation-based fuzzing performs up to 76% better when compared to mutation-based fuzzing techniques for this format.


Presenters:

  • Charlie Miller - Senior Security Analyst, Independent Security Evaluators
    Charlie Miller spent five years as a Global Network Exploitation Analyst for the National Security Agency. During this time, he identified weaknesses and vulnerabilities in computer networks and executed numerous successful computer network exploitations against foreign targets. He sought and discovered vulnerabilities against security critical network code, including web servers and web applications. Since then, he has worked as a Senior Security Architect for a financial firm and currently works as a Senior Security Analyst for Independent Security Evaluators, a security consulting firm. He was a technical editor for the upcoming fuzzing book authored by Sutton, Greene, and Amini. His areas of expertise include identifying vulnerabilities in software, writing exploits, and computer attack methodology. He is a Red Hat Certified Engineer (RHCE), GIAC Certified Forensics Analyst (GCFA), and is a Certified Information Systems Security Professional (CISSP). He has a B.S. from Truman State University and a Ph.D. from the University of Notre Dame.

Links:

Similar Presentations: