Evidence-based Security Assessment

Presented at ShmooCon I (2005), Feb. 6, 2005, 10 a.m. (60 minutes)

How to decide "Is this thing secure?" is a tough problem. It is a lot tougher than most naive security product consumers think it is. Issues like "what threats are you considering?" and "how much is /insecurity/ costing you?" make it tougher. It is also a lot tougher than most security professionals think it is; Alan Turing's Halting Problem proves that /automatic/ assessment of system security is undecidable, and so the question of "is this thing secure?" will always involve human intervention. Unfortunately, the human approach to assessing security to date has also been sadly lacking. At the formal/government end, we have the Orange Book, TCSEC, and Common Criteria. Having discovered that assessing /actual/ security is hard, these procedures instead produce very expensive piles of documentation of how hard the vendor /tried/ to provide security. A system can be Common Criteria certified with a mountain of documentation, and have a remote root exploit come out the next day. At the informal/hax0r end, we have random penetration testing by "the community", ideally with full disclosure, and sometimes by forensic examination of compromised systems. Here occasional disclosure of a vulnerability definitively shows a product or system to be *insecure*, but we /never/ get any assurance of security, and can only infer security from long silence. We propose a panel on a new approach to assessing security: evidence-based security assessment. It's time to seek security expressed in disprovable hypothesis, and experiments designed to test them. This is the heart of the scientific method, and its time to apply it to security. Is this product or that more secure? Is that "best practice" really better? Can your CISSP-style stand up to the fury of our drunken master style? We will talk about how broad theories are better than narrow ones, and how simple tests are better than complex ones, allowing us to move to more interesting hypotheses and proofs than "This is secure; 0wn.c; patch; goto 10" It's time to compare and contrast. It's time to test. It's time to demand evidence based security. This panel will feature speakers, presenting the world's fastest re-introduction to the scientific method, followed by the underlying hypothesis for other approaches to security, and testing them. We'll also show some examples of how to use evidence based approaches to testing a variety of technologies that are out there today.

Presenters:

  • Ed Reed - Novell Security Czar
    Ed Reed is Novell's Security Tzar, responsible for leading security product strategy and direction across the company. A part of Novell's Office of the Chief Technology Office, Ed works with architects and business planners to fashion Novell's enterprise-oriented identity-based computing efforts to meet customers rapidly evolving needs. During his time at Novell, Ed has led both Product Management and Architecture teams in the areas of Directory and Security products. A graduate of Purdue University (BS), and Rochester Institute of Technology (MSCS), Ed is a frequently requested speaker at industry, technology and analyst briefings and conferences. His standards activities have included work with the IETF (LDAP, LDUP), DMTF, and OASIS.
  • Al Potter - Security Evaluator for ICSA Labs
    Al Potter is an Information Security veteran who has been a senior member of the technical staff at ICSA Labs since 1997. Since 2001, he has been dedicated to the Labs emerging wireless programs as the Manager of Technical Services. In this role, Mr. Potter worked to develop and refine test methodology and infrastructure, and actively participates as a voting member of IEEE 802.11 standards committee. Al's prior roles at the Labs have included hands-on testing of commercial firewall products and management of the Network Security Labs (including the firewall, IPSec, cryptography and IDS testing programs). Mr. Potter was deeply involved in the development of the criteria, processes, tools and procedures for the delivery of TruSecure Corporation's TruSecure product family, and he served as the initial Technical Lead for TruSecure Delivery Services. Prior to joining ICSA Labs, Al spent three years as a senior INFOSEC analyst with Science Applications International Corporation (SAIC), providing Unix system administration services to US Government customers. Before joining SAIC, Al served nine years as an Artillery Officer in the US Army, with extensive experience in tactical automation, nuclear surety and an overseas tour as a Battery Commander and Liaison Officer to the German 12th Panzer Division. He is currently a Major in the retired reserve. Mr. Potter holds a B.S. degree in Mathematics from Davidson College and completed more than three additional years of professional military education.
  • Adam Shostack - Entreprenure & Technologist
    Adam Shostack is a technologist, startup veteran and regular public speaker. He has published papers on the security, privacy, as well as economics, copyright and trust. Adam joined Zero-Knowledge systems in 1999 to build and run the Evil Genius group of advanced technology experts, researching future privacy technologies, including privacy enhancing networks, credentials, and electronic cash. He joined Zero-Knowledge Systems from Netect Inc., a mass-market security software company where he served as Director of Technology. As leader of the core design team for Hackershield, he introduced numerous innovations in security scanning. Shostack sits on the Advisory Board of the Common Vulnerabilities and Exposures initiative, the Technical Advisory Board of Counterpane Internet Security, Inc, and the Privacy Enhancing Technologies Workshop steering committee.
  • Crispin Cowan - PhD, CTO & Co-founder of Immunix
    Dr. Crispin Cowan, CTO and founder of Immunix Inc., is a pioneer in intrusion prevention, beginning in 1998 with the StackGuard compiler defense against buffer overflows. He holds a PhD and professorship in computer science, has published over 35 refereed conference and journal papers, and sits on numerous program committees and editorial boards, including USENIX, ACM, and IEEE. Crispin is a member of The Shmoo Group.

Links:

Similar Presentations: