Meaningful Measurement: It's About Time We Got This Right

Presented at 44CON 2016, Unknown date/time (Unknown duration)

That cyber-crime has driven the rise of malware during the last decade is not in doubt; how large that increase has been most certainly is. This measurement has, I would argue, been more speculative than evidential. The problem being that attempts to quantify malware usage are lacking any meaningful industry accepted standard when it comes to the metrics concerned.

When the numbers put forward by vendors, industry bodies and the media all vary so widely (not just between those sectors but within them as well), is it any wonder that any serious attempt to establish the scale, the cost or the impact of such attacks is doomed to failure? The disconnect between the reporting of cyber-crime and the actual metrics that are most important for both businesses under attack and the industry that exists to mitigate them will remain until the difficulties of comparing oranges with apples become apparent.

Attempting any such comparative exercise is fraught with peril and serves to highlight where we, as an industry, are getting our metrics wrong; the largely accepted cost per record breach metric is far too broad a brush to paint any kind of recognizable real world picture. When reporting and discussing the scale and impact of cyber-crime it is imperative that we move away from sensationalizing of one part of the story or consequence of the breach, that which will create the biggest search engine feeding frenzy. Who the criminals were is of less import than how they got in; compromise indicators are more valuable to other businesses than the financial cost to that particular victim.

The measurement metric dial has, ultimately, moved too far towards attribution and needs to be reset to prevention and a business-based analysis of risk once more. That business-based analysis itself needs to be more realistic, so there also has to be a move away from the kind of threat intelligence reporting which is almost exclusively dominated by data derived from the large enterprise sector and consequently of little relevance to the Small and Medium Enterprise (SME) market.

The data upon which threat intelligence and attack surface trend analysis resources are based must become more granular if it is to become more relevant across all business sectors. If we continue to go down the road of never disclosing or identifying the security components that failed or the components that were not in place when a breach happened, we will never make any progress against an elusive enemy.


Presenters:

Similar Presentations: