After 15 years of developing a network sensor, log analyzer, and SIEM, based primarily on open-source tools, the future still points to open source. Something is inherently different about open source that makes it more viable for security analysis. Too many analysis processes need to run narrowly and in parallel, or sometimes serial. These require rich interconnections and openness between each specialized tool. The open source community has provided these with thousands of developers working on the projects they are passionate about and fulfilling a function, narrowly, and extremely well. This is what’s lacking in the closed source world where vendors keep out the competition in an attempt to provide a “complete security stack” which has ruined more than a few initially powerful open source tools.
In this talk the presenter recalls his 15-year journey to build and continuously improve his company’s detection platform. His experience with integrating software tools like Bro/Zeek, Snort, and ELK, and with low-level performance tuning of multi-core CPUs and network interfaces provide insight into the powerful advantage of open source. Using the specific example of modifying open-source full packet capture systems to add indexing, Joe demonstrates how just having a decent API is not enough. Open source gives you the total flexibility needed to build a rich cybersecurity SOC platform. Besides, you can’t afford to “test” the non-free stuff.