Twitelligence, an open-sourced threat intelligence platform with customizable twitter feeds powered by big data and machine learning

Presented at SOURCE Seattle 2017, Oct. 4, 2017, 3:30 p.m. (40 minutes)

Social media as sources of information are playing a significant role delivering real-time information to users in a way that conventional media can hardly compete. In the area of security research, many active researchers share security news, and exchange time-sensitive security information through various social media, especially twitter. The live exchange of security information allows researchers and authorities to be aware of newly emerging threats and respond to security attacks in a timely manner. There are more than 2000 security researchers and authorities active daily on twitter, tweeting several thousand tweets per day. With such a huge amount of info from thousands of sources, researchers may need to spend hours per day mining valuable security information matching their interest.

In this work, we are releasing an open source platform to perform automatic security information mining, grouping and ranking. The platform is highly customizable based on personal preference on different areas of security researches, supported by big data streaming processing and unsupervised machine learning models. We aim to provide relational mapping of selected security researchers and authorities with valid credentials. With them being reliable resources plus trained machine learning model, our tool can collect, analyze, categorize and rank the most significant events in real-time, so as to bring convenience to security researchers and improve the efficiency for threat information collection. Our platform can be easily extended to cover other social media sources and evolve to a comprehensive social-media-based threat intelligence platform.


  • Xiaoning Li - Chief Security Architect
    Xiaoning Li is a securty expert and now is chief security architect in one of the Fortune 500. He has presented on many security conferences including BlackHat, BlackHat Asia, USENIX, source conference, etc
  • Yanhui Zhao - Sr. Software Engineer
    Yanhui is a senior software engineer in one of the Fortune 500. Yanhui works on mobile security and big data. Yanhui won Microsoft bug bounty and GeekPwn hacking award at 2016. Yanhui also presented on Source conference 2016.
  • Li Shen - Senior Engineer
    Li is a senior engineer and security researcher in Fortune 500, and has been focusing on applying machine learning models to solve security challenges. He worked on projects such as cracking google recapture and homomorphic encryption using intel SGX. He is a speaker at USENIX 2017.

Similar Presentations: