The Evolution of the Application of ML to Threat Prediction

The founding team at CYR3CON didn’t suddenly consider the application of machine learning to cybersecurity overnight.  The ideas evolved from a series of efforts that spanned over a decade. 

Application of Machine Learning to Threat Prediction

Much of the work started circa 2007.  At the time, the US was involved in two major counter-insurgency wars in Iraq and Afghanistan.  Understanding these asymmetric threats forced analysts to adapt their methods in a variety of ways. New projects were funded by organizations such as DARPA and other funding agencies such as IARPA were created around this time. 

One such project – called SCARE - dealt with a thorny issue of predicting where insurgents would hide their weapons when planning roadside bombings. It turned out that as the insurgents were constrained by socio-cultural variables (i.e. they could only hide munitions in neighborhoods friendly to them) as well as logistics issues (they needed to keep the munitions close to attack sites) that one could then predict these cache sites. Later the techniques used in SCARE would be adapted to other domains such as stopping poachers in wildlife preserves. 

Terrorist and insurgent organizations were often de-centralized making them difficult to track and understand their leadership structure. One fear was that if a terrorist leader was captured, would the rest of the group become more decentralized – and hence more dangerous?  Using algorithmic techniques to understand what happens to a terrorist organization after a leader is captured became an important way to address this problem. 

The problems posed by network-structured decentralized organizations spread beyond terrorism and played a role in counter-gang operations in law enforcement. The ORCA software was used by the Chicago Police to analyze and counter criminal gang operations based on understanding social relationships as well as gang member usage of social media. 

In the aftermath of the US war in Iraq, a new group emerged in the Middle East known as ISIS that rapidly gained control of large swaths of territory. They relied on a combination of traditional military and terrorist tactics which made predicting their actions difficult, However, using techniques from artificial intelligence made such predictions possible. 

ISIS also became known for their use of social media to influence followers – something called “pathogenic social media”. AI algorithms to detect and counter such social media campaigns became important in the waning days of ISIS. 

ISIS was not the only asymmetric group to leverage social media and the darkweb. Actions of various hacking groups such as Anonymous and LulzSec illustrated how collective action was possible and could lead to cyber attacks. This led to research into hacker online communities using a combination of data mining and machine learning and this initial research showed that indicators of new cyber weapons could be identified. 

These techniques were then extended and applied to the problem of predicting which software vulnerabilities would be exploited. Earlier scientific work relying on factors such as NIST CVSS scores and social media had proven to be ineffective – largely failing once they were applied in more realistic scenarios (see this meta-study by MIT Lincoln Labs). However, with novel methods from machine learning combined with a variety of data sources, accurate predictions became possible.  This research from 2017 became the core of the CYR3CON offering. 

AI and machine learning will continue to evolve and affect our lives and work in new and interesting ways.  As you can see from this post, research done in one domain will often have unexpected consequences in another. AI and machine learning are right now gaining more traction than ever among research and industry – so more innovations are expected.  CYR3CON will continue to innovate in this area. Keep following this blog to learn the latest.