By Larry Lunetta, VP Business Development, Niara
Enterprise product/market dislocations generally require the confluence of several factors. We are in the middle of a dislocation in the security space that is still so new there isn’t even an agreed-upon name for it. The closest we’ve seen is “Big Data Security Analytics (BDSA)”. It’s certainly new and like almost all security products, the name is heavy on the new technology and light on the customer problem.
In fact, BDSA embodies two technologies that are new to the security space: big data (e.g. Hadoop) and “advanced analytics”—in fact some of the same techniques are used with machine learning to suggest a new book or to serve pinpointed ads in real time. But, it’s the current state of the security threat environment that makes these technologies so vital.
Enterprises can no longer rely exclusively on their perimeter, real-time defenses (such as firewalls) to protect them from every attack. It is clear that newer classes of malware and other targeted “advanced threats” are specifically crafted to elude those products. So, the security team now has a much bigger attack surface to defend (think mobile, cloud, etc.) and an almost 100% guarantee that some number of slowly gestating but lethal attacks will get inside their networks and create havoc. The headlines tell the story.
Those threats that get to the inside do not have the characteristics or behavior of conventional attacks. If a firewall or sandbox could find and deal with them, they would. They deal in the world of black and white. It’s good or bad. Threats on the inside reside in the world of grey. By definition, they have gotten through the “black and white” filter and can only be found by finding and correctly interpreting small, faint signals as they progress though their kill chain. A file that looks suspicious, a domain name that looks machine generated, an abnormal system access. Individually, these “weak” signals cannot be raised to full-scale alerts because it is only when they are chained together and put in context (“these are all happening to the same user”) that action can be taken.
So, we have attacks designed to hide in the white noise intended to do massive damage to enterprise operations and reputation. That’s the market side of the dislocation.
Returning to the technology side, the “advanced threat” problem is not and cannot be solved with existing security products and technologies. The only way to effectively find and remediate “grey zone” threats is to collect and analyze all the security-relevant information that comes from the IT ecosystem. Logs, flows, packets, files, alerts, emails, HR actions, etc. are all “feedstock” for the other part of the technology innovation: sophisticated models and algorithms deployed more comprehensively and consistently than have previously been deployed.
This includes a number of statistical and predictive mathematical models to establish profiles of “normal” behavior for users, systems, etc. while looking for deviations from those baselines that might be indicative of an incipient attack. The power in this approach is that the models are specifically designed to operate on “gray” data, to interpret behavior and determine when anomalies turn malicious and provide the forensic support security analysts require to confidently take action.
Why now? The security challenges are clear and the stakes are sky high. Why here? BDSA requires the volume and compute scale of a platform like Cloudera’s Enterprise Data Hub for Cyber Security to rebalance the threat landscape in favor of the security team.