At this point we’ve all witnessed the seemingly endless parade of headlines regarding yet another cyber attack. The latest news came September 22, 2016 from Yahoo! and includes the staggering revelation that “at least 500 million accounts were stolen” in, wait for it, “late 2014”. Those 500M+ accounts can be added to the already hefty pile of previously compromised credit card numbers, social security numbers, credit reports, health records and federal background investigation dossiers. Throw in recent breaches related to the 2016 U.S. presidential campaign, and every software developer living in the free world has to wonder if democratic elections are being tampered with. This is why we, the technical community, need to bring our talents to bear in a unified front against cybercrime.
Worldwide cybersecurity spend will reach $170B by 2020, VCs poured $3.3B into funding new cybersecurity products in 2015, and with all of this money our personal information continues to be stolen from businesses we trust. If we don’t radically change our thinking and approach to cybersecurity here is what’s going to happen… Creative CISOs are going to find new innovative solutions that will push them one step ahead of the bad guys, but ultimately, like the ocean tide continuously pounding against a cliff, they will be brought down by the black hat community. And this cycle will repeat itself over and over again because of the power of the hacker community. But it doesn’t have to be this way, there is a way to break the cycle.
To defeat the well organized black hats, developers must unite in an open, collaborative community dedicated to fighting cybercrime. Community approaches to solving moon shot problems are not new, and in fact, many valuable innovations have come from these collaborative, ‘open source’ efforts. Open source has allowed the average person to access more information on the web than anyone could have dreamed of 50 years ago. Open source has helped detect child sex traffickers. And open source can stop cyber criminals.
If you’re ready to get involved and help turn the tide against cyber criminals, Intel and Cloudera, along with their partners Centrify, Webroot, StreamSets, eBay, Jask, Cybraics, Cloudwick, Anomali, and Endgame, formally invite you to put on the white hat and join Apache Spot (incubating), the community dedicated to ending cyber threats.
Spot is an open source community committed to accelerating the development of new cybersecurity analytics solutions that harness big data technologies as well as the formidable machine learning and data science skills embodied in the technical community at large. By taking advantage of technologies like Apache Hadoop for massively scalable log management and data storage, and Apache Spark for machine learning and near real time anomaly detection, the Spot community is unlocking new analytic functionality for detecting cyber threats.
Since Spot’s initial launch as the Open Network Insight project earlier this year, momentum continues to build as the community rallies behind the vision for Spot:
Comprehensive Cybersecurity Hub and Open Data Models
Provide organizations a single platform for collecting and managing all of their security data using common, open data models for network, endpoint, and user. These Open Data Models provide a standard format of enriched event data that makes it easier to integrate cross application data to gain complete enterprise visibility and to develop net new analytic functionality.
Facilitate analytic collaboration
The bad guys collaborate all of the time; Spot allows the technology industry do the same. Leading cybersecurity data scientists have been contributing to Spot and developing out of the box machine learning capabilities that allow organizations to quickly discover abnormal and malicious behaviors leveraging Spark. Spot’s Open Data Models ensure that organizations will be able to quickly share new analytics with one another as new threats are discovered. Hadoop allows organizations to run these analytics against comprehensive historic data sets, helping organizations identify past threats that have slipped through the cracks.
Foster a rich application ecosystem
As cybersecurity applications continue to be developed in both the open source and by independent software vendors (ISVs), Spot will accelerate the development of these solutions. Using Spot as a cybersecurity analytics framework, organization and ISVs can spend more time building the analytics and visualizations that help discover cybercrime and less time building systems to ingest, integrate, store, and process any volume or variety of security data.
The developer community has a proven track of record of solving difficult problems and producing amazing solutions. When cyber criminals hack these solutions it’s an affront to all of us. Spot is an opportunity for developers to come together as a community and fight cybercrime.
To learn more about Apache Spot please…
- Visit Apache Spot’s website
- Read the Cloudera Engineering Blog post about Spot’s use cases and open data model approach
- Join our live demo at Strata Hadoop World NYC at Cloudera’s booth (#721) on Wednesday, September 28th at 11:10am EST