One of the most talked about technology trends right now is Internet of Things / Internet of Everything. The concept of IoT/IoE is not new but the scale and application is significantly more than we have ever seen before. From connected cars sending back data about their performance, location, health etc, to medical devices monitoring patient vitals and environmental factors, to connected home appliances automating our homes – we are seeing the application of IoT across all verticals in some way or the other. This paradigm is causing a disruption across the board and we are only getting started here. It is also leading to net new business models for different industries. Traditional and new businesses are all exploring ways to leverage this paradigm to augment existing offerings as well as creating new ones. For example, per mile auto insurance based on the driver’s driving patterns, pro-active support for home appliances as a premium offering, predictive maintenance of large machines.
We are seeing new use cases crop up across all verticals. In the healthcare space, wearable devices that track different aspects of your day to day life including things like movement, posture, heart rate etc and analytics to go along with those are fast becoming commodity. Hospitals are looking to measure more metrics and correlate the data with environmental data in order to provide a more informed diagnosis of problems as well as proactively provide recommendations that could prevent people from developing conditions and come back to get more care.
In the automobile sector, there is big disruption going on with vehicles getting connected and being able to record and transmit data back to the home base. Data such as speed, location, braking intensity, and health of the engine are being tracked at the minimum. This can be used to provide value add services such as pro-active support and more detailed diagnostics than was possible before. This can also be used to target content and offers based on driving behavior and location. Connected cars also lend themselves to a new model of insurance that’s based on driving quality and amount of driving. Self driving cars are a further realized version of the connected car where each car has an insight into the surroundings (road conditions, traffic, weather, accidents, etc) along with its own state and immediate surroundings that allow it to drive more efficiently and safely than human beings can.
Home appliances is another area where there’s much activity and is likely the area where most development is happening. From connected switches and appliances that can be operated using smart phones over the internet to intelligent appliances that can order things for you online as you run out of them. Intelligent thermostats and smoke detectors fall in this category as well, along with intelligent, connected locks and security systems. All of these allow for a data driven approach to the home, that starts to automate different parts of our day-to-day life in a more optimized and less error prone manner than today. Vivint is a Cloudera customer that is doing this today.
As you can see, the world of IoT is already a reality in some industries and the prevalence is increasing by the day. As this is happening, different technologies are being adopted to solve the problems that come along with IoT. This is also giving rise to different kinds of architectural patterns since the use cases are not necessarily standardized across the board and each of them have a slightly different requirement than the previous one.
Typical architectural patterns for IoT solutions
As it stands today, there is no ‘typical’ technology stack for IoT solutions. We haven’t standardized on architectures, protocols, stack components etc just yet. This leads to different vendors and developers creating their own specialized stacks for their use cases. As this is developing, we are seeing a few different patterns emerge that we’ll highlight here. An IoT solution consists of the following tiers:
With this, there are primarily three kinds of architectural patterns that are currently being used for IoT solutions:
Device-centric architectures. These entail devices that are intelligent enough to make some level of decisions at their level without having to reach back the central control plane.
Gateway-centric architectures. End devices are connected to control hubs or gateways that drive some part of the application logic for the the IoT solution. In gateway-centric architectures, the gateway system drives the devices without having to reach back to the central control plane.
Centralized control plane-centric architectures. These architectures have a centralized control plane at their core. Data is stored, processed and decisions are made at this level and actions are sent out to the end devices or gateways.
In most cases, IoT solutions consist of a combination of the above architectural patterns. The central control plane is where the long term data is stored and processed and that’s where an enterprise grade, scalable data platform is required.
Data platform for IoT
One common thing that is emerging is the need for a new kind of data platform for IoT solutions. Per Gartner’s report on DBMS Characteristics for the Internet of Things, much of the data being generated by IoT devices is time-series, event based data and needs a database that can cope with the variety, velocity and volume that IoT data entails. Along with that, it needs to have enterprise grade features to enable secure deployments in different kinds of environments that are compliant with regulations pertaining to the particular industry. These line up very closely with Cloudera Enterprise, the fastest, easiest and most secure Hadoop distribution, and we have been working hard to improve the platform’s capabilities to satisfy such requirements. Some of the key abilities in Cloudera’s platform that make it uniquely suited for IoT solutions are:
Scalability – Cloudera Enterprise is based on the scalable Apache Hadoop platform and includes a scalable storage substrate in HDFS, along with scale out ingestion and processing frameworks that make the overall platform easy to scale so it can handle the velocity and volume of data that comes along with IoT use cases.
Flexibility – Cloudera Enterprise offers users flexibility at different levels. It offers flexibility in terms of the infrastructure that it can be deployed on – from bare metal commodity servers to private and public cloud environments. It also offers flexibility with the kinds of access patterns that it can support, which range from batch processing, iterative in-memory processing, low latency analytics, stream processing, search and low latency serving. This uniquely solves the data problems that come along with IoT use cases since they require a variety of different access patterns to be handled equally well.
Security and Governance – With IoT use cases, more data, more users, and more tools create complexity. At the same time, in today’s environment of increased regulation and concern about individual privacy, security and data governance have never been more important. IT needs to balance these concerns with the desire to support business agility and self-service. In traditional architectures, it can be complex to secure all data for all users, so it’s often easier to simply leave data out of analysis, or lock users out of analytics. Cloudera’s enterprise data hub offering provides a rich set of capabilities to address these problems – strong authentication, access control, and auditability are the three core pillars of this.
Manageability – Cloudera Enterprise comes with sophisticated enterprise-grade management software that enables operators to reliably operate their clusters and get support from Cloudera as and when required.
Open Standards – Cloudera Enterprise is built on an open source core and all the projects in the core CDH stack have multiple commercial vendors participating and supporting them.
Rich partner ecosystem – A very important aspect of an IoT solution today is the need for multiple products to interplay in order to make for a complete solution. Cloudera has a rich partner ecosystem that includes Global System Integrators, Independent Software Vendors, Hardware and Cloud vendors. With this, customers can pick and choose their technology stack and be rest assured that it’ll work as a single solution backed by their individual vendors.
With Internet of Things becoming a reality in a variety of use cases across verticals, the need for an appropriate data platform is growing as well. We at Cloudera have been and continue to stay committed to building the best platform for the Internet of Things, working with our partners to bring to you a complete solution. Some of our current customers that are using Cloudera Enterprise as their IoT data platform are Monsanto, Camstar, Opower.