Simplifying Big Data in the Cloud

Categories: Cloud Data Engineering Product Spark

In recent years, as public cloud adoption has accelerated and customers have started looking towards cloud for large-scale data workloads, we sought to reimagine how to most effectively offer Cloudera capabilities in the cloud.

Our customers wanted to understand how to leverage the agility, scale, and ease-of-use offered by the cloud to efficiently and cost-effectively gain insights from their ever-growing business data. In addition, customers wanted to do so without sacrificing the enterprise capabilities and reliability of the Cloudera platform.

With cloud a company-wide priority, we worked to achieve a balance of enterprise capabilities and ease-of-use, while maintaining interoperability across the Cloudera stack. We made our platform more cloud-friendly with features such as:

  • Optimized read and write paths to hyperscale cloud storage
  • Faster cluster provisioning times
  • Integrated security with cloud service provider solutions
  • Ability to elastically scale clusters
  • Resiliency for preemptible instances  

And we continued to honor our open source roots by contributing the new cloud-native capabilities of our core components back upstream. As we progressed, we exposed these new cloud-native features in the platform. But we ultimately had a more radical goal in mind.

We wanted to take a giant leap forward in providing a cloud experience that focuses on the end user by hiding the complexities of cloud and cluster operations. To this end, we worked in parallel to create a platform-as-service offering backed by the enterprise and cloud-native capabilities of Cloudera.

Today, we’re excited to introduce Cloudera Altus — our initial platform-as-a-service offering that focuses on data engineering workloads that run in the cloud. Data engineers continually ingest data from myriad data sources from IoT devices to clickstreams to user records to log files and more — and subsequently process this data for further analysis, visualization, and reporting. Enterprises can run up to tens of thousands of data engineering workloads a day. And they must do so reliably, cost-effectively, and securely while meeting tight service level agreements.

Altus for data engineers does the following to address the increasingly complex needs of the data engineer:

  • Allows the user to focus on his or her workloads while Altus manages the clusters
  • Supports full data pipeline workflows from Altus deployed data engineering clusters to Cloudera Director-deployed Cloudera clusters for BI analytics and data science
  • Provides an easy workload migration experience from on-prem to cloud with minimal risk

Cloudera Altus features include:

  • Quick and simple on-demand cluster provisioning
  • Job-oriented workflows that allow users to focus on their workloads
  • Easy job troubleshooting workflows
  • Intuitive UI and command line interface

More details on Cloudera Altus can be found on our Engineering blog.

This is only the beginning. We have an exciting and aggressive cloud roadmap ahead. We invite you to try Cloudera Altus, and come with us on our big data journey to the cloud.


Leave a Reply