Strata+Hadoop World, NYC is behind us but the excitement and memories linger on. The event far surpassed my expectations in terms of quality of sessions and speakers. In fact as many will attest, the majority of sessions were filled to capacity.
Two standout sessions for me were:
Big data governance – a panel discussion moderated by our very own Steve Totman with speakers from Capital One, CapTech Consulting and MasterCard Advisors. Speakers discussed everything from what makes big data challenging and the importance of data governance to best practices for preventing data swamps. Panelists voiced the importance of understanding how to catalog the data, the need for having the right controls in place to verify data quality, integrity and source while ensuring ease of access by end users.
Speakers also discussed which teams or individuals in the organization should own data governance and who should ultimately be held accountable. On this point, there was unanimous consensus that data governance is a joint responsibility that should be held by both business and IT users. There is no single custodian of data especially in large matrixed organizations. While data stewardship may vary from corporation to corporation, by and large the operations, IT, risk management and compliance teams as well as the Chief Financial Officer are often involved. Panelists also stated that regulatory compliance was a catalyst in driving rigorous data governance standards in the financial services industry. This has created both a set of challenges and opportunities for financial institutions to exploit.
Hadoop and self-service analytics – an outstanding presentation given by Allstate on their use of EDH and Tableau to build a centralized big data analytics hub. The speaker shared insights on the business drivers behind this strategy. For instance, prior to building a centralized big data analytics hub, much of the data resided in multiple systems. With millions of customers, getting a complete view of each policy was a cumbersome process. Moreover, it was difficult to get a real-time view of all the policies for analysis. The carrier is leveraging Impala, a key component of EDH and Tableau to gain granular insights across key areas of its business. With a centralized big data analytics hub, data scientists, various business and IT users can perform more analysis with more types of data faster.