This is a report about the recent workshop held by the European Commission titled “Big Data Skills for Europe” in Brussels.
I was recently invited to participate in a work workshop organised by the DG Connect (DG Communications Networks, Content, Technology) group, which is part of the European Commission (EC) located in Brussels, and the Representation of the State of Baden-Württemberg to the European Union. The goal was to assemble a wide variety of stakeholders and experts, i.e. people with industry insight across different verticals and at varying positions, as well as EC members that have an interest in the topic.
For context here a quote from the invitation summarising the need to address the topic:
Big Data analytics are becoming increasingly important for sound decision-making within firms and other organisations and are expected to contribute significantly to innovation and economic growth in the near future. In particular, the value of world Big Data technology and services is expected to grow from $6 billion in 2011 to $23.8 billion in 2016. At the company level ‘ data-directed decision-making’ could lead to a 5-6% increase in productivity and within the public sector savings are expected of more than EUR 100 billion for Europe.
Exploiting the potential of Big Data will create increasing demand for Big Data related skills. Predictions for the UK point to an over 200% increase in labour market demand for Big Data skills (within large organisations) between 2012 and 2017 to 69,000. Evidence already shows an emerging shortage of analytical and managerial skills necessary to make the most of Big Data.
The day was divided into two parts but with the same structure. First a group of selected experts were to make a case for the morning’s or afternoon’s topic by presenting their thoughts and ideas within 5-8 minutes. Afterwards the experts divided across different tables with each table addressing a specific topic question. The experts (plus further attendees to make up the needed count) would host the table, introduce the question and discuss with the table visitors, made up from the remaining attendees. After 30mins the table visitors were shuffled once more and another round of discussion was held. At the end the table hosts were each reporting on their findings on stage.
The two topics for each half day were
- identify the Big Data skills mix needed to achieve the potential gains (see above), and
- identify solutions to achieving the Big Data skills mix required.
As for the goal of the day, I quote once more the invitation text:
The objective of the workshop is to come up with a set of policy recommendations on Big Data Skills for Europe.
The expert attendees were from Global System Integrators (GSIs, e.g. Capgemini, Accenture), vendors (e.g. Cloudera, Oracle, TeraData, IBM), academic institutions (e.g. UC London, Karlsruhe KIT, Open University London, Barcelona GSE), research institutions (e.g Fraunhofer), and many others. Interesting was that a few of the selected experts already work together partners or competition, or presented together in other Big Data related events. As said it before: The Big Data World is indeed a small one. This on the other hand drives home the point that only few people in the world (in comparison) are working on this topic, and hence the need to build up a broader community with more skilled people being of interest and essential to everyone.
The day was kicked off by keynote speaker introducing the topics and setting the stage for the day. Then the first round started and we heard the chosen experts in their panel presenting their take on how to identifying the needed skill mix for Big Data. The round table iterations afterwards saw many engaging discussions. I joined table 3 with the question:
Which are the Big Data jobs that are and will be in highest demand in the future? And which skills does each of them require?
We came to the conclusion (for the people on the table) that we already have a good education and academic system in place covering all levels of professionals involved in data processing. It was nice to see that I had the same already addressed in my slide deck (see below) for the afternoon. For example this slide:
We did talk about the (kind of) new role of the data scientist, the cross discipline magic unicorn. Again though the data scientist is a combination of existing disciplines, just in the right mix. What is lacking really is to train all vertical roles to be prepared for Big Data, speaking the same lingo. We get to more of that below.
The next table was #6, with the question:
What are the foundation skills for Big Data within compulsory education and are they currently being provided?
We discussed this again very actively with many arguments leading to the following list of take-aways:
- Soft skills matter more than hard skills
- Big Data has to be introduced early to raise interest early
- Changes needed are not too intrusive, just needs adding of overarching story
- Current eduction does not teach much or anything about Big Data
With that we concluded the the round tables and all table hosts reported on the findings. I’ll defer to the final report once available for the specific details.
After lunch we started the second round trying to list solutions to achieve the necessary Big Data mix. I was part of the panel and presented my deck, named “Big Data is not Rocket Science”.
The main theme resonated with the earlier discussions. I was making a point that we need vertical training of everyone involved in Big Data and in addition need to develop easier to access platforms, leading to DaaS, or Data-as-a-Service. With that we can further the adoption of Big Data by all working towards a common goal with support from the platform. My own action items I listed were:
- Combine existing educational material to reflect new challenges
- Train staff to understand challenges concerning their responsibilities
- Develop new middleware that makes adoption of platform easier
After the other panel members presented their messages, we started the round table again, this time I was a host though on table 6 and staid there the entire time. The question we discussed was:
Should Big Data have its own undergraduate/postgraduate courses or should specialisation in Big Data be offered within existing relevant academic tracks (e.g. computer science, statistics, business, etc.)?
We listed the following findings:
- If foundation is right, you can add the Big Data flavour
- It should be identifiable what the skills are
- Big Data is different for every role in an organization
- But each should be taught the basics of what Big Data means
- Might be a single or full course, or even specialisation
- Can’t be a single course only, but must spanning disciplines
- Defining Big Data is difficult, might take too long to try and might be inaccurate
- Each school is free to define what they want
For a last time the hosts presented their findings and after a break the DG Connect team presented the compiled findings.
We contributed a bit more feedback, for example I asked if we could organise the solutions by time, for example short term solutions to close the immediate skill shortage gap, as well as mid-term and long term ones. The latter are adding the required courses to the academic education. For the short term solutions we have to train current IT, business, and management staff to start thinking in Big Data. This could be through schools, but also private training facilities.
The DG Connect team will compile the final results and publish it in due course. I found the day very interesting and am glad I was invited. The conversations with the other attendees were very enlightening and did show how much we are all agreeing about the current state in the industry. We need more skilled professional, but not just the fabled data scientist unicorns. It requires a training on every level and we all have to contribute one way or another.