9 Challenges with Data Science & AI

The world renowned scientist - Professor Stephen Hawking - famously declared that "Success in creating AI would be the biggest event in human history.". Yet, he went on to proclaim that, "Artificial intelligence has the power to eradicate poverty and disease or hasten the end of human civilisation as we know it".

So, what do these challenges mean for you and how might the knowledge presented in this article help you prepare for an increasingly uncertain future?

#1 The Talent War - Finding Talent

In a recent interview, I discussed the key determinant for staying competitive in today's data world - talent. Why?

I discussed how - in the recent past - competitive advantage was typically gained via capital required to acquire technology. Technology was king and technology was expensive - presenting significantly high barriers to entry for start-ups or small businesses. This was given the extensive upfront investment required to acquire sufficient resources - or capacity - to enable viability and to scale. Perhaps to a lesser degree, this situation presented sustainability challenges for some established organisations too - as time to value appropriation was seemingly interminable and with any poor decisions potentially financially catastrophic.

However, in today's world, such barriers to entry are remarkably diminished by arguably one of the most outstanding technological innovations of our time - the Cloud. With the Cloud, access to incredulously abundant managed resources are not only easier than ever to obtain but are available on a "pay as you go" and "only pay for what you use" basis - all but completely wiping out antecedent technological advantages of large and established organisations.

This new order has driven a notable adjustment in the balance of power - giving rise to garage tech companies growing into world-beating giants at speeds previously unthinkable and leading to the demise of those established icons that have struggled to re-invent. It has also moved the battle lines - for competitive advantage - from technology and capital to people and talent.

This war has not only led to unprecedented wage inflation - across data and artificial intelligence related professions - but has extended to countless intangibles. In what is increasingly a candidate led market, organisations are now compelled to make innumerable adjustments - to working conditions, culture and even technology choices - in order to tempt the best talents in and convince them to stay.

So, if talent were the silver bullet, can it be argued that - since money can effectively buy talent - money is, in fact, the silver bullet? The answer, arguably, is - no. Whilst money (and financial investment more broadly) undoubtedly has an important part to play, it has - since the shift away from the historically successful 20th century scientific management model - ceased to be the sole ingredient required for acquiring or retaining talent.

#2 The Talent War - Finding A Job

With the Harvard Business Review having declared data science "The Sexiest Job of the 21st Century", it is no wonder why the talent market for data jobs seems a ferocious battlefield - not only for candidates looking to get into data professions but for organisations trying to attract and retain the best of the best.

So, when I caught up with my People Resourcing Partner, Pippa Lee - to reflect on the highlights of 2018 and to look ahead to the most exciting prospects for 2019 - it was little surprise that a most incredibly important conversation point was "what kind of mindset are you looking for in a team member?".

This is not only really important given its obvious pertinence in today's world but especially so because I personally believe that,

in an increasingly data dominated world which is understandably focused around big data, cloud data, data science, machine learning and artificial intelligence, we seem to have lost sight of ourselves - the ultimate creators and the ultimate consumers.

In today's diverse data landscapes, there is undoubtedly very high demand for heritage skills (including Oracle, SQL Server, MS Business Intelligence, Informatica, Qlikview, and SAP’s Business Objects) as well as those skills needed for the more contemporary data platforms or services (like Python, PySpark, DataBricks, Java, SQL, DevOps - and other ancillary competencies - within the Amazon AWS, Google GCP and Microsoft Azure Cloud ecosystems).

However, in a world that is changing at a much faster rate than one can get a university degree, it is no longer sufficient just to know how to "do" but it is now vitally important to know how to "learn".

So, "what kind of mindset is essential to succeed in such a rapidly changing world"? "What key attributes have enabled incredibly successful use cases - including a school teacher's transformation into a data engineer within our data department @AXA"? "What key tools would keep us ahead of the game - in an increasingly competitive talent marketplace"?

The three key attributes - that not only radically enhance ones job acquisition and retention potential but are also extremely sought after by the best data organisations - are the inquisitive mind, being a life-long learner and intellectual humility.

#3 The Speed of Data – Answers

It has been argued that getting data answers "late" is like getting paid the day after being evicted, like "closing the stable door after the horse has bolted", or - most distressingly - like targeting the perfect customer just moments after they click "buy" on a competitor's website.

So, speed of answers - simply put - is the time it takes for data to be delivered in response to a request being made. This could be from a document (a file of sorts), a set of files (or a file system), a database, a website, a data stream, or other means of data capture (at rest, in transit or in flight).

There are countless examples of data systems that can respond to questions asked - including folder search bars on our computers, searches on our App Stores (such as on iPhones and Androids) and Apps, our work databases (such as MS SQL and Oracle), or data platforms (such as Apache's Hadoop and Kafka, Amazon's Redshift, Interana or Snowflake).

However, perhaps most would agree that the most popular data system in today's world is Google- which executes a staggering 3.5 billion searches per day and incredibly returns answers in as little as 0.2 seconds!

#4 The Speed of Data – Changes

Mobile phone technology has come a long way since April 1973 - when Motorola's Martin Cooper made that novel call.

They "have the incredible characteristics of being both flexible and solid at the same time.".

Mobile networks seamlessly allow "new mobile phones to be brought online - and others taken offline - with no noticeable effects on the rest of the network or on the customer experience.".

Simply put, the speed of change is the distance between wanting something and getting it (be this "something new" or be this a change to "something old").

A previous article discusses delivering seamless "data change" - at pace - with an approach "atypical of the traditional". The approach shares significant characteristics with the "mobile phone networks" of today's world.

#5 Friction between People and Teams

Given significant advancements in technology - not least with Big Data, the Cloud, and IoT at the Edge - why does data still feel, for most, like being back in the stone ages?

The simple - single word - answer seems to be "friction".

There seems to be a kind of friction that causes data to slow down - as it travels from its sources to us - as and when we want it. There seems to be a sort of friction that drags out timelines when we try to deliver data projects - even within environments that should be Agile. There also seem to be some organisation friction between people and teams who work across data - such as the frustrations sometimes evident between data engineers and data scientists.

A key challenge seems to be around how people and teams are organised - with the pertinent question being, "how can we best organise ourselves to optimise for frictionless interaction?".

Team are predominantly organised either as "functional" silos or as "autonomous" silos - with the functional silo model typically optimised for frugality and efficiency, whilst the autonomous model typically optimised for flexibility and agility.

What model best suits your organisational goals or ambitions? Does your current organisational model align with your organisational goals?

#6 The Fear of Failure

The World's richest man (Jeff Bezos) has controversially predicted that his company (@Amazon) will fail. This "leaked" news, perhaps not too surprisingly, sent shockwaves through the stock market and Amazon's share price rapidly tumbling down - towards earth.

But for some, it all seemed a bit strange.

In an interesting debate - amongst a group of friends - one's challenge back to the news was "but isn't learning from failure the new buzz in town and meant to be the new way of doing business?"

He was absolutely right. Learning from failure indeed seems to be the new way to succeed - in business. In fact, billionaire inventor Sir James Dyson declared that it took "15 years creating 5,126 versions that failed" to eventually make his first Dual Cyclone vacuum cleaner, a Morihei Ueshiba quote suggests that "Failure is the key to success; each mistake teaches us something" and the Harvard Business Review has actually documented Strategies for Learning from Failure.

So, "how do we conceivably reconcile these polar notions of failure? One seems ultimately good and the other ultimately bad?", our conversation continued.

One of the answers settled on was that, there was a need to consider two broad categories of failure: the iterative and the incremental.

Iterative failure seems to associate itself with the kind of intentional experiments routinely conducted by the Netflix Chaos Monkey - which randomly chooses servers within their "production environment and turn them off during business hours" in order to extend their known boundaries of operational resilience (also available on GitHub for those who might fancy a bit of added chaos across their technology landscapes). This fail fast approach to innovation and feature delivery now seems the standard that underpins what is fast transforming into a speed economy - across the business world - and is habitually employed by such big hitters as, Google, Facebook, Apple and LinkedIn.

Incremental failure - on the other hand - seems more closely linked to the total failure experienced by the likes of Blockbusters (which I alluded to in a previous piece), the big Lehman Brother's crash that caused global financial systemic failure (here is a previously published paper on how data might prevent a future occurrence) or of the ilk that Jeff Bezos fears is an inevitability for Amazon. The drivers that underpin this kind of failure seem to include cumulative lack of innovation, complacency, indecision, inactivity or hyperactivity.

Interestingly, both "doing nothing" and "doing too much" were considered potential danger scenarios!

Dosage seemed the other key factor to consider.

To explain this, the analogy of vaccinations was chosen - where minute quantities of a disease causing bacteria may be introduced into our bodies in order to protect us from larger quantities and, as such, from the disease. Too little and the vaccination would be infective, yet too much and it could lead to fatality.

So, what does the World's richest man's prediction - that Amazon will fail - tell us? If nothing else, it says,

"failure is inevitable, and so, our best route to success would be to deeply understand its various forms and to deeply understand how to either avoid them or learn from them."

#7 Trust and the Data Economy

In Our World Today

Can we trust with our data - or those who we entrust our data with? While a lot of our attention has recently been consumed by the Cambridge Analytica data breach, our data expands much beyond Facebook likes and Twitter tweets. Our data defines us. Our data confirms who we are, where we live, where we go, how much money we have, what we own and even the people we care about.

Some wonder why there seems to be all this fuss - in recent times - especially given that data has always been with us (from the beginnings of time). But, while this is indeed the case, the currently emerging landscape does significantly differ.

On the one hand, personal data (such as to do with who we are and who our friends or loved ones are) has never - in history - been so easily accessible. Such data would previously have been locked into little silos (such as our diaries, address books, filing cabinets or similar). But, this new and emerging data age goes even further from and beyond that. Data that, arguably, may never have been expressed before - such as our thoughts, intentions, emotions, beliefs, our likes and our fears - are captured and processed at rates never previously known to man (or woman).

And it is the creation, capture, collection and conversion of these new data assets that forms the foundation of a new data economy. In this new economy, an organisation's value no longer solely lies in the products and services that they can offer (today), but in their ability to continuously improve consumer experience to retain loyalty (tomorrow). 

On the other hand, respected commentators - such as Rachel Botsman - have recently focused a lot of attention on trust and its pertinent importance for the emerging economic revolutions of our time.

Her TedTalk on collaborative consumption and reputation capital discuss how the data that enables the creative connection of you and me - on platform forms such as Uber and Airbnb - is inadvertently creating a trust denominated currency.

Whilst trust is by no means new, perhaps an area of notable change is in the shift from its harder face (such as might have been focused around the accuracy of financial numbers) to a much softer face (as may concern such elusive notions as reputation, credibility, influence and status).

With research suggesting organisations typically lose over 5% of their entire market value within the first 5 days of a "trust" damaging "data" leak or hack, with the data that flows through the veins of the world's cryptocurrencies (such as BitCoin) underpinned by the decentralised "trust" network which is BlockChain (in effect creating cash-like "tokens of trust" that can be exchanged for financial value), and with both international regulators and global financial leaders convinced that the timely availability of trusted data is not only "a key step" but "an important step" for mitigating the risk of global financial systemic failure, the link between "trust" and "data" cannot be over emphasised.

And it is this intrinsic connection - between "trust" and "data" - that led the CEO of Gigya, Said Patrick Salyer, to suggest that, “ if data is the new oil, then trust is the ultimate currency that drives this new data economy.”

In Our World Tomorrow

Imagine waking up in the morning to your preferred high energy music from the futuristic Vobot sMart Alarm Clock - with the weather and traffic situation read to you as you lazily peal yourself off your bed and sleepwalk into the shower. Imagine sitting down to your first cup of coffee whilst the intelligent Amazon Echo Spot reads you pertinent news - after having ensured that your sMart Nest Learning Thermostat has the living room heated to the perfect temperature and having already orders your Uber taxi. Imagine settling into your driverless Uber - where your preferred newspaper and bottle of sparkling water await - just as an alert on a Ring sMart Security Camera alert on your sMart phone confirms an Amazon drone delivery into your back garden and your Garmin Fenix sMart Sports watch reassures you of your gradually improving stress levels. Happy days?

But, every one of these interactions is powered by artificial intelligence, robotic automation, machine learning, the internet of things (IoT) and/or blockchain. And, every one of these interactions was enable by data - which is in cases personal or sensitive. And, every one of these interactions increasingly requiring entrusting you valuable data to a third party managed service - and by implication to a third party organisation.

And so, in our future world - which is not only set to be fundamentally driven by data but to be increasingly infiltrated by artificial intelligence, robotic automation, machine learning, the internet of things (IoT) and blockchain - the emerging battleground for competitive advantage is undoubtedly going to be underpinned by trust.

So, what do all these imply - for the future of data, data science and artificial intelligence?

#8 The Future of Jobs

In an incredibly insightful article, Jon Williams discusses some thoughts and opinions on what challenges and opportunities the future of work might present.

A telling analogy was the Uber and gig economy enabling GPS, data and AI technology juxtaposed alongside the near overnight extinction of the printed street map publishing industry. However, whereas a sub section of the publishing industry undoubtedly did suffer consequences, the overall economic impacts are arguably positive - with it not effacing the taxi driving profession (if anything making it even easier and more accessible) and with it providing more consumer choice. This revolutions has exerted significantly more upwards pressure on efficiency and downward pressure on price - a win-win situation for economic productivity and ultimately for customer experience.

However, most incredibly - given the number of factors that come into play in any attempt to predict a most uncertain future - it remains incredibly difficult to be sure of what jobs and/or skills are likely to be required or are likely to be "safe".

One of the only certainties, therefore, is the "need to do whatever we can to provide access to and support for a lifetime of learning" - especially given as it has become increasingly clear that "no-one can afford to set and forget their working life any more."

#9 The Future of Learning

So, "will robots take our jobs?". The mere fact that this search term returns no fewer than 69 million Google results makes clear that this subject is not only ubiquitous but is a matter of pertinent concern.

As the world continues on what seems an incessant and accelerating technologically enhanced drive towards utopia, there is an evidently inherent risk of today's "jobs" disappearing.

It is, however, quite telling that "as technological innovation accelerates at an ever-increasing rate, human and business capability to derive value from this productivity potential lags ever farther behind - because education and skills cannot keep pace with this rate of change."

And so, it is becoming increasingly clear that, as Mauricio Macri - President of Argentina - stated at the 2018 G20:

“The future of work will be a race between education and technology.”

But one needs not look to seemingly distant research to see evidence of this - in reality.

Anyone who has experienced a technology transformation - perhaps a move to the cloud, machine learning or other digital enabled transformation - would know only too well that, it is not the "legacy" or "heritage" technology that is ultimately left behind but the people.

And so, the trend seems to be that there is an every increasing gap between the rate of technology change and the rate at which people or skills are able to keep up. It is, therefore, quite clear that if the future of work is a race between technology and education, then education is increasingly showing to be the clear loser.

So, what can be done about this and what needs to change?

There can be little doubt that, in order to remain relevant in tomorrow's world, there must be a move from a "knowing" mindset to a lifelong "learning" mindset - or, in other words, a shift from knowing how to "do" to knowing how to "learn". And so, one can imagine a future world where there is a radical shift in focus towards skills of adaptability and speed of assimilation - increasingly driven by absorptive capacity, agility and flexibility. And this would, undoubtedly, have radical and profound implications not only for universities - and other educational institutions or systems of learning - but also on organisations and the fundamental concept of knowledge.

A fundamental shift for those businesses looking to remain competitive would most certainly see capital investment increasingly channelled towards training and development - especially as more and more organisations come to realise that they cannot sustainably buy all their talent requirements anymore but would need to look to build some "internally" too.

So, what do these challenges mean for you and how might the knowledge presented above help you prepare for an increasingly uncertain future?

Do click "like" (below) if you like this, follow me on Linkedin, if you want more, and "comment" (below) if you would like to make a contribution.

Thank you