#Artificial Intelligence #25 - My challenges with the definition of data centric vs model centric

Welcome to #Artificial Intelligence #25

Introduction

Like almost all of us, I also got the first experience of AI through Andrew Ng’s flagship course. However, recently, Andrew has been speaking of Data centric vs model centric approaches and has proposed a definition of Data centric AI as ?“ Data-centric AI is the practice of systematically engineering the data used to build AI systems.”. ?

I have some challenges with this – and I see it as wider issue with deep learning per se. I hope here, I can create a wider discussion which will benefit us all as an industry.

Background

On first impressions, the definition of data centric AI sounds perfect. Good data is needed for good models of course. MLOps helps to make high quality data throughout the lifecycle. All fine. ?

What’s not to agree? ?

For me, the challenge comes in the second part: Data centric vs model centric

Here are some of the reasons why:

1)????Restricted dichotomy: ?In a nutshell, the problem is framed as A vs B when there could be options C, D or E or various combinations therof as we see below

2)????Paradoxical starting point: The data centric approach is supposed to hold the model fixed and iterate on the data. This raises a paradoxical question: How exactly do we choose a model? And when do we fix the model? What is the criteria for fixing a model? Ironically, to be able to fix the model, we need to know in advance about a range of different models – which makes it like a recursive argument since in that case, models come before data.

3)????Known unknowns: ?There is a general guideline but no fixed way to choose a model. Are we saying here that we choose a model (ex: logistic regression or SVM) and keep throwing data at and tuning it (ignoring any other models) when there could be a better model downstream? (we suffer from the known unknowns problem)

4)????The baseline approach: In how to get baseline results and why they matter, Jason Brownlee says “You may need to collect more or different data from which to model. You may need to look into using different and perhaps more powerful machine learning algorithms or algorithm configurations. Ultimately, after rounds of these types of changes, you may have a problem that is resistant to prediction and may need to be re-framed.” This statement sounds more pragmatic to me i.e. many things could change including models or indeed the problem itself.

Other possible options

Then, there are a number of possibilities which are possible outside of data centric vs model centric approach.

1)????Human in the loop: The data centric v.s. model centric argument ignores other strategies like human in the loop. In the article?An Inconvenient Truth About AI won't surpass human intelligence any..., the author Rodney Brooks, makes a point for a human in the loop and that you cannot trust AI alone. “Just about every successful deployment of AI has either one of two expedients: It has a person somewhere in the loop, or the cost of failure, should the system blunder, is very low.” ?Data centric approach puts our faith entirely on data ignoring other strategies like human in the loop.

2)????Models are themselves evolving in a number of directions: ?For example causal models re work of Judea Pearl – Book of why and Bayesian approaches in general.

3)????There are also alternate views more critical of deep learning itself : Deep learning with gary marcus - and leaning to symbolic ideas Do we still need models or just more data and compute? Max Welling, A

The risks of Industry consensus

There is a risk of groupthink in the industry as a whole and we see it in many ways. It’s a narrative that the deep learning industry wants us to believe. Lots of data and lots of compute ?alone will drive AI. But there is a dark side to this. Large companies want your data and large companies are then in a dominant position to anyone else who have no data and cannot pay for compute. That was also the broader concern raised by?Timnit Gebru?. There is also a wider backstory to data centric vs model centric question

Deepmind says reward alone is enough . Yoshua Bengio has a very detailed NeurIPS talk (Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)) which attempts to show how neural networks / connectionist approaches will address even complex phenomenon such as system 2 (by Kahneman) . ?A recent paper,?Deep Learning for AI by Yoshua Bengio, Yann Lecun, Geoffrey Hinton also says “The performance of deep learning systems can often be dramatically improved by simply scaling them up.?“. Collectively, these approaches address the shortcomings of deep learning such as the "bigger is better" mindset (breakthroughs are based on creating larger models and datasets); machines learn in a relatively narrow way and need much more data to learn when compared to humans; neural networks are vulnerable to small changes in data ?– for example adversarial attacks etc. There are various approaches to achieve this including handling out of order distributions; attention mechanisms; transfer learning; importance of priors; composability and self-supervised learning.

Conclusion and discussion

To conclude, the definition is agreeable i.e. you agree to it intuitively but it creates a restricted dichotomy (ignores other options) and a contrived scaffolding (ignores different viewpoints)

But more than semantics, I think the wider industry viewpoint may need to evolve. I believe that the next generation of deep learning would be radically different from the last. My personal bet is on Bayesian models, Causal models and their interplay with existing deep learning models. There are more complex possibilities on the model development side for example the Bayesian Active Learning library (BAAL). So, in a wider sense, models will play a key part in the future

AI for Marketing conference

Finally, I am planning to chair an online conference at the guild. If you are in marketing: what topics would you suggest for AI and marketing? From the AI side I know, I am keen to hear from you if you have a marketing background and using AI and plan to use AI

Interesting book

I read an interesting book – about Fourier series and Fourier transforms written by Japanese music students - Who Is Fourier?: A Mathematical Adventure . The book is unique and unclassifiable and spans ?music, mathematics, physics, engineering, and complex science. In parts – hard reading – but I really liked the effort to combine domains like maths and music and create something new

?

Many thanks?

If you want to study with me at #universityofoxford see my courses

Digital Twins: Enhancing Model-based Design with AR, VR and MR

This course is for aspiring and seasoned simulation engineers that want to develop digital twin models of engineering components and incorporate these models into AR-VR-MR technologies.

and

Artificial Intelligence: Cloud and Edge Implementations

This is a pioneering full-stack AI course covering AI, MLOps and Edge.

The course helps developers to transition their careers to AI.