Artificial Intelligence #86: Artificial Intelligence for Engineering sciences

Hello all

Background

We completed the Digital twins course and its nice to see students who have benefited from the course. Some comments from our students:

Joanna Trela-Zielińska?

Anwaar Ulhaq, PhD

Yassine Oukaci

Digital twins is a part of a larger philosophy we are using at the University of Oxford for understanding Artificial Intelligence for Engineering sciences.?

I shared the image before which I use for the wider understanding of Digital Twins.?

In this post, I will explain more how we see Digital twins as the central element of Artificial Intelligence for Engineering Sciences

Note that this area is a bit complex but we are working on a more 'low code' version of this for engineers who want to learn data sciences. If you are interested in this, please let me know.

In general, you model phenomenon that ‘moves’ - for that reason, you see the world as a graph or a time series.?

Our overall thesis is:

• Information / Data comes in two main formats - time series and graph
• ?It can then be modelled in various formats (process mining, graph, causal, simulation(discrete event simulation, agent dynamics, systems modelling, sequential decision analytics) , multivariate time series etc
• from an output perspective - it can be rendered as a simulation, a visualization or a feedback loop
• the feedback loop could be IoT based or human

Each of these areas are very complex - making AI for engineering sciences a complex discipline.?

I will explain some of these areas in more detail (from sessions in our course)

Graph neural networks

Typically the system (twin) sees the model as a graph or as a time? series. My colleague Nikita on our course did a great job of explaining the significance of graph neural for digital twins Why Graph Neural Networks could be useful for Digital Twins?

Time series

You could also see data as time series.You can see time series in a wider context as below

Univariate: ARIMA, Exponential Smoothing, Theta Method. Classical techniques which operate only on the history of the timeseries and nothing else. for example: Weather forecast of a single location

Multivariate time series:? Machine Learning (Many-to-one) and Deep Learning (Many-to-Many). Modern techniques which are capable of modelling multi-variate timeseries. Ex Weather forecast of all locations in a country

Feature Engineering to convert TSF to ML: Time Delay Embedding, Temporal Embedding. These include techniques to convert timeseries problems into regression problems. Lags, Rolling features etc. are very commonly used techniques to capture temporal dependence

Machine Learning for TSF: Linear Regression, Gradient Boosting(LightGBM), and Machine Learning workflow

Deep Learning for TSF: PyTorch, RNN, LSTM, Attention, Transformers. Understand how we can use Deep Learning (PyTorch) for TSF, by understanding a few common architectures (with code examples)

Global Forecasting Models:? Assuming multiple timeseries originating from a single data generating process and using a single model to forecast all of them. Applicable to both ML and DL

Global Forecasting Models (ML): LightGBM. How to use any ML model in the global forecasting paradigm to generate forecast for an entire dataset of timeseries (with code examples)

Global Forecasting Models (DL): RNN, LSTM, Transformers. How to use any DL model in the global forecasting paradigm to generate forecast for an entire dataset of timeseries (with code examples)

Specialised DL Architectures: N-BEATS, N-BEATSx, Informer, Autoformer

Note this does not include Reinforcement learning because in RL (by definition) the agent alters the environment (whereas in a timeseries the environment is not altered - although both problems are temporal)??

We cover global time series models in our course through Manu Joseph author of? Modern Time Series Forecasting with Python: Explore industry-ready time series forecasting using modern machine learning and deep learning - a book that I recommend

?Simulation

Simulation is at the heart of AI for Engineering Sciences for us at Oxford.?

Its also a big focus going forward ex from werner Vogels (amazon CTO) talk of their new product Simspace weaver based on simulation and what-if analysis?

The three main models (Systems dynamics, agent based modelling, discrete event simulation) are all key and have synergies with digital twins as a simulation technique

Discrete event simulation: Discrete Event Simulation and Digital Twins: Review and Challenges for Logistics

Agent based modelling: The Use of Agent-Based Models Boosted by Digital Twins in the Supply Chain

Systems dynamics - Product Processes based on Digital Twin

You can model this functionality through Python packages like? simpy

Similarly causal machine learning is also a part of the simulations for example Causal Machine Learning:A Survey and Open Problems and CausalCity: Introducing a high-fidelity simulation with agency for advancing causal reasoning in machine learning

Other modelling strategies for artificial intelligence and engineering sciences

Some of the other areas we follow are:

Physics informed machine learning

Process mining for discovery using pmtk?

We are working on a more 'low code' version of this for engineers who want to learn data sciences. If you are interested in this, please let me know.