Domain Models/ Statistics Models and AI Models

Recently I was talking to few executives of a Process Plant( Fertilizer company) where they have been contemplating acquisition of a high end engineering solution which maps in detail all the mass and energy balances in a fertilizer plant. Objective of this system was to improve plant’s overall efficiency, bring about dramatic changes in understanding of material consumption, plant’s blind spots and take plant to the next frontier of efficiency. Normally most of the process plants carry out such initiatives to improve overall plant capacity, also known as debottlenecking.

This debottlenecking yields good results if done properly over and over for few times. Next level is to see energy consumption optimization which again like de-bottlenecking is a continuous journey. Further to this process plants try to employ systems which have strong engineering model of plant capturing mass and energy balance equations to stat with and other next level of engineering equations and models. Let’s call collective of all such models in this case specifically as Engineering Model and more generally a “Domain Model”. This particular “Domain Model” builds from the understanding of processes, chemistry and other engineering sub-branches. This “Domain Model” also couples years of understanding about various systems and processes in a plant.

Such Domain Models have rich information and equations and models about how particular domain works, what are subsystems, how various parts of systems interact with other, how various systems interact with plant as a whole. Domain models give good results in the initial journey of efficiency and quick results in terms of other performance measures or KPI’s as planned by the domain specialists.

Now comes the next set of Models, the “Statistical Model”.

Domain Model limitations as well as reducing marginal utility gives rise to the need of next model. In this case the “Engineering Model” which the defacto “Domain Model” bases all its calculations, relationships and equations on the understanding about the plant and its components. Mostly the design philosophy is top-down plus starting from lower granularity. This gives rise to un-adressed issues of “systems view” issues which range from lot of undiscovered relationships, dependencies, causal relationships, interaction issues between various independent variables. Here comes the need and use of statistical modelling which starts with mapping independent variables, dependent variables then move on to modeling various Factors, through Factor Analysis and Principal Component Analysis. Then moving to Regression Analysis( various ways of doing this through linear, logistic regression, kernel regression) moving to multivariable analysis. Eventually covering SVM (Support Vector Machines) and other methods in statistical analysis and modeling.

Statistical modeling requires considerable training as a statistician and knowledge and practice of how to create good data samples, elaborate repertoire of techniques, algorithms to the designer of these models. Yes, statistical model draws from the “Domain Model” and builds on top of it. It has its advantages in creating not so bottom-up models, statistical models allows to start from mid and go up, so lets call these kind of models as “Mid-to-top” models. Statistical modeling gives good results in terms of creating “predictive models” for various Performance Measures. Simple to complex regression can help model Plant performance measures like Plant Yield, Throughput, Efficiency, Energy Consumption, and Profitability and so on. Insightful relationships can be discovered, all in all statistical modeling will result in basic descriptive statistics as all as to some extent predictive modeling provided a lot of trained statisticians work

closely along with domain specialists( in this case plant operators, chemical engineers, instrumentation engineers).

The journey of statistical modelling assumes lot of about data availability. The case in point has elaborate control systems cross the plant and various generations of PLC ( Programmable Logic Controllers), SCADA( Supervisory Control and Data Acquisition) and DCS ( Distributed Control Systems) churning out not only control signals but collecting and acting on vast amounts of data coming out of process loops. These are control loops across the plant maintaining level, flow, temperatures, composition and various other process parameters required for smooth functioning of fertilizer plant. A typical mid-scale plant will have closed to 5000 such loops ( also called as Control loops).

These triad of systems PLC/SCADA/DCS along with plant sensors and valves ensure that plant run the way have to alongwith maintaining material flow, energy flow and information flow inside various areas of plant, systems like boiler systems, compressors, blenders and hundreds of other systems in a typical plant.

Classical Statistics in University Under-graduate courses or even Graduate courses starts with descriptive statistics and then moves into distribution fitting and then all the way to complex multivariate analysis. Essentially covering hypothesis testing, correlation, regression, factor analysis and Principal Component analysis. 

       Statistics assumes a lot of a-priori knowledge about the data and its properties and does not necessarily cover a lot of trial and error or even tinkering.

Machine Learning in new age looks at wide array of techniques and algorithms which themselves learn from the data. Deep Machine Learning, Supervised Learning and Reinforcement Learning covers very interesting algorithm which learn themselves from the wide array of data. So data becomes input and model becomes output. This happens without any human intervention (except in supervised learning). This is the real beauty of ML over conventional statistics. Although new age ML ( covering CNN/Deep Learning/Reinforcement Learning) draws a lot from statistics, cognitive biology, neuroscience, mathematics and control theory, most of the ML applications have been very new and have large technical and business impact.

In Reinforcement Learning classical optimization functions are used and behaviorism invested in psychology by Skinner comes into play in terms of “reward and punishment”. So behavior of the RL Algorithm is shaped in the same way a child’s behaviour is shaped by parents. Eventually use of Dynamic Programming from the classical optimization (Operations Research) is used along with Bellman’s optimality conditions and MDP ( Markov Decision Process)

RL ensures that you can start “learning” with minimum domain or problem knowledge. Algorithm has power to learn and come up with its parameters depending on the error conditioning and reward optimization. Multiple algorithms like Temporal Difference Learning, Deep ! Learning and Actor Critic Methods ( A3c) ensure that algorithms in RL have power to create truly domain independent ways to learn in many many new domains without need to have domain knowledge.

ML Tribe( collection of AI Scientists, Data Analysts, ML practitioners, Students, Professors and Industry Professionals) is significantly different from old school statistics in many ways. Statistics assumes a lot of knowledge about the system. Statistical thinking in many ways is top-down, a-priori thinking. ML( Broad umbrella of algorithms in RL, Deep Learning) thinking is inherently is posterior, does not assume much and is bottom-up. In many ways as Richard Dawkins puts it “The Darwinian thinking is mindless, purposeless bottom-up processes involving R&D, Trial and Error and Tinkering all the way”. ML resembles our own biological evolution. The same way as biological evolution ML algorithms are also evolving. The big advantage is ML algorithms evolution is much faster unlike biological gradual, slow evolution.

ML works a lot like biological processes seen elsewhere in nature. Sometimes ML does not necessarily try to Optimize in the classical Optimization Sense (finding the best possible solution from large scale solution space). ML tries a process of sophisticated tinkering which moves from finding one sub-optimal solution and then move ahead. This process ensures continuity in learning as well as learning becomes in many ways autonomous.

Statistics used to need a lot of careful sampling, sometimes meticulously planned data cleaning would pre-date a rigorous statistical analysis. ML works with existing data and tries to create inferences.

One of the families of ML algorithms, Bayesian Inferencing using basic Bayes Probability coupled with state-space generators like Monte Carlo simulation so that you create simulated data where data is non-existent or not accurate. ML algorithms this way build a kind of robustness against the Data Quality problems.

In next diagram I have listed down one of my favourite comparison between domain model, statistical model and AI models.

There are 3 clear choices of building models

·      Domain Models

·      Statistic Models

·      Artificial Intelligence Models

Domain Models are built up with lot of domain knowledge. Case in point is a fertilizer plant where the domain model becomes set of engineering models like mass and energy balance equations.

?     Domain Model

?     Engineering Models

?     Built from Ground-up

?     Granular Models/Equations/Formulations

?     A-priori Models

?     Initial Impact/results/outcomes are good

?     Statistics Model

?     Designed at mid-level ( not fully TOD/BUP)

?     Variables, interactions, correlations

?     Logistic Regression, Random Forests, Boosting

?     Needs Intelligent Designers during lifecycle

?     Draws from Domain Model, next level of impact/results/outcomes

?     Artificial Intelligence Models

?     Data As Soil, Algorithms as Seeds and Plants as AI Programs

?     Bottom-up as far as working mode is concerned

?     Reinforcement Learning ( Stimulus/Response)

?     Continuous results, improvement process becomes autonomous, biological and anti-fragile.