Regression Modeling For Design Engineers Part #1: Introduction

Modeling is the cornerstone of Engineering Design. Models, in Engineering Design Process, are used to demonstrate that the design, in fact, works as intended. Models can be very useful in giving an in-depth understanding of how the product will respond to different inputs (accuracy, precision, repeatability), to perform design optimization, and perform design trade-offs during the design phase.

These models can be derived from:

1. Mathematical equations based on Physics.
2. Historical Data of measured performance.
3. Numerical Simulations and analyses.
4. Prototypes that are built and tested.
5. Data from Actual system.

Mathematical models are equation-based models that are derived from the basic physics of the problem to be solved. Equations related to the specific problem based on its physics are available from handbooks, textbooks and scientific publications. Numerical simulations are widely popular that use some form of a computer program such as Ansys, Matlab, LabView etc that uses discretization as a means to numerically approximate the physics behind the problem. Prototypes are mock-ups of the final product that can provide significant insights into the functioning and performance of the end product.

Sometimes, engineers will have a repository of historical data or data gathered from experiments based on other models exist that can be used to develop new models or improve upon existing models.

The flowchart below gives a clear understanding of how Regression can be useful in determining an Accurate Approximation Model.

Regression is a process of extracting a Model Y = f(X) from a dataset. A very important first step in Regression Modelling is that the "equation type" must be specified upfront! Regression just fits the coefficients. The coefficients are computed so that the data points fall as close to the specified equation as possible. Hence, the model derived from Regression is always a mathematical equation except for non-parametric regressions techniques.

Regressions Techniques

1. Linear Regression

Liner Regression uses least-square fitting to calculate coefficients for models that are linear with respect to the coefficients. Types of Linear Regression equation-form are shown below.

2. Non-Linear Regression

This type of regression uses least-square fitting to calculate coefficients for models that are non-linear with respect to coefficients. Example equation is shown below:

3. Non-parametric Regression

This type of regression does not require a specific equation type and doesn't fit coefficients. Instead, various techniques are used to build a predictive model of the data set. Example of this type is neural networks, projection pursuit regression etc. In most cases, the regression model takes the form of a Look-Up table.

NEXT: Regression Modeling For Design Engineers Part #2: Methodology