How do you validate engineering models with limited data?

1 Use existing data sources

One of the first things you can do is to look for existing data sources that are relevant to your model. These can be from previous projects, literature, databases, or benchmarks. You can use these data to compare and calibrate your model, as well as to identify the key parameters and variables that affect your system. However, you should also be aware of the limitations and assumptions of these data sources, and make sure that they are compatible with your model scope and objectives.

2 Apply sensitivity analysis

Another useful technique is to apply sensitivity analysis to your model. Sensitivity analysis is a method of quantifying how the model output changes with respect to changes in the model input. By doing this, you can assess how sensitive your model is to different factors, such as uncertainty, variability, or error. You can also identify the most influential input parameters and the range of values that produce acceptable results. Sensitivity analysis can help you reduce the complexity of your model, prioritize the data collection, and improve the model quality.

3 Use surrogate models

A surrogate model is a simplified or approximate representation of the original model that can be used to replace or supplement it in certain situations. Surrogate models can be useful when you have limited data because they can reduce the computational cost, time, and effort required to run the original model. For example, you can use surrogate models to perform optimization, uncertainty propagation, or design of experiments. Some common types of surrogate models are polynomial regression, artificial neural networks, or kriging.

4 Perform cross-validation

Cross-validation is a technique of dividing the available data into subsets and using some of them to train the model and others to test it. This can help you evaluate the performance and generalization ability of your model, as well as to avoid overfitting or underfitting. There are different methods of cross-validation, such as k-fold, leave-one-out, or bootstrap. The choice of the method depends on the size and characteristics of your data, as well as the purpose of your model validation.

5 Incorporate expert knowledge

Finally, you can also incorporate expert knowledge into your model validation process. Expert knowledge can be derived from domain experts, stakeholders, or users who have experience and insights about the system or problem that you are modeling. You can use expert knowledge to define the model objectives, assumptions, and criteria, as well as to provide feedback and suggestions for improvement. You can also use expert knowledge to supplement or validate the data that you have or generate, such as by using expert elicitation, judgment, or opinion.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?