What are the best practices for selecting features in Robotics ML?

2 How to define features

Before selecting features, you need to define them based on the data and the problem you are trying to solve. Features can be raw, derived, or engineered. Raw features are the original data attributes, such as sensor readings, images, or text. Derived features are obtained by applying some transformation or aggregation to the raw features, such as statistics, histograms, or Fourier transforms. Engineered features are created by using domain knowledge, intuition, or creativity to capture some meaningful or useful aspects of the data, such as landmarks, distances, or angles.

The choice of features depends on the type and complexity of the data, the ML algorithm, and the objective of the model. For example, if you are working with images, you might want to use features that capture the shape, color, texture, or orientation of the objects. If you are working with text, you might want to use features that capture the semantics, syntax, or sentiment of the words. If you are working with time series, you might want to use features that capture the trend, seasonality, or autocorrelation of the signals.

3 How to measure feature importance

Once you have defined your features, you need to measure their importance or relevance for the ML model. Feature importance is a metric that quantifies how much a feature contributes to the model's prediction or output. There are several ways to measure feature importance, such as filter methods, wrapper methods, and embedded methods. Filter methods use statistical or information-theoretic criteria to rank features, while wrapper methods evaluate features based on the model's performance. Embedded methods use the ML algorithm itself to select or weight features. Filter methods are fast and simple, but they do not consider interactions between features or the ML algorithm. Wrapper methods are more accurate and comprehensive, but they are computationally expensive and prone to overfitting. Embedded methods are efficient and adaptive, but they may not be transparent.

4 How to select features

After measuring the feature importance, you need to select the optimal subset of features that maximizes the model's performance and interpretability. Feature selection can be done in different ways, such as thresholding, ranking, or searching. Thresholding involves selecting the features that have an importance score above a certain threshold, which is simple and intuitive but may not account for the trade-off between the number and quality of features. Ranking selects the top-k features that have the highest importance scores, which is easy and flexible but may not capture the dependencies or complementarities between features. Searching searches for the best subset of features that optimizes an objective function like accuracy, precision, or recall; it is comprehensive and effective but can be very time-consuming and complex.

5 How to validate features

Finally, you need to validate your selected features by testing them on new or unseen data and evaluating their impact on the model's performance and interpretability. Feature validation can be done in various ways, such as cross-validation, hold-out, or bootstrapping. Cross-validation is reliable and robust, but it can be computationally intensive. On the other hand, hold-out is simple and fast, but it can be biased. Bootstrapping is flexible and stable, but it can introduce noise and variability in the results.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?