Data Transformations in Machine Learning: A Deep Dive with the Breast Cancer Dataset

Data transformation is a critical step in the machine learning pipeline, ensuring that data is in the optimal format for modeling. In this article, we will explore the importance of data transformations, different types of transformations, and how they can be applied to the Breast Cancer dataset using Python. We will also include visualizations to illustrate the effects of these transformations.

Why Data Transformation is Important in Machine Learning

Data transformation is vital in machine learning for several reasons:

1. Model Performance: Many machine learning algorithms, such as linear regression or neural networks, perform better when the input data is normalized, standardized, or transformed to a specific distribution.
2. Model Interpretability: Transformed data can lead to models that are easier to interpret, especially when dealing with features on different scales.
3. Algorithm Compatibility: Some algorithms require specific data formats (e.g., categorical data needs to be encoded as integers or one-hot vectors).
4. Handling Non-Linearity: Polynomial transformations and other non-linear transforms allow for capturing relationships that a linear model might miss.

1. Loading and Understanding the California Housing Dataset

The California Housing dataset includes features such as median income, median house value, and other socio-economic indicators for various districts in California. This dataset is widely used for regression tasks.

from sklearn.datasets import fetch_california_housing
import pandas as pd

# Load dataset
housing = fetch_california_housing()
df = pd.DataFrame(housing.data, columns=housing.feature_names)
df['target'] = housing.target

# Display the first few rows
df.head()        

1. Numeric Data Type Transformations

Numeric data types include integers and floating-point values, which often require scaling or normalization to enhance model performance.

1.1 Normalization Transform

• Purpose: Scale numerical data to a range between 0 and 1. This is particularly useful for algorithms like neural networks where data in different scales can slow down or skew learning.
• When to Use: Apply when the features have different scales or when using algorithms sensitive to the magnitude of data.

from sklearn.preprocessing import MinMaxScaler
import matplotlib.pyplot as plt

# Normalize data
scaler = MinMaxScaler()
normalized_data = scaler.fit_transform(df[['MedInc', 'AveOccup']])

# Visualizing the original and normalized data for 'MedInc'
plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plt.hist(df['MedInc'], bins=30, color='blue', alpha=0.7)
plt.title('Original Median Income')

plt.subplot(1, 2, 2)
plt.hist(normalized_data[:, 0], bins=30, color='green', alpha=0.7)
plt.title('Normalized Median Income')
plt.show()        

1.2 Standardization Transform

• Purpose: Adjust data to have a mean of 0 and a standard deviation of 1. This is helpful when features have different units or when the data follows a Gaussian distribution.
• When to Use: Useful when dealing with Gaussian distributions or when your algorithm assumes normally distributed data.

from sklearn.preprocessing import StandardScaler

# Standardize data
scaler = StandardScaler()
standardized_data = scaler.fit_transform(df[['MedInc', 'AveOccup']])

# Visualizing the original and standardized data for 'MedInc'
plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plt.hist(df['MedInc'], bins=30, color='blue', alpha=0.7)
plt.title('Original Median Income')

plt.subplot(1, 2, 2)
plt.hist(standardized_data[:, 0], bins=30, color='red', alpha=0.7)
plt.title('Standardized Median Income')
plt.show()        

1.3 Power Transform

• Purpose: Make the data more Gaussian-like, which can improve the performance of models that assume a Gaussian distribution.
• When to Use: When your data is skewed and you want to apply a transform that stabilizes variance.

from sklearn.preprocessing import PowerTransformer

# Power transform
pt = PowerTransformer()
power_data = pt.fit_transform(df[['MedInc', 'AveOccup']])

# Visualizing the original and power transformed data for 'MedInc'
plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plt.hist(df['MedInc'], bins=30, color='blue', alpha=0.7)
plt.title('Original Median Income')

plt.subplot(1, 2, 2)
plt.hist(power_data[:, 0], bins=30, color='purple', alpha=0.7)
plt.title('Power Transformed Median Income')
plt.show()        

1.4 Polynomial Transformation

Purpose: Polynomial Transformation is used to create new features by taking existing features and raising them to a specified power (e.g., squared, cubed). This process allows machine learning models to capture more complex, non-linear relationships between features that might not be apparent when using the original features alone.

By expanding the feature set to include polynomial terms (e.g., x2x2x2, xyxyxy, etc.), polynomial transformations can enhance the model’s ability to fit the data more closely, particularly when the relationship between the features and the target variable is non-linear. This is especially useful in linear models where the original features do not fully capture the underlying patterns in the data.

When to Use:

• Non-Linear Relationships: Use polynomial transformations when you suspect or know that there are non-linear relationships between your features and the target variable. For instance, if a scatter plot of the data suggests a curved relationship, polynomial features can help the model fit the curve.

from sklearn.preprocessing import PolynomialFeatures

# Polynomial transformation (degree 2)
poly = PolynomialFeatures(degree=2, include_bias=False)
poly_data = poly.fit_transform(df[['MedInc', 'AveRooms']])

# Visualizing the relationship between original features and transformed features
plt.figure(figsize=(12, 6))
plt.scatter(df['MedInc'], df['AveRooms'], c=df['target'], cmap='viridis', alpha=0.5, label='Original Data')
plt.scatter(poly_data[:, 0], poly_data[:, 1], c=df['target'], cmap='plasma', alpha=0.5, label='Polynomial Transformed Data')
plt.title('Polynomial Transformation: Median Income vs Average Rooms')
plt.xlabel('Median Income')
plt.ylabel('Average Rooms')
plt.legend()
plt.show()        

1.5 Quantile Transform

• Purpose: Imposes a specific probability distribution (e.g., uniform or Gaussian) on the data.
• When to Use: When you need to force a specific distribution on a variable, especially in cases with non-Gaussian distributions.

from sklearn.preprocessing import QuantileTransformer

# Quantile transform
qt = QuantileTransformer(output_distribution='normal')
quantile_data = qt.fit_transform(df[['MedInc', 'AveOccup']])

# Visualizing the original and quantile transformed data for 'MedInc'
plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plt.hist(df['MedInc'], bins=30, color='blue', alpha=0.7)
plt.title('Original Median Income')

plt.subplot(1, 2, 2)
plt.hist(quantile_data[:, 0], bins=30, color='orange', alpha=0.7)
plt.title('Quantile Transformed Median Income')
plt.show()        

2. Categorical Data Type Transformations

Categorical data often needs to be encoded in a way that machine learning algorithms can process, such as converting categories into numbers.

2.1 Ordinal Transform

• Purpose: Convert categorical variables into ordinal (ranked) integers.
• When to Use: When your categorical variables have a meaningful order.

import numpy as np
import pandas as pd
from sklearn.datasets import fetch_california_housing
from sklearn.preprocessing import OrdinalEncoder
import matplotlib.pyplot as plt

# Load the dataset
housing = fetch_california_housing()
df = pd.DataFrame(housing.data, columns=housing.feature_names)

# Create a categorical variable based on 'MedInc'
df['Income_Category'] = pd.cut(df['MedInc'], bins=[0, 2, 5, np.inf], labels=['Low', 'Medium', 'High'])

# Ordinal Encoding
encoder = OrdinalEncoder(categories=[['Low', 'Medium', 'High']])
df['Income_Category_Ordinal'] = encoder.fit_transform(df[['Income_Category']])

# Visualization
plt.bar(df['Income_Category'], df['Income_Category_Ordinal'])
plt.title('Ordinal Transform of Income Category')
plt.xlabel('Income Category')
plt.ylabel('Ordinal Value')
plt.show()        

2.2 One Hot Transform

Purpose: Convert categorical variables into a series of binary variables.

When to Use: When there is no intrinsic order in the categorical variables.

from sklearn.preprocessing import OneHotEncoder

# One Hot Encoding
onehot_encoder = OneHotEncoder(sparse=False)
onehot_data = onehot_encoder.fit_transform(df[['Income_Category']])

# Visualization
plt.imshow(onehot_data, cmap='viridis', aspect='auto')
plt.title('One Hot Transform of Income Category')
plt.xlabel('Income Category')
plt.ylabel('Sample Index')
plt.colorbar(label='Binary Encoding')
plt.show()        

Visualization Explanation:

• The heatmap shows how each category (‘Low’, ‘Medium’, ‘High’) is represented as a binary vector. Each column corresponds to one category, and each row corresponds to a sample in the dataset.

2.3 Discretization Transform

Purpose: Convert continuous numeric data into discrete bins, effectively converting it into ordinal data.

When to Use: When you want to segment continuous data into categories.

from sklearn.preprocessing import KBinsDiscretizer

# Discretization
discretizer = KBinsDiscretizer(n_bins=3, encode='ordinal', strategy='uniform')
discretized_data = discretizer.fit_transform(df[['MedInc']])

# Visualization
plt.plot(df['MedInc'], label='Original Median Income', alpha=0.5)
plt.plot(discretized_data, label='Discretized Income', alpha=0.5)
plt.legend()
plt.title('Discretization Transform of Median Income')
plt.xlabel('Sample Index')
plt.ylabel('Income Value / Discretized Value')
plt.show()        

Visualization Explanation:

• The line plot compares the original MedInc values with their discretized versions, showing how continuous income data is segmented into ordinal bins.

Conclusion

Data transformations, particularly for categorical data, are essential steps in preparing data for machine learning models. Properly encoding categorical variables into ordinal, one-hot, or discretized formats can significantly influence the performance and interpretability of the models. The examples above demonstrate practical implementations using the California Housing dataset, providing a clear understanding of when and how to apply these transformations effectively.

These visualizations illustrate the impact of different categorical data transformations, making it easier to decide which transformation to use based on the data’s characteristics and the model’s requirements.

Stay Tuned!!

Thank you for reading!

If you’d like, add me on Linkedin!

Add me on Medium