Autoencoder for Data Compression, Denoising, and Anomaly Detection

In the world of machine learning, Autoencoders, as specialized neural networks, play a pivotal role by learning efficient data representations, enabling them to compress, reconstruct, and interpret data. This article explores how autoencoders apply data compression, denoising, and anomaly detection using the MNIST dataset.

Understanding Autoencoders

An autoencoder is a type of neural network with two main parts:

• Encoder: Compresses the input data into a compact latent representation.
• Decoder: Reconstructs the original data from the compressed representation.

The process of encoding (mapping each item to a specific location) and decoding (reconstructing the item from its location) is what an autoencoder does in machine learning.

The encoder takes the input (e.g., an image, sound, or data point) and maps it to a lower-dimensional space called the latent space or embedding.

The decoder takes the encoded data and attempts to reconstruct the original item as accurately as possible.

Training the Autoencoder: Minimizing the Reconstruction Error

The goal of training an autoencoder is to minimize the difference between the original input and its reconstructed output. The loss function used during training typically focuses on reducing this reconstruction error, ensuring that the decoded output is as close as possible to the original.

Applications Explored

Once the autoencoder is trained, it can be used for a variety of applications such as:

1. Data Compression and Reconstruction.
2. Denoising Corrupted Data.
3. Detecting Anomalies.

Let’s dive into these tasks with practical code snippets and results.

Task: Data Compression and Reconstruction

Step 1: Preprocessing the MNIST Data

We begin by loading the MNIST dataset, which consists of 28×28 grayscale images, and preparing it for training.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split

# Load MNIST data from CSV
data = pd.read_csv('mnist_data.csv')
y = data.iloc[:, 0].astype(int).values
X = data.iloc[:, 1:].astype(np.float32).values / 255.0  # Normalize pixel values to [0, 1]

# Reshape for visualization
X_images = X.reshape(-1, 28, 28)

# Split data
X_train, X_test, _, _ = train_test_split(X, y, test_size=0.2, random_state=42)

print(f"Training Data Shape: {X_train.shape}, Test Data Shape: {X_test.shape}")
        

Step 2: Designing the Autoencoder

The architecture reduces the dimensionality to a 32-dimensional latent space and reconstructs the original 784-dimensional data.

from tensorflow.keras import layers, models

# Define encoder and decoder
input_dim = 784  # Flattened size of the image
encoder_input = layers.Input(shape=(input_dim,))
x = layers.Dense(512, activation='relu')(encoder_input)
x = layers.Dense(256, activation='relu')(x)
x = layers.Dense(128, activation='relu')(x)
latent_space = layers.Dense(32, activation='relu')(x)

# Decoder
x = layers.Dense(128, activation='relu')(latent_space)
x = layers.Dense(256, activation='relu')(x)
x = layers.Dense(512, activation='relu')(x)
decoder_output = layers.Dense(input_dim, activation='sigmoid')(x)

# Autoencoder Model
autoencoder = models.Model(encoder_input, decoder_output)
autoencoder.compile(optimizer='adam', loss='mse')
autoencoder.summary()
        

Step 3: Training and Evaluation

# Train the autoencoder
history = autoencoder.fit(X_train, X_train, validation_data=(X_test, X_test), epochs=50, batch_size=128, verbose=2)

# Visualize loss
plt.plot(history.history['loss'], label='Training Loss')
plt.plot(history.history['val_loss'], label='Validation Loss')
plt.legend()
plt.title("Loss Over Epochs")
plt.show()
        

Reconstruction results validate the model’s capability to compress and reconstruct images.

Task 2: Denoising Corrupted Data

Adding Noise

# Add Gaussian noise
noise_factor = 0.5
X_train_noisy = np.clip(X_train + noise_factor * np.random.normal(size=X_train.shape), 0., 1.)
X_test_noisy = np.clip(X_test + noise_factor * np.random.normal(size=X_test.shape), 0., 1.)

# Visualize noisy vs. clean images
plt.figure(figsize=(10, 5))
plt.subplot(1, 2, 1)
plt.imshow(X_test[0].reshape(28, 28), cmap='gray')
plt.title("Original Image")
plt.subplot(1, 2, 2)
plt.imshow(X_test_noisy[0].reshape(28, 28), cmap='gray')
plt.title("Noisy Image")
plt.show()
        

Training the Denoising Autoencoder

# Train the denoising autoencoder
denoiser = models.clone_model(autoencoder)
denoiser.compile(optimizer='adam', loss='mse')
denoiser.fit(X_train_noisy, X_train, validation_data=(X_test_noisy, X_test), epochs=50, batch_size=128)
        

Clean reconstruction demonstrates the denoiser's ability to restore corrupted images.

Task 3: Anomaly Detection

Detecting Outliers

Reconstruction error serves as a basis for anomaly detection. Here, random noise is used as anomalous data.

# Simulate anomalies
anomalies = np.random.uniform(0, 1, size=(100, X_train.shape[1]))
X_test_combined = np.vstack((X_test, anomalies))
reconstructions = autoencoder.predict(X_test_combined)

# Compute reconstruction error
errors = np.mean((X_test_combined - reconstructions) ** 2, axis=1)
threshold = np.percentile(errors[:len(X_test)], 95)
predicted_anomalies = (errors > threshold).astype(int)

print(f"Anomaly Threshold: {threshold:.4f}")
        

Performance Metrics

from sklearn.metrics import precision_score, recall_score, f1_score

true_labels = np.hstack((np.zeros(len(X_test)), np.ones(len(anomalies))))
precision = precision_score(true_labels, predicted_anomalies)
recall = recall_score(true_labels, predicted_anomalies)
f1 = f1_score(true_labels, predicted_anomalies)

print(f"Precision: {precision:.2f}, Recall: {recall:.2f}, F1-score: {f1:.2f}")
        

Histograms of reconstruction errors highlight the separation between normal and anomalous data.

Conclusion

Autoencoders provide elegant solutions for tasks involving data compression, noise removal, and anomaly detection. With compact latent spaces and robust reconstruction capabilities, these models bridge human intent with machine learning, unlocking insights from data with precision and clarity.

Use the techniques shared here to integrate autoencoders into your next data science or machine learning project!