Building a Neural Network from Scratch

Picture yourself as a digital architect, about to construct a living, breathing computational brain. Yes, you read that right – we're about to explore neural networks: the beating heart of modern artificial intelligence, where lines of code transform into intelligent thought.

This is the first part of a weekly series where we’ll break down neural network concepts step by step exploring the intricate world of Machine Learning. In this part, you’ll learn:

• How data is prepared for ML
• Why and How we scale inputs before training a model
• Setting up a simple neural network in Python

Let's say you want to predict an output value y given some input value x. Let's take an example of predicting the likelihood of a food insecurity crisis (on a scale of 0–100) based on regional rainfall patterns and crop yield forecasts

When we approach this problem with machine learning. We first need data. Imagine that, over the past five years, humanitarian organizations have recorded seasonal rainfall totals, average crop yields, and food security scores for several vulnerable regions.

Consider the given dataset:

Region | Rainfall (mm) | Crop Yield (tons/hectare)

---------------------------------------------------------------------

Region A | 350 | 2.1

Region B | 420 | 2.8

Region C | 210 | 1.2

Region D | 580 | 3.5

Region E | 120 | 0.9

Region F | 290 | 1.7

We can store this data in two-dimensional numpy arrays using Python:

Python code to store Input Data :

Input Data Code (X)

This code stores the rainfall (mm) and crop yield (tons/hectare) for each region in a numpy array.

import numpy as np

# Input data: [rainfall (mm), crop_yield (tons/hectare)]
X = np.array([
    [350, 2.1],  # Region A
    [420, 2.8],  # Region B
    [210, 1.2],  # Region C
    [580, 3.5],  # Region D
    [120, 0.9],  # Region E
    [290, 1.7]   # Region F
])
        

Output Data Code (y)

This code stores the food insecurity score for each region on a 0–100 scale.

# Output data: [food_insecurity_score (0-100 scale)]
y = np.array([
    [45],  # Region A
    [30],  # Region B
    [75],  # Region C
    [20],  # Region D
    [85],  # Region E
    [60]   # Region F
])
        

Now that we have the data, we'll train a machine learning model to predict which areas are most at risk of food crises in the upcoming season. This is called a supervised regression problem, where we have inputs and outputs, and the goal is to predict a continuous output value (the food insecurity score).

In this case, we will use an artificial neural network (ANN), which is loosely based on how the neurons in the human brain work. ANNs have been particularly successful recently in solving complex problems.

Before we feed our data into the model, we need to account for differences in the units of our data. One input is rainfall in millimeters, another is crop yield in tons per hectare, and the output is a food security score. Neural networks are smart but not smart enough to automatically handle such differences. It's like asking the model to compare cats to dogs—it needs standardized units for accurate comparisons. To solve this, we scale the data to a 0–1 range:

Mathematical Representation of Scaling

To scale the data, we divide each value by its respective maximum value. The scaling formulas are:

• Scaled Rainfall = Rainfall / Max Rainfall
• Scaled Crop Yield = Crop Yield / Max Crop Yield
• Scaled Food Insecurity Score = Food Insecurity Score / Max Food Insecurity Score

| Region | RF (mm) | CY (tons/ha) | FIS (0-100) | Scaled RF | Scaled CY | Scaled FIS |

__________________________________________________________________________________________

| Region A | 350 | 2.1 | 45 | 0.603 | 0.600 | 0.529 |

| Region B | 420 | 2.8 | 30 | 0.724 | 0.800 | 0.353 |

| Region C | 210 | 1.2 | 75 | 0.362 | 0.343 | 0.882 |

| Region D | 580 | 3.5 | 20 | 1.000 | 1.000 | 0.235 |

| Region E | 120 | 0.9 | 85 | 0.207 | 0.257 | 1.000 |

| Region F | 290 | 1.7 | 60 | 0.500 | 0.486 | 0.706 |

Elaborations:

• RF: Rainfall (in millimeters)
• CY: Crop Yield (in tons per hectare)
• FIS: Food Insecurity Score (0–100 scale)
• Scaled RF: Scaled Rainfall = Rainfall / Max Rainfall
• Scaled CY: Scaled Crop Yield = Crop Yield / Max Crop Yield
• Scaled FIS: Scaled Food Insecurity Score = Food Insecurity Score / Max Food Insecurity Score

Python Code for Scaling the Data:

# Find max values for scaling
max_X = np.max(X, axis=0)  # Max of each column (rainfall, crop yield)
max_y = np.max(y)  # Max of output (insecurity)

# Scale the data efficiently
X_scaled = X / max_X
y_scaled = y / max_y

        

These networks are often called deep neural networks or deep learning when they involve many layers.

In neural network visuals, circles represent neurons, and lines represent synapses (connections). Synapses multiply the input value by a weight and pass it to the neuron. Neurons add up the values from synapses and apply an activation function. The sigmoid activation function is often used in this type of model, as it helps the network model nonlinear relationships.

Our network will have weights connecting each input to each hidden unit and weights connecting each hidden unit to the output. These weights can be represented as matrices:

Our network has two inputs(rainfall and crop yield), three hidden units, and one output(he food insecurity score). These are examples of hyperparameters. Hyperparameters are constants that define the structure and behavior of our network but are not updated during training. Our learning algorithm cannot decide that it needs another hidden unit. This is something we must predefine before training.

Python Code Step by Step:

To ensure reproducibility, we set a random seed using:

np.random.seed(42)
        

Next, we define the hyperparameters based on our network’s structure:

inputLayerSize = 2     # Number of input neurons
hiddenLayerSize = 3    # Number of hidden neurons
outputLayerSize = 1    # Number of output neurons        

Now, we initialize the weights and biases. Neural networks learn the parameters, specifically the weights on the synapses. We initialize weights randomly using small values to prevent symmetry issues during training:

W1 = np.random.randn(inputLayerSize, hiddenLayerSize) * 0.01  # Shape (2,3)
W2 = np.random.randn(hiddenLayerSize, outputLayerSize) * 0.01  # Shape (3,1)        

We also include bias terms, which help the network adjust outputs independently of input values. We initialize them as zeros since they will be updated during training:

b1 = np.zeros((1, hiddenLayerSize))  # Shape (1,3) - Bias for hidden layer
b2 = np.zeros((1, outputLayerSize))  # Shape (1,1) - Bias for output layer        

Explanation:

• X: Input matrix with 2 set of inputs from the dataset (2×2 matrix)
• W1: Weights matrix from the input layer to the hidden layer (2 inputs × 3 hidden units).
• W2: Weights matrix from the hidden layer to the output layer (3 hidden units × 1 output).
• b1 and b2 are bias terms added to the hidden and output layers respectively.

In this part by normalizing data and constructing a simple neural network, we’ve laid the groundwork for predicting food crises in vulnerable regions.

In the next part, we’ll take this further by implementing activation functions ,forward propagation, backpropagation, and training our neural network using gradient descent. We’ll also explore how to evaluate model performance and improve accuracy.

Why this neural network example? Because in the realm of humanitarian aid, predictive models can be lifelines. Imagine using these networks to forecast food insecurity, optimize disaster response, or efficiently allocate limited resources in crisis-stricken regions. This isn't just technology, it's a potential beacon of hope for communities facing unprecedented challenges.

Stay tuned for Part 2, where we'll dive deeper into the world of Neural Networks!

Feel free to share your thoughts, questions, or insights in the comments! Let’s learn together!

If you found this helpful, don’t hesitate to share it with others who might benefit!