Mathematical Foundations

Mathematics is more than just a set of tools—it is the very language through which we articulate the abstract concepts that power machine learning and artificial intelligence. Long before neural networks and optimization algorithms, thinkers like Euclid, Newton, and Leibniz wrestled with foundational questions of numbers and change. Their discoveries laid the groundwork for what we now recognize as the essential mathematical structures that underpin AI. When we discuss creating models like LLMs, we’re essentially talking about encoding the intricacies of human language, thought, and reasoning into patterns that machines can understand, manipulate, and generate. But before we can reach that level of complexity, we must first build a strong mathematical foundation.

In this article, we start from first principles—axioms and fundamental definitions of arithmetic and algebra—and work our way up to the calculus that drives optimization algorithms. We’ll see how each mathematical concept serves as a vital stepping stone toward creating intelligent systems capable of understanding and generating human language.

As we navigate the mathematical landscape, it’s important to remember that every concept and result we encounter is built upon the fundamental axioms we accept at the outset. These axioms form the bedrock from which all mathematical truths are derived. While we’ve chosen to deep dive into proofs selectively to keep our journey concise, rest assured that every theorem, rule, and formula we use is rooted in these first principles, ensuring a solid and rigorous foundation for the advanced topics we explore.

Mathematics, in many ways, is like constructing a great cathedral. At its foundation are the axioms—the first principles that are laid down like stones, sturdy and unyielding. Upon them rise the walls of theorems and proofs, each layer carefully placed and interconnected to ensure stability. With each brick of logic we stack, we ascend higher, crafting arches of equations and pillars of reasoning that reach toward the sky of abstract truth. And though our journey may take us through intricate halls and winding staircases of complexity, every corner turned reveals yet another elegant piece of the grand design, grounded in those original stones at the base. It is this architectural precision that allows the mathematical edifice to stand the test of time, beautiful and unshakable.

Axioms and Definitions

The Axioms of Arithmetic and Algebra

Mathematics is often like playing a game, but before you start, you need to agree on the rules. These rules, in mathematics, are called axioms. An axiom, in its purest form, is a statement so fundamental that we accept it without proof—just as in chess, the movement of each piece is defined and accepted. Axioms form the foundation of mathematical thought, guiding everything we build upon them. For example, the axiom of addition states: for any two numbers a and b, there exists a unique number c such that a + b = c. This is a statement we accept as true and use to derive all other properties of addition.

The commutative property of addition, another axiom, tells us that a + b = b + a, and although this seems simple, it forms the bedrock of many operations we perform in machine learning, such as summing weights in a neural network. From these simple axioms, we derive the familiar rules of arithmetic that are fundamental to everything from basic algebra to advanced optimization.

Functions and Graphs

Moving beyond basic arithmetic, the concept of functions becomes central—rules that assign to each input exactly one output. Functions are the workhorses of mathematics, providing a way to model relationships between variables. Imagine functions as mathematical machines that take in some input and spit out an output. For instance, in machine learning, we use functions to represent the relationship between input data and predicted outcomes.

Consider the linear function f(x) = mx + b. Here, m represents the slope, which tells us how steeply the function rises or falls, and b represents the y-intercept, where the function crosses the y-axis. This simple linear function can be visualized as a straight line on a graph. In machine learning, such a function could represent a linear classifier—a model that separates data points into categories based on a linear decision boundary. We normally use the notation

to explain how the function f takes an element x from R and maps it to another element y=mx+b of R.

As you begin to visualize these relationships, you'll see that functions—much like gears in a machine—are crucial when moving on to more complex models. For example, we can combine functions to form new ones, a process known as function composition. This is similar to feeding the output of one machine into another. Mathematically, if we have two functions, f(x) and g(x) , the composition is written as

which means first applying f to x , then applying g to the result. This layered approach of combining functions is a powerful tool, especially in machine learning, where complex models are often built by composing many simple functions together. Neural networks, for instance, are composed of layers of functions, each transforming the input data in a specific way to generate predictions or classifications.

The Heart of Optimization

At the core of algebra is the process of solving equations, which is essentially the process of finding values that satisfy a given relationship. This is crucial in machine learning, where we often want to find the set of parameters (weights) that minimize a cost function—essentially solving the equation that defines the optimal state of the model.

Consider a simple quadratic equation ax^2 + bx + c = 0. The solutions to this equation give us the points where the graph of the function crosses the x-axis. In machine learning, quadratic equations often arise in the context of loss functions. For example, in linear regression, the loss function is quadratic, and solving it gives us the best-fit line for our data.

In neural networks, the equations become more complex, but the principle remains the same: we are always searching for the set of parameters that satisfy an optimal relationship between the model and the data. Just as Newton once searched for the fundamental rules governing the heavens, machine learning searches for the "rules" that govern relationships in data. Mastering the techniques of algebra—factoring, expanding, and solving equations—gives us the tools we need to tackle these challenges.

Mathematical Spaces

At the heart of modern linear algebra and machine learning lies the concept of tensors, mathematical entities that generalize vectors and matrices to higher dimensions. Tensors can be seen as the most flexible structures for storing and manipulating data in multi-dimensional spaces. They encapsulate scalars (rank-0 tensors), vectors (rank-1 tensors), matrices (rank-2 tensors), and beyond, allowing us to explore transformations, relationships, and interactions in both physical and abstract spaces. Imagine tensors as the scaffolding of modern machine learning.

A tensor is defined by its rank, which refers to the number of dimensions or indices required to specify each element of the tensor. For instance, a scalar is simply a number, requiring no indices for its definition—this is a rank-0 tensor. A vector is a rank-1 tensor that requires a single index to specify each of its components. A matrix, meanwhile, is a rank-2 tensor that is characterized by two indices, representing rows and columns. As we ascend into higher ranks, we encounter tensors that may require three, four, or even more indices, providing the mathematical language to describe systems as intricate as the curvature of space-time or the weights in a deep neural network.

In machine learning, tensors are ubiquitous. Data itself is often represented as a tensor, with rows, columns, and even additional axes representing complex features, sequences, or time steps. More fundamentally, the operations that transform these tensors—such as matrix multiplications in neural networks—are essential to the functioning of modern algorithms. Understanding tensors and their operations is key to building sophisticated machine learning models, as it allows us to model relationships in high-dimensional spaces with elegance and precision.

The Axioms of Vector Spaces

Linear algebra begins with the concept of vector spaces, which are collections of vectors that can be scaled and added together according to specific rules. A vector space is defined by a set of axioms, including the existence of a zero vector and the commutative and associative properties of vector addition. These axioms provide the foundation for all the operations we perform in linear algebra.

A vector, as a rank-1 tensor, can be thought of as a point in space, with each element of the vector representing a coordinate along a particular axis. For example, a 2-dimensional vector v

represents a point in the xy-plane. Importantly, vectors are not confined to two or three dimensions; we can also have vectors in 3000-dimensional spaces, or higher. The vector space that \mathbfv belongs to is defined by all possible vectors of the same dimensionality, following the rules set by the axioms. The notation R^n is used to denote n-dimensional vector spaces, for example points in the xy-plane are 2-dimensional vectors from R^2.

Matrix Operations: The Language of Transformation

Matrices, which can be viewed as rank-2 tensors, are arrays of numbers that represent linear transformations—operations that map vectors from one space to another. For example, a 2x2 matrix can represent a transformation that rotates or scales vectors in the xy-plane. The power of matrices lies in their ability to encode complex transformations as simple multiplication operations, distilling intricate processes into a concise mathematical form. An example of this would be the matrix multiplication Ax_i = x_o,

where A is a matrix, x_i is a vector of inputs, and x_o is the resulting vector of outputs.

Matrix multiplication, one of the most important operations in linear algebra, follows a set of rules derived from the axioms of vector spaces. To make this concrete, consider the following example:

In this case, the matrix A applies a combination of scaling and shearing to the vector x_i , transforming the point (1, 2) into the new point (4, 6) in the 2D plane. This is a simple example of how matrix multiplication can be interpreted as a transformation that changes the geometric properties of a vector. The general rule of matrix multiplication is to sum the products of corresponding entries from the rows of the matrix and the columns of the vector (or another matrix) for each resulting element.

In the context of machine learning, matrix multiplication plays a critical role in neural networks, where each layer transforms the input data by multiplying it with a matrix of weights - one could even argue that our current AI systems are fundamentally just a series of matrix multiplications with some other minor functions in between. These weight matrices adjust the input data as it moves through the network, allowing the model to leverage learned patterns and make predictions. From a high-level perspective, matrix multiplication can be interpreted as a way of applying linear transformations to vectors or data points, encoding complex operations such as rotations, scalings, and projections into a single mathematical operation. The sheer importance of matrix multiplication in machine learning has driven the development of specialized hardware, such as GPUs we encountered previously in the series, to efficiently handle the vast number of operations required at scale.

As we expand beyond matrices, into higher-rank tensors, the operations grow increasingly sophisticated, allowing us to model even more complex relationships in high-dimensional spaces. Yet, whether working with simple vectors, two-dimensional matrices, or higher-order tensors, the underlying principles of linear algebra remain consistent and powerful, offering a language through which the mysteries of both mathematics and machine learning can be explored and understood. In addition to multiplication, operations such as transposing—which involves flipping a matrix A along its diagonal to become A^T—are essential, as they enable us to manipulate and transform matrices in different ways, opening up further possibilities for how data is structured and processed.

Imagine matrices as the recipe for a transformation, where each vector is an ingredient waiting to be altered. Just as a recipe provides detailed instructions for mixing, scaling, or modifying ingredients to create a final dish, matrices apply precise mathematical operations to vectors. These operations combine individual elements, rotate or stretch them, and blend them together, producing a new outcome that retains the essence of the original ingredients but presents them in a completely different form. The matrix multiplication process is like following a step-by-step recipe: ingredients are transformed, combined, and balanced to create something new, whether it's a scaling of the input or a complex reshaping of data.

In machine learning, this analogy is especially fitting. Each layer of a neural network acts as a recipe stage, with matrices of weights adjusting and transforming the data as it flows through the model. These weight matrices, like a series of precise instructions, refine and enhance the input data, allowing the network to "cook up" accurate predictions or insights. Just as a chef refines a dish to perfection, neural networks adjust their matrices during training, finding the right balance of transformations that reveal hidden patterns and produce optimal results.

To move forward, we must not only understand how data points relate to one another in vast spaces, but also how machines can learn to to find patterns in data. The current generation of AI systems is mostly dependent on starting with a random state and slowly iterating towards an optimal state. The AI systems doing these small iterations are guided by how small changes in the parameters or weights can ripple through these systems, affecting their performance.

This is where calculus becomes indispensable. At its core, calculus is the study of change, and in the context of AI, understanding how models react to changes in their inputs or parameters is essential for optimization. When training a model, we continuously adjust parameters—such as the weights in a neural network—based on how the model's output changes in response to new data. To do this effectively, we need a mathematical framework that helps us quantify these changes, guiding us toward improvement.

Analyzing Change: Calculus

What drives AI models to learn and improve? How do we measure the impact of changes in data on a model's output? How can we find the best parameters to optimize performance? These questions lead us to calculus, the mathematical study of change. Through derivatives, we can understand how to adjust models for better accuracy and efficiency.

The Concept of the Derivative

At the heart of calculus is the derivative, which measures how a function changes as its input changes. In simple terms, the derivative of a function at a point gives us the slope of the tangent line to the function's graph at that point. This slope provides a snapshot of how steeply the function is rising or falling at any particular moment, much like checking the gradient of a hill to determine how steep it is.

The derivative is formally defined as:

This limit definition encapsulates the idea of instantaneous rate of change—how much f(x) changes with a very small change in x. The derivative serves as the key tool in optimization, as it tells us how to adjust the inputs to minimize or maximize the function's output.

In machine learning, derivatives help us determine how a model's output changes with respect to small adjustments in its parameters, guiding us in the training process. For instance, in a neural network, derivatives are used to calculate the gradients, which then tell us how to modify the weights in order to reduce the error.

There are two common notations used to express derivatives: f'(x) and df/dx . Both represent the same concept, though they emphasize different perspectives. The notation f'(x) is more compact and often used when discussing the derivative of a specific function, such as f(x) . On the other hand, df/dx explicitly highlights the operation being performed: the rate of change of f with respect to x . This latter form is especially useful when we are dealing with multiple variables and want to make clear which variable the function is being differentiated with respect to.

To visualize the derivative, imagine driving a car along a winding road. As you drive, your speedometer shows how fast you're going at any given moment. This speed is your derivative—it's the rate of change of your position over time. If you're driving on a straight, flat highway and the speedometer reads a constant value, this corresponds to a linear function with a constant derivative. The road is smooth, and you're traveling at a steady pace.

But now imagine you're driving up a hill. As you ascend, your speed starts to decrease—the steeper the hill, the slower you go. The derivative, in this case, reflects the slope of the hill. A steep incline means a large negative derivative, indicating that your speed is rapidly decreasing. On the other hand, when you come down the hill, your speed increases as gravity helps you along, and the derivative becomes positive again, reflecting the increasing rate of change in your position.

The same principle applies to functions in calculus. The derivative tells us how the function is changing at each point, whether it's increasing, decreasing, or staying constant. Just as the speedometer gives you instant feedback on your speed, the derivative provides instant feedback on how the function is behaving at any given value of x.

Handling Multivariable Functions

In many real-world problems, the functions we work with depend on multiple variables. In such cases, we use partial derivatives to measure how the function changes with respect to each variable independently. The partial derivative of a function f(x, y) with respect to x is defined as:

Partial derivatives are the building blocks of gradients in multivariable calculus. When training a machine learning model, we compute the gradient of the loss function with respect to each parameter (weight). This gradient is a vector of partial derivatives, each one telling us how the loss changes as we tweak a specific parameter.

For example, in a neural network with many weights, each partial derivative tells us how changing one weight will affect the overall loss. By combining these partial derivatives, we can form a gradient vector that guides the entire optimization process.

The gradient is a vector that points in the direction of the steepest ascent of a function. In mathematical terms, the gradient of a function

with respect to its variables is the vector of partial derivatives:

The gradient tells us how much the function f will increase if we move in the direction of each variable. This concept is crucial in machine learning, where we often want to minimize a loss function by adjusting the model's parameters. By computing the gradient of the loss function with respect to the parameters, we can determine the direction in which to adjust them to reduce the loss.

The gradient can be thought of as the guide in a vast, multidimensional landscape. Imagine you're hiking in a mountain range, not just in the familiar three dimensions of space, but in a surreal world with many dimensions. Each step you take represents a small adjustment to one of the variables, like moving up or down on one of the axes. The gradient is like a trail marker at every point, showing you the direction of the steepest path up the mountain.

If you're trying to climb to the peak, the gradient tells you exactly where to step to gain the most elevation. Conversely, if you're trying to descend into a valley (which is what we do when minimizing a loss function), the gradient points you down the steepest slope. In this multidimensional world, where each axis represents a different parameter of your model, the gradient acts as your compass, guiding you through the complex terrain toward the optimal solution. Without this guidance, you’d be wandering aimlessly, unsure of whether each step takes you closer or farther from your goal.

The Chain Rule: Propagating Changes Through a System

One of the most powerful tools in calculus is the chain rule, which allows us to compute the derivative of a composition of functions. In mathematical terms, if y = f(g(x)), then the derivative of y with respect to x is given by:

This rule is crucial in machine learning, where models are often composed of multiple layers of functions. For example, as we will later see, in a neural network, each layer applies a function to the data, and the chain rule allows us to propagate the gradients backward through the network during training.

Imagine a neural network as a series of interconnected gears. When we adjust one gear, the chain rule tells us how each connected gear will move in response. This understanding is essential for fine-tuning the network to improve performance.

Convergence

Distances and the Curse of Dimensionality

In the world of machine learning, where data is often represented as points in multi-dimensional spaces, understanding the concept of distance is crucial. At first glance, distance may seem straightforward—a simple measure of how far apart two points are, much like measuring the length between two cities on a map. But as we venture into higher dimensions, where data is abstract and complex, the notion of distance takes on a new and far more intricate meaning.

Formally, a distance, or a metric, is a function that takes two points from a space, such as the familiar R^n—the set of all vectors in n-dimensional space—and returns a real number that quantifies how far apart these points are. This function, which we denote as

, must satisfy several intuitive properties. First, the distance between any two points must always be non-negative and the only time this distance is zero is when the two points are identical. Furthermore, the distance between two points must be symmetric. Finally, this distance should satisfy the triangle inequality, which ensures that traveling directly between two points is never longer than taking a detour through a third point.

The most familiar of these metrics is the Euclidean distance, the straight-line distance between two points in space. For two points x and y in n-dimensional space, the Euclidean distance is given by:

This is the mathematical embodiment of what we experience in everyday life: the shortest distance between two points is a straight line. It works beautifully in two or three dimensions, where we can visualize the space as a flat plane or a three-dimensional landscape. But as we begin to explore higher dimensions, the nature of distance begins to shift in ways that challenge our intuition.

Imagine yourself standing on the surface of a small, manageable field. The space around you is familiar, with recognizable landmarks—the trees, hills, and pathways all behave as expected. Now imagine that as you start walking, the field expands. The landmarks grow farther apart, and before long, what was once a short walk now feels like an endless journey. This is the effect of moving into higher-dimensional spaces. As the number of dimensions increases, the space becomes vast and sparse, and everything seems to spread out.

In high-dimensional spaces, a strange paradox emerges: as the number of dimensions grows, the distances between all points tend to converge, and everything starts to feel uniformly far away. This is the curse of dimensionality. In low-dimensional spaces, points can be close together or far apart, and the distances between them are meaningful. But in spaces with hundreds or thousands of dimensions, the points are no longer distinct in the same way. They seem to drift into a kind of sameness, where the very idea of "closeness" begins to lose its meaning.

This presents a serious challenge in machine learning. Many algorithms rely on the notion of distance to make decisions—whether it's clustering similar data points or classifying new observations based on their proximity to known examples. But as the dimensions increase, these distances become unreliable. Features that once helped to distinguish between different points now contribute to the overall complexity of the space, making it difficult to tell what is truly important.

The curse of dimensionality can be imagined as a quiet, creeping force—one that turns a rich, densely populated city into a sprawling desert. The landmarks that once guided us are now scattered across a vast, seemingly infinite landscape. For large-scale AI systems, this curse manifests itself in subtle but powerful ways. High-dimensional data, whether in the form of text embeddings or complex parameter spaces in models, can become a kind of wilderness, where the distances between meaningful data points blur into a haze.

Yet, understanding this curse is the first step in overcoming it. Just as explorers must learn to navigate the deserts by finding new ways to measure their surroundings, AI researchers and engineers develop tools and techniques to cope with high-dimensional spaces. Recognizing the challenge of distance in these vast spaces leads to innovations in how we process, model, and interpret data at scale. And in this way, the curse of dimensionality, though daunting, becomes a key to unlocking deeper understanding in the pursuit of intelligent systems.

But this is only part of the journey towards building intelligent systems.

Are we there yet?

In any optimization algorithm, particularly those used in machine learning, convergence is a key concern. Convergence refers to the process by which an algorithm iteratively approaches a solution, ideally reaching the minimum of the loss function as closely as possible. The speed and reliability of this convergence are crucial for training effective models.

For gradient-based algorithms like Stochastic Gradient Descent, convergence is influenced by several factors, including the learning rate, the shape of the loss function, and the presence of noise in the gradient estimates. Understanding the mathematical principles that govern convergence allows us to design algorithms that are both efficient and stable.

One of the most important results in this area is the convergence theorem for gradient descent, which states that under certain conditions, the gradient descent algorithm will converge to a global minimum. This result provides a theoretical guarantee that the algorithm will find the best possible solution, given enough time. Importantly though, this result applies when the given conditions are there. We will study this topic in more detail in this section.

Much like a traveler navigating toward a distant destination, an optimization algorithm is always asking, "Are we there yet?" The journey is rarely straightforward. Some roads are smooth, while others are winding, full of detours and obstacles. Convergence, in this sense, is the moment when the traveler arrives at their destination—the minimum of the loss function—after a long and often unpredictable journey. But, just as with any voyage, the traveler must be mindful of the path taken. A poorly chosen route may lead them astray or cause them to arrive much later than expected, while a well-planned course ensures a quicker, more efficient arrival. In optimization, as in life, the art of the journey lies in knowing which steps to take, and in understanding that even the smallest misstep can lengthen the road ahead.

Global and Local Minima

Imagine you're hiking in a mountainous region, searching for the lowest point in the landscape. A global minimum is like the deepest valley in the entire region, while a local minimum is a smaller, nearby dip that may feel like the lowest point but is not. If you're not careful, you might settle in a local minimum, unaware that a deeper valley exists beyond the next ridge.

Convexity is a property of functions that plays a critical role in optimization. A function is convex if, for any two points on its graph, the line segment connecting them lies above the graph. This property ensures that any local minimum of the function is also a global minimum, which simplifies the optimization process.

In optimization, a global minimum is the lowest point across the entire function, while a local minimum is the lowest point within a small, restricted region. For convex functions, these two concepts are identical—the function's structure guarantees that finding a local minimum automatically leads to the global minimum.

In machine learning, convex functions are often encountered in the context of loss functions, particularly in simpler models like linear regression or logistic regression. For these models, the convexity of the loss function guarantees that gradient descent will converge to the global minimum, provided the learning rate is appropriately chosen.

However, many of the loss functions used in deep learning, including those for transformers, are not convex. This lack of convexity introduces challenges for optimization, as the algorithm may get stuck in local minima or saddle points—points where the slope is flat but it's neither a maximum nor a minimum. Without convexity, the optimization process becomes more difficult, as local minima may appear to be solutions, even though better solutions (global minima) exist elsewhere in the search space. Understanding the implications of convexity (or the lack thereof) allows us to better navigate these challenges and develop more robust optimization strategies.

To visualize convexity, imagine hiking through a valley surrounded by mountains. A convex valley represents a smooth, bowl-shaped curve, where no matter which direction you walk down the slopes, you'll always reach the lowest point—the global minimum—at the center of the valley. The entire landscape guides you downward with no false peaks or traps. On the other hand, a non-convex landscape is like traversing rugged, mountainous terrain. There are multiple valleys and peaks, and you might find yourself stuck in a small depression—a local minimum—thinking you've reached the lowest point, only to discover that deeper valleys exist elsewhere. Convexity, in this sense, is the promise of an easy descent to the bottom, while non-convexity presents a more complex challenge, where the path to the optimal solution may not be as clear or direct.

We Need a Calculator

At this point in our journey, we've constructed a solid mathematical foundation—beginning with the axioms of arithmetic and algebra, progressing through calculus, and ultimately mastering the linear algebra that underpins modern machine learning. Throughout, we've explored how these mathematical ideas are not mere abstractions but powerful tools that help us build, understand, and optimize artificial intelligence systems.

Yet, as the complexity of these calculations grows, performing them by hand becomes impractical. To tackle this, we turn to the machine itself. But this raises an important question: how exactly do we perform mathematics on a computer?

Math on Computers

Numerical Precision

In the world of computers, everything boils down to numbers. Yet, unlike the idealized numbers we work with in theoretical mathematics, the numbers that live inside a computer are constrained. They are stored in a format that allows only a finite amount of precision—think of them as grains of sand in an hourglass. There’s only so much space to hold them, and inevitably, some grains will slip through the cracks.

This constraint is rooted in the very architecture of computers we saw in "Building A Computer" with data represented in binary form, the simplest numerical system composed of just two digits: 0s and 1s. Every calculation performed by a machine, whether it's predicting the next word in a sentence or solving complex differential equations, is ultimately grounded in this binary arithmetic. However, representing real numbers—like 3.14159 or 0.000025—using only a limited sequence of 0s and 1s introduces an inherent limitation: finite precision.

This inherent limitation—finite precision—can be understood with an analogy to painting a detailed landscape on a canvas that’s too small to capture every nuance. Imagine you’re an artist trying to paint a vast, beautiful scene: towering mountains, shimmering lakes, and intricate clouds, all with the finest details. But instead of a large canvas, you’re given a tiny postcard to work with. No matter how skilled you are, there’s only so much detail you can include. Some of the subtleties—like the texture of the leaves on distant trees or the delicate ripples in the water—will inevitably be lost or simplified.

In a similar way, when a computer tries to represent real numbers using binary code, it has a limited number of bits to work with, akin to the small canvas. While it can approximate many values with impressive accuracy, the finer details of certain numbers—especially those with long decimal expansions like Pi or 0.000025—are cut off or rounded, much like how the details of the landscape must be simplified on the postcard. This is what we mean by finite precision: the computer can only store and compute numbers up to a certain degree of accuracy, with some of the finer information lost in translation.

Finite precision means that the real, continuous world we try to model must be squeezed into the discrete, finite framework of a computer. This squeezing process introduces approximations, and with approximations come errors. While these errors may seem small, in the context of large-scale computations—like training AI systems with billions of parameters—they can accumulate and lead to significant issues.

Numerical mathematics, therefore, is the science of understanding these approximations, managing errors, and ensuring that our computational methods remain robust and reliable. It is the glue that holds the theoretical foundations of mathematics and the practical needs of machine learning together. As we explore this essential discipline, we'll discover how to navigate the challenges of finite precision and why it is so crucial in the pursuit of building powerful AI systems, such as large language models.

We will briefly review how numbers are converted into binary units that computers can store and process. To convert a decimal number to binary, we express it in terms of powers of 2, the basis of the binary system. For example, consider converting the decimal number 13 to binary. We first identify the largest power of 2 less than or equal to 13, which is 2^3 = 8. Subtracting 8 from 13 leaves 5. The next largest power of 2 is 2^2 = 4, and subtracting 4 from 5 leaves 1. Lastly, 2^0 = 1 fits exactly, resulting in zero. By placing a '1' for each power of 2 used (and '0' for those skipped), the binary representation is obtained: 13_10 = 1101_2. This binary representation, composed of 0s and 1s, directly maps onto the physical states of transistors within a CPU's Arithmetic Logic Unit (ALU), where each bit is stored as an electrical signal, either high (1) or low (0), enabling the ALU to perform essential operations that drive all computational tasks in a computer system.

The conversion to binary numbers we saw for the number 13 sadly becomes more complicated when trying to convert other numbers such as 1/3. To handle such cases, computers store numbers using floating-point arithmetic, a method that resembles scientific notation. Just as we might write the number 4,500 as 4.5 \times 10^3, computers represent numbers as a combination of a base or mantissa (in this case 4.5) and an exponent 3. However, instead of using base 10, computers use base 2.

For example, the number 5 in a floating-point system might be stored as something like 1.25 x 2^2. We would save 1.25 as the mantissa and the exponent 2 in two binary numbers close to each other so we could reconstruct the full number whenever needed. This compact representation allows computers to handle a vast range of numbers, from the extremely large to the vanishingly small.

Rounding Errors

Rounding errors are the inevitable consequence of working with finite precision. When a number cannot be represented exactly in floating-point form, the computer rounds it to the nearest representable value. This rounding process introduces a small error, often referred to as a "round-off error."

To visualize this, imagine you’re walking along a path but can only take steps of a fixed length. If the distance between where you stand and your destination isn't a perfect multiple of your step length, you'll either fall short or overshoot with each step. In numerical calculations, each “step” corresponds to a floating-point operation, and the slight deviations accumulate over time.

In the context of AI and machine learning, these rounding errors can manifest in subtle yet impactful ways. For example, when summing a large series of numbers—a common operation when calculating loss functions or model predictions—rounding errors can lead to a cumulative drvaift from the true sum. This drift might seem insignificant at first, but in a deep neural network with millions or billions of operations, it can lead to noticeable degradation in model performance.

For example, if we have a floating-point number with a mantissa that allows for seven digits, then a number like 123.4567 is considered to be precise to those seven digits. If on the other hand we try to represent a number like 123.456789 with the same precision, it would be rounded to fit within the available significant digits, leading to a loss of information.

Numerical Stability

Error Propagation

When a small error is introduced into a calculation, it can propagate through subsequent operations, often growing larger with each step. This phenomenon is known as error propagation, and it’s a critical consideration in numerical mathematics.

Imagine you’re constructing a tower of blocks. If the first block is slightly off-center, each subsequent block you place will exacerbate the tilt, leading to an unstable structure. Similarly, in numerical computations, an initial rounding error or imprecise input can lead to progressively larger inaccuracies in the final result.

In AI systems, error propagation can be particularly problematic during iterative processes, such as training neural networks. Each iteration of training involves multiple calculations—adjusting weights, calculating gradients, updating parameters—and any small error in these calculations can compound over time. This is why careful attention to numerical stability is essential for ensuring that the model trains effectively and produces reliable outcomes.

Stability of Numerical Algorithms

Numerical stability refers to an algorithm's ability to produce accurate results despite the presence of small errors, such as those caused by rounding. A numerically stable algorithm ensures that these small errors do not grow uncontrollably, leading to inaccurate or unusable results.

Think of a numerically stable algorithm as a well-designed ship that can navigate through rough seas without capsizing. Even in the presence of turbulent waters (errors), it stays on course, ensuring that the final destination (the result) is reached accurately.

Precision Issues in Machine Learning

In machine learning, precision is not merely a technical detail—it’s the bedrock upon which reliable models are built. Every calculation, from adjusting weights to computing gradients, depends on a fine balance of accuracy. However, machine learning operates in a world of finite precision, where floating-point numbers—those incredibly small, fractional approximations of real values—introduce subtle, yet significant challenges.

Imagine building a large, intricate mosaic out of tiny, imperfectly cut tiles. Individually, each tile's flaw might seem trivial, but as you place more and more of them together, these imperfections compound, potentially distorting the overall image. Similarly, in AI, the limitations of floating-point arithmetic—rounding errors, truncation errors, and finite representation of numbers—can accumulate as models undergo iterative processes like training. Each step in the training process, which relies on recalculating gradients and adjusting parameters, risks magnifying these small inaccuracies.

One of the most glaring consequences of this phenomenon is the problem of vanishing and exploding gradients, particularly in deep neural networks. In an ideal world, gradients—those signals that guide the learning process—should accurately reflect how much a model’s parameters need to change to improve. But when finite precision disrupts this, gradients either become too small (vanishing) or too large (exploding). For examples, when a gradient vanishes it becomes so small that the finite numerical representation of the gradient gets rounded to 0, causing training to slow to a crawl or veer wildly off course. Imagine trying to climb a hill blindfolded: if your sense of direction (the gradient) is too faint, you might barely move; if it’s too forceful, you risk overshooting your destination entirely.

Engineers have devised several strategies to counteract these precision pitfalls. One straightforward approach is to use higher precision arithmetic, opting for 64-bit floats instead of the more common 32-bit, thereby reducing the risk of rounding errors. However, this solution is not without its trade-offs: increased precision demands more computational power and memory, making it impractical for many large-scale applications. Think of it as choosing between flying a small, nimble plane with limited fuel (low precision) or a bulky, fuel-guzzling aircraft (high precision)—both have their merits depending on the journey’s demands.

Other strategies focus on the structure of the model itself. Proper initialization techniques, like Xavier or He initialization, help distribute the initial weights in a way that preserves the flow of information through the network, minimizing the risk of gradients spiraling out of control. Additionally, methods like gradient clipping, where the gradients are capped at a certain threshold, prevent them from reaching values that would destabilize the training process. Regularization techniques, such as L2 regularization, also play a vital role by keeping weights within a manageable range, ensuring that they don’t grow excessively large, which could further exacerbate precision errors.

In the grand scheme of AI development, precision might seem like a footnote, a technical quirk of floating-point numbers. Yet, as we push machine learning systems to their limits, understanding and addressing these precision challenges becomes essential. Like the foundation of a skyscraper, it is invisible to the naked eye but critical to the structure’s stability. In many ways, ensuring numerical stability is the quiet, behind-the-scenes work that makes the towering achievements of AI, like GPT, possible. Without it, the entire edifice risks crumbling under the weight of its own complexity.

We're Ready for Machine Learning

As we move from the intricate dance of numerical mathematics to the theoretical foundations of machine learning, it’s important to recognize how the concepts we've explored connect to the broader architecture of AI systems. The finite precision and numerical stability discussed here are the very principles that govern the behavior of the hardware components, from transistors to GPUs, when performing math. These components, in turn, form the bedrock of the computational frameworks from computer science, where hardware meets software in executing complex algorithms. Furthermore, mathematical foundations underpin the calculations and optimizations central to training machine learning models. Thus, the numerical methods we've examined are not just abstract considerations but are crucial to ensuring that the algorithms we design—whether for basic calculations or advanced machine learning—operate correctly and efficiently, setting the stage for understanding the theoretical principles of AI in the next section.