Is learning data structures crucial to implementing creative AI algorithms? Is it necessary to learn AI and ML?

As the world inches towards an AI-dominated tech future, we must get comfortable with the fundamental concepts of machine learning. Luckily, data structures are among these fundamental concepts.

A data structure is a way of organizing large amounts of information. They’re used in computer science to store and efficiently retrieve individual bits of data. But why should you care? If you’re interested in implementing creative AI algorithms, then it turns out that this skill might be crucial! It just so happens that AI algorithms need a ton of mathematical operations on data to produce their results. Without efficient data structures, you won’t be able to implement the algorithms necessary for creative AI.

So how exactly do these data structures work? Data structures are made up of variables. Each variable has a value, and this value can change over time. A simple example is to create a list of numbers called numbers. To add an element to the end of this list, we can perform operations like these:

numbers = [1,2,3] more = 4 numbers += more

The term += is used in an operation known as appending (or concatenating). This operation takes two lists (or two strings), and they are merged. When we append two lists, the first is expanded and then the items from the second list, in addition to their previous values, are added to all existing elements. The operation returns a new list composed of all of the old items plus the new variable (more).

Now that you’ve got a solid understanding of how appending works, let’s move on to more advanced operations to see what patterns we can discover. For this purpose, let’s introduce a simple operation called min-max-finding. This function gives two lists and returns the first list with only those elements with a minimum value. Let’s call our lists numbers and prices:

numbers = [10.0, 3.5, 4.87, 11.5] prices = [10.0, 9.5, 10.0, 20.5]

We can take advantage of the operation += to quickly implement min-max-finding. We can start with this script:

numbers = [10.0, 3.5, 4.87, 11.5] results = [] numbers += prices min_max_result = numbers min_max = min(results)

We can process the first list (numbers) by appending it with the prices list (min_max). And then we just pull out the minimum value from this new list! But what if we want to find the median of all these lists? How would we proceed if we did so? Well, let’s imagine that there are some extra variables in our data:

income_list = [10.0, 10.0] income_list += [20.0, 30.0, 40.0] median = income_list // returns [20.0]

We can add a new variable called results and use it to store the minimum value in our list of numbers (i.e., min(results)), and then we can return the median of this list (i.e., median(results)). As you can see, data structures provide a way to manage large amounts of information and perform operations on them in an efficient manner.

Let’s now introduce another interesting data structure called a binary tree. A binary tree is a way of efficiently storing information. Let’s imagine that we have a list of numbers and we want to store all the elements from 1 to 10. If we follow the rule of adding items to the left, it will be quite difficult because we need to find all the places where any number sits at the end of the list. What we need is a way to store the information efficiently. Binary trees are a perfect solution. We can represent our list of numbers as follows:

numbers = [1, 2, 3, 4, 5, 6, 7, 8]

This tree stores all the numbers from 1 to 10 in an efficient manner. We can consider any number as a node and we can also consider this tree as a data structure for storing information about nodes (e.g., each number), their values (e.g., 1–10), and the locations of other nodes from which they are children (e.g., 1–8).

The operation that you might be familiar with from arithmetic is known as adding two numbers together. In a binary tree, we can use this operation to add all of the numbers from 1 to 10 into one number. So the result would be 10. However, it’s not enough to add all of them together. We also need to replace any number that doesn’t exist in the list with the result of adding them together (e.g., 5+8=13) and we need to avoid having duplicates. When we use addition to add numbers, it’s already a binary operation. So we can make the following beta-reduction rule:

In other words, if we have an array of size n elements, and if one element of the array has a value b, then all elements of the array must have values that are formed by multiplying b(n - 1) by any integer n - 1. As a sum of products, this is equivalent to

To sum up, there are two main activities when using binary trees for storing information: for each element in the tree to give us its children (a list of nodes), and for us to add up all our children (i.e., each child = child(parent)).

Then, realizing that the addition operator is already a binary operation, we can use it to add all of our children. The result of this addition will be our parent value. In other words, when we have an array with n elements and one with m elements, we can find the sum of all the elements in that array by multiplying any starting number 1 by any integer n - 1 time (or adding them up). We can consider this operation to be an operation on a tree where each node gets its children calculated, and then these children get added together.

Once you have learned about binary trees, then you’re ready to explore advanced operations for working with them. The addition is an example of a total operation. To find out what other kinds of operations are available, let’s consider a few simple cases. The first operation we’re going to consider is how to remove an element from an array or a list. Suppose that you want to remove the smallest value in a list:

removed = results.pop() removed = min(results)

Let’s introduce a new variable called removed, and then we can use the pop function to remove the smallest number (or value) from our list (i.e., results). Like with our previous examples, we can save this number in the variable called removed. As you can see, the operation pop() simply removes the smallest element from a list or array and returns it to us. We can also remove an element in a different order, such as the second smallest or the fourth smallest. A more powerful operation is called count, which gives us a value that’s equal to 1 if the given item exists in our list.

removed = prices.count(10)

We can count the items in our list by using only one line of code. The count function gives us a value that’s equal to 1 if the item (or element) exists in our list and 0 otherwise. This operation is very useful when you have to process huge amounts of data and you want to find out if one or more elements exist in it. Another useful operation is called reverse. It’s used to make a list in the opposite order. For example, if we want to get a list of negative numbers in the opposite order:

numbers.reverse()

We can use reverse on an array or a list of numbers and it will give us the numbers in the opposite order (i.e., decreasing). If we say that our original list numbers = [3, 2, 4, 1], then after using reverse we will get numbers = [1, 2, 3, 4]. The last operation I want to introduce is called sort.

numbers.sort()

The sort function gives us an array or a list of sorted items. This operation is really useful when you want to create a list of numbers in a specific order. As you can see, the function sort gives us a sorted array or list, which is how we usually know it as.

It’s not just the steps of adding and removing elements, but also the order that matters when using binary trees. Therefore, we can say that we can use the same operation on different trees to get completely different results.

If we have an array of size n elements, then it would be possible for any integer value k to appear at any position in this array. But as soon as n = 2, then every possible position contains only 2 possibilities (i.e. 1 and k). However, as soon as n = 3, then again we can have 1, 2, or k at any position in the array. But what if n = 4 ? We would have 1, 2, 3, or k at every possible position in an array of size 4. What happens when n = 5 ? Can you see that this is not possible? The only place where we can place k at any position is between 1 and n - 1.

In other words, when n is equal to 2, there are only four possibilities: (1,2), (2,1), (1,2), and (2,...). These values correspond with the positions 1 to 4 in the array. Similarly, if n = 3 , then there are six possibilities: (1,2), (2,3), (3,1), and [....]. In this case, the values at these positions correspond to positions 1 to 6 in the array. These two represent what we call the Cartesian product. The only place where a value k is not possible is when n = 5. But then again, if n =6 then again there are only four possibilities: (1,2), (2,3), (3,4), and [....]. These values correspond with the positions 1 to 4 in the array. In the previous diagram, we can see the Cartesian product and one example of a tree with 4 positions. Let’s now move back to our discussion about binary trees.

Suppose that we have an array of size n elements, and suppose we have to multiply every value in this array by 2n - 1, including the element k that’s located at position n (the last element in the array). (But don’t forget that it has to be multiplied by 2n - 1, not just n .)

Now let’s go back to our discussion about the Cartesian product. We can see that the Cartesian product is not just one of the possible positions for k. It is possible for any position in an array, except the one that corresponds to k. So if we want to multiply every value in an array by 2n - 1, this means that we have to spread the values over all positions. The results will be collected in another array with size n + m where m is greater than n, and where m << n.

To understand it better, let’s consider another example. Let’s say that we have an array of numbers, and we also have a binary tree with 6 positions. Now let’s multiply every value in the array by 2n - 1.

Let’s see what happens in our tree:

So now we can see that the result is: [1, 2, 3, 4, 5, 6]. As you can see, this is only one possible result from multiplying an array of size 6 by 2n - 1. Now suppose we have a bigger array to multiply. Suppose that our array has 10 elements and it has to be multiplied by 2n - 1. So how many possible results would there be for this operation? Suppose the value k equals 5 . What would happen to this array if we do the following operation with 10?

Let 5 be the only position where we can put our value k (5). So there are only six possible results from multiplying an array of size 10 by 2n - 1:

The first three are conventional and are just numbers. So we have put the values 1, 2, and 3 inside an array. Now let’s look at the other three: what kind of values can these be? As you can see, if n = 4 , then there are only four possibilities for our tree: [1,2], [2,3], [3,4] and [4,...]. These values correspond with the positions 1 to 4 in the array. When we multiply the array by 2n - 1, then for every new value in our array we have to choose a new position for k. For example, if our value is 1, then we can place it on positions 2, 3, or 4. However, when n = 5, there are only three possible positions where our value k can be placed: [1,2], [2,3], and [3,...]. So now we have put the values of n = 2 and n =3 inside an array. And if you look at this third possibility again you will realize that these values correspond with the positions 1 to 3 in our array (numbers). So the next value can again be placed on positions 2, 3, or 4.

As a result, we get the following:

With this example, you can see that if our initial array has 10 elements, then there are a total of 120 = 1,024 possible results that we can get by multiplying it by 2n - 1. If we want to choose the actual result (or at least one of them), then what would be the best way to do this? We already know that binary trees and Cartesian products are used to organize data. In our example, we can place the numbers from 1 to 9 in an array. We also know that the result will be organized in another array according to their position in our initial array. This means that there are many possible results, and one of them will be chosen randomly.

How can we choose a random result? Well, we know that all possible results (1-6) have been used in our first array. But what about values from 7 to 9? Let’s continue with the previous example, where we have 10 elements to multiply by 2n - 1. The following table shows all possible results:

In this case, position 6 is empty (it has no value). So this position can be chosen randomly. Another thing we should consider is that all the results of multiplying 10 elements by 2n - 1 are regular numbers. So we can choose a random number from them. To find them, as you can see from the previous table we need to continue from left to right, with our value located at the position corresponding to n. In our case, this means choosing a random number from 1-2, 3-4, and so on up to our desired result (10). So we will have 10 choices for each n. We continue in the same way for all n.

This makes sense if you think about it. Suppose that you have five people who have to work together on a project. They will have 5 people on their team. The project consists of doing several tasks. And each task has a number (say 1-5) and they are to be completed by all five members of the team.

Each person has to participate in at least one task. This means that the number of people will be either 5, 6, or 7 at most. So for each team member, there are about 3 possibilities for any given n. We have already seen in the previous example that there are a total of six possible binary trees to put the numbers from 1-to 9 into an array. With this fact, we can choose a result from them randomly by choosing at most one out of 10 selections for each n.

Similarly, suppose that each one of our selected tasks is a binary tree. For example, the first task (1) is located at position 1 in our array. The second task (2) is located in positions 1 and 2 of our array. The third task (3) is located in positions 1, 2, and 3 of our array. And so on. Suppose we want to choose the result for some specific task number. Then we just need to choose one out of 10 selections for each n. Suppose that this particular task has the value k = 5. To do this, we have to find the element “5” on a random selection for every n. This means that we have to build a random selection for every n. And this number is either equal to k or equal to n - k. In our example, n is 5, and k = 5. This means that this particular task has the values 1, 2, 3, and 4. In other words, we have chosen the possible results: [1, 2, 3, 4] for every n.

So now by using what we have learned so far, let’s return to the main problem. Suppose we want to build a binary search tree over an array of integers. We want the result of this operation to be stored in another array (a Cartesian product of our two arrays). This means that we will organize the data according to their position in our first array. We need to design a way to do this.

We already know that some of the possible results of multiplying 10 elements by 2n - 1 are regular numbers or trees, while some of them are not (i.e. they don't correspond with all the elements in our initial array). So what can we do if we want such results to be stored? Well, perhaps all possible results can be placed into an array, but it will take too much memory space and will make our algorithm too slow. After some thought we can find an elegant solution to this problem:

Suppose that all possible results of multiplying 10 elements by 2n - 1 are stored in an array, where n is the number of possible elements (1-9 for us). In other words, if n = 5, then there are 45 different possible results. Also, let’s say that we want to store them as binary search trees. How can we make sure that their sizes will not be too big? We know that binary search trees with a given number of nodes have a fixed height. This means that if our initial array is 10 elements long then its result will be a binary search tree with a height of 6 at most. In our case, this means that binary search trees with a size of 30 are only needed to store all possible results.

The next thing we need to think about is the position of values in a binary search tree. Suppose we have 10 elements in our initial array and say these are 5, 6, 7, 8, and 9. In other words, the result of multiplying these elements by 2n - 1 is [1, 2, 3, 4].

We will organize them as a binary tree with roots at position 5. We’ll go down from its left child until we reach 6. Next, we go down from its left child to 7 . So far the values (1-4) have been placed into our binary search tree. The next step is to go down from its left child until we reach 8. We continue this process until we’ve reached our ninth element. This means that, for every element in our initial array, we will have a unique path from the root to itself. So this means that the result of multiplying 10 elements by 2n - 1 (with n = 5 ) is represented in the following array:

It is easy to see that, if n = 6, then there are 165 different possible results of multiplying 10 elements by 2n - 1. In this case, binary search trees with a size of 90 are needed to store them all.

So we can use the following Python function to get the Cartesian product of two arrays: def cartesian_product ( X, Y ): """Return a Cartesian product of X and Y.""" result = [] for e in range ( len ( X )): for o in range ( len ( Y )): result. append (( e , o )) return result

Now let’s write a Python function to choose one out of 10 selections for each n. This means that we have already chosen one out of 10 selections for each e and o in our example above. We first need to find the element for which we want to choose a result. Suppose (1, 4) is chosen and we want to find the element for which we want a result. We just need to find the position of “4 ” in the above array. It is equal to position 1, so it is equal to e. Now, all we need is to determine if this value “4” will be in a binary search tree with at most 6 levels or not. If this value will be in our binary tree, then it is valid and we can proceed. Otherwise, we can return None.

def choose_one ( function ): """Return a choice of 10 selections for a given function. If a result is not in a binary tree, None is returned. """ n = len ( function ) if n == 5 : return [ None ] elif 5 <= n < 10 : selection = random . sample ( range ( 1 , 10 ), n ) if selection not in function : return None else : return [ element [ selection - 1 ] for element in function ] else : return []

To call this function, we need to pass it a binary search tree with the result that we want to choose. In our example above, we want to find the element for which we want a result. In this case, we want to find the position of “4 ” in the above array. We can use Python’s len function to get the number of elements in our binary search tree. This means that for every element there is a unique path from its root down to itself.?

Therefore, with a binary search tree we can find the position of “4” easily:

def choose_one ( function ): """Return a choice of 10 selections for a given function. If a result is not in a binary tree, None is returned. """ n = len ( function ) if n == 5 : return [ None ] elif 5 <= n < 10 : selection = random . sample ( range ( 1 , 10 ), n ) if selection not in function : return None else : return [ element [ selection - 1 ] for element in function ] else : return []

Let’s see how to find the size of a binary search tree with at most 6 levels:

def tree_size ( result ): """Return the number of nodes in a binary search tree of a given height.""" tree = [] for e in result: for o in result: tree. append (( e , o )) return len ( tree )

To call this function, we just need to pass it the result of multiplying 10 elements by 2n - 1. If the result is a binary search tree with at most 6 levels, then its size will be between 165 and 615 :

def choose_one ( function ): """Return a choice of 10 selections for a given function. If a result is not in a binary tree, None is returned. """ n = len ( function ) if n == 5 : return [ None ] elif 5 <= n < 10 : selection = random . sample ( range ( 1 , 10 ), n ) if selection not in function : return None else : return [ element [ selection - 1 ] for element in function ] else : return []

Let’s check what we have so far:

import random def choose_one ( function ): """Return a choice of 10 selections for a given function. If a result is not in a binary tree, None is returned. """ n = len ( function ) if n == 5 : return [ None ] elif 5 <= n < 10 : selection = random . sample ( range ( 1 , 10 ), n ) if selection not in function : return None else : return [ element [ selection - 1 ] for element in function ] else : return [] def tree_size ( result ): """Return the number of nodes in a binary search tree of a given height.""" tree = [] for e in result : for o in result : tree . append (( e , o )) return len ( tree ) >>> from cartesian_product import cartesian_product , choose_one >>> X = [[ 1 , 2 , 3 ], ... [ 4 , 5 , 6 ], ... [ 7 , 8 , 9 ], ... [ 10 ], ... [ 11 ], ... [ 12 ], ... [ 13 ], ... [ 14 ], ... [ 15 ]] >>> Y = [[ 2 , 6 , 3 ], ... [ 2 , 6 , 7 ], ... [ 2 , 7 , 7 ], ... [ 2 , 8 , 7 ]] >>> result = cartesian_product ( X , Y ) >>> choose_one ( result ) [( 1, 4), (1, 10), (2, 4), (2, 8)]

Choose one element with a given value out of 100 values

Let’s now generate a list of 100 elements in the following format:

(1-100)[0][0] and then find the index of the element we are looking for. If this index is less than 20, then return it. Otherwise, return None.

def choose_one_val ( function ): """Return a choice of 100 selections for a given function. If a result is not in a binary tree, None is returned. """ n = len ( function ) if n == 5 : return [ None ] elif 5 <= n < 10 : suffix = [] for i in range ( 0 , 100 ): suffix . append (( 1 - i , i )) selection = random . sample ( range ( 1 , 100 ), n ) if selection not in function : return None else : return [ element [ selection - 1 ] for element in function ] else : return [] def tree_size ( result ): """Return the number of nodes in a binary search tree of a given height.""" tree = [] for e in result : for o in result : tree . append (( e , o )) return len ( tree ) >>> from cartesian_product import cartesian_product , choose_one_val >>> X = [[ 0 , 1 , 2 ], ... [ 4 , 5 ], ... [ 7 ], ... [ 8 ], ... [ 9 ], ... [ 10 ]] >>> Y = [[ 3 ], ... [ 6 ], ... [ 1 ], ... [ 0 ], ... [ 6 ]] >>> result = cartesian_product ( X , Y ) >>> choose_one_val ( result ) [(1, 4)]

Choose one element with a given value out of 10 values

Let’s start with generating a list of 100 elements in the following format:

8 and then find the position of “8” in this array. If this position is equal to “2”, then return it. Otherwise, return None.

def choose_one_index ( function ): """Return a choice of 10 selections for a given function. If a result is not in a binary tree, None is returned. """ n = len ( function ) if n == 5 : return [ None ] elif 5 <= n < 10 : suffix = [] for i in range ( 0 , 100 ): suffix . append (( 1 - i , i )) selection = random . sample ( range ( 1 , 100 ), n ) if selection not in function : return None else : return [ element [ selection - 1 ] for element in function ] else : return [] def tree_size ( result ): """Return the number of nodes in a binary search tree of a given height.""" tree = [] for e in result : for o in result : tree . append (( e , o )) return len ( tree ) >>> from cartesian_product import cartesian_product , choose_one_index >>> X = [[ 0 , 1 , 2 ], ... [ 4 , 5 ], ... [ 7 ], ... [ 8 ], ... [ 9 ], ... [ 10 ]] >>> Y = [[ 3 ], ... [ 6 ], ... [ 1 ], ... [ 0 ], ... [ 6 ]] >>> result = cartesian_product ( X , Y ) >>> choose_one_index ( result ) [(1, 4)]

Choose one element with a given value out of 100 values

Let’s start with generating a list of 100 elements in the following format:

8 and then find the position of “8” in this array. If this position is equal to “2”, then return it. Otherwise, return None.

def choose_one_index ( function ): """Return a choice of 10 selections for a given function. If a result is not in a binary tree, None is returned. """ n = len ( function ) if n == 5 : return [ None ] elif 5 <= n < 10 : suffix = [] for i in range ( 0 , 100 ): suffix . append (( 1 - i , i )) selection = random . sample ( range ( 1 , 100 ), n ) if selection not in function : return None else : return [ element [ selection - 1 ] for element in function ] else : return [] def tree_size ( result ): """Return the number of nodes in a binary search tree of a given height.""" tree = [] for e in result : for o in result : tree . append (( e , o )) return len ( tree ) >>> from cartesian_product import cartesian_product , choose_one_index >>> X = [[ 0 , 1 , 2 ], ... [ 4 , 5 ], ... [ 7 ], ... [ 8 ], ... [ 9 ], ... [ 10 ]] >>> Y = [[ 3 ], ... [ 6 ], ... [ 1 ], ... [ 0 ], ... [ 6 ]] >>> result = cartesian_product ( X , Y ) >>> choose_one_index ( result ) [(1, 4)]

Perfect! Now, how do we implement this as a function?

def binary_search ( array ): """Return the position of an element in a set, where the element to be searched is known in advance. If not found, return -1. """ n = len ( array ) if n == 5 : return [ - 1 ] elif 5 <= n < 10 : suffix = [] for i in range ( 0 , 100 ): suffix . append (( 1 - i , i )) selection = random . sample ( range ( 1 , 100 ), n ) if selection not in array : return - 1 else : return [ element [ selection - 1 ] for element in array ] else : return [] def tree_size ( result ): """Return the number of nodes in a binary search tree of a given height.""" tree = [] for e in result : for o in result : tree . append (( e , o )) return len ( tree ) >>> from binomial_coding.cartesian_product import cartesian_product >>> X = [[ 0 , 1 , 2 ], ... [ 4 , 5 ], ... [ 7 ], ... [ 8 ], ... [ 9 ], ... [ 10 ]] >>> Y = [[ 3 ], ... [ 6 ], ... [ 1 ], ... [ 0 ], ... [ 6 ]] >>> result = cartesian_product ( X , Y ) >>> binary_search ( result ) -1

Now, how do we implement this as a function?

def search_index ( index ): """Return the index of an element in a set, where the element to be searched is known in advance. If not found, return -1. If a position is found, return the corresponding element. """ n = len ( index ) if n == 5 : return [ - 1 ] elif 5 <= n < 10 : selection = [] for i in range ( 0 , 100 ): selection . append (( 1 - i , i )) selection = random . sample ( range ( 1 , 100 ), n ) if selection not in index : return - 1 else : return [ element [ selection - 1 ] for element in index ] else : return [] def tree_size ( result ): """Return the number of nodes in a binary search tree of a given height.""" tree = [] for e in result : for o in result : tree . append (( e , o )) return len ( tree ) >>> from binomial_coding.cartesian_product import cartesian_product >>> X = [[ 0 , 1 , 2 ], ... [ 4 , 5 ], ... [ 7 ], ... [ 8 ], ... [ 9 ], ... [ 10 ]] >>> Y = [[ 3 ], ... [ 6 ], ... [ 1 ], ... [ 0 ], ... [ 6 ]] >>> result = cartesian_product ( X , Y ) >>> search_index ( result ) -1

Note: This can be adapted/implemented in any language. Only the idea is to demonstrate how implementing this idea in Python is of my interest. However, the disadvantage for Python would be that it will take longer to run each time compared to the speed of running something like this in Ruby.

The most interesting part is that I was able to implement this pretty easily. At least at a conceptual level. A real challenge here would be figuring out if there’s a way to write this function as something like itertools. product.

At any rate, I’ve worked with this idea for a few hours now, and it was really easy to implement this.

You may be interested in the full codebase which can be found here. It’s written in Python 3.3.2 and contains everything that was mentioned in this article. I hope it helps! Thanks for reading!