An A to Z of AI

Is it just me, or is it hard to pick up a newspaper or go to an event and not read or hear someone talking about AI?!? AI seems to be everywhere!? Which is great news for an AI patent nerd like me, but it can be confusing for anyone who does not work in the AI space.? Since AI is being used for everything from drug design to music generation, and medical diagnoses to autonomous vehicles, whatever your field of expertise, you may need to have some understanding of AI lingo. So, in this month’s edition of the brAIn blog, I provide an A to (almost) Z of AI to help you to get to grip with some of the most commonly used terminology.?

A is for… Artificial Intelligence.?

This is generally referred to mean any computer system which is taught to have human-like intelligence and mimic human capabilities such as reasoning, learning, and perception.?

B is for… bias.?

In order for an AI model to obtain its intelligence, the model needs to be provided with data to learn from.? However, data which is incomplete or skewed in some way causes the AI model to be skewed in one direction.? It’s very similar to how humans learn – if we are only exposed to one viewpoint out of a number of possible viewpoints, our perceptions are skewed towards the viewpoint we’ve seen.? In the context of AI, if an AI model is being taught to recognise animals in images, but is provided with a dataset of images which show dogs, the AI model will learn to recognise dogs but will fail to recognise cats, fish, elephants, etc.?

Bias is a big problem because good quality datasets are difficult to obtain.? That’s why some folk are either building techniques to reduce bias in the learning process (see F below), or are generating synthetic datasets – see this previous brAIn blog article on artificial data.

C is for… convolutional neural networks.?

Firstly, if you are not sure what a neural network is, take at look at N below.? Convolutional neural networks (CNNs) are a type of neural network where the hidden layers include one or more layers that perform convolution, which is the mathematical way of combining two signals. ?In simple terms, convolution involves a multiplication.? A CNN’s earlier layers typically focus on identifying simple or straightforward features in an input data item, which may be an image.? Later layers of a CNN focus on more details, until, for example, an object in the image has been identified.? There are lots of CNN-based architectures, and these are used for a variety of purposes, such as computer vision (see I below).

D is for… deep learning.?

This is a type of machine learning algorithm (see M below), where deep refers to the number of layers in the neural network (see N below).? As noted above for CNNs, early layers in the neural network identify simple features, such as edges or colours in an image, while deeper/later layers may identify concepts that are relevant to a particular task.? For example, if the task is to process images to identify dogs, early layers may identify edges or colours in the image, while later layers may identify specific features of dogs.

E is for… embeddings.?

Machine learning algorithms are essentially a lot of maths!? The algorithms take an input data item, perform a series of mathematical operations on the input data item, and produce an output. All of this is done numerically.? This means that the input data item has to be converted into a numerical input, and then all the mathematical operations are performed on that numerical input.? An embedding is a mathematical representation of data, whether that data is text, images, or audio.? In other words, an embedding is a vector which represents important features in an input, such as an image.? (See also V below). Each value of the vector (a, b, c, d,… ) is a point in multi-dimensional space.? Embeddings enable machine learning algorithms to perform tasks such as image classification.? For example, if an algorithm has learned an embedding for a dog, it can determine whether an image contains a dog by comparing the learned embedding of a dog with an embedding generated for the image.?

F is for… federated learning.?

This is a machine learning technique that trains an algorithm in a decentralised way.? Traditionally, an algorithm is trained using a centralised dataset.? For example, an algorithm may be trained on a server, and the server stores all the data needed for the training.? However, as noted above in B above, when a dataset is incomplete, it can lead to bias.? Take object detection in images as an example.? In order to train a model to recognise a ball in images, you need a dataset of images depicting lots of different balls in lots of different contexts, lighting conditions, and so on, so that a model can accurately identify a ball, whether that’s a rugby ball, tennis ball, beach ball, snooker ball or ping pong ball.? It needs to be tell the difference between a ball and other round things, like a balloon.? A dataset containing such a variety of images is hard to obtain but, people take lots of photos using their smartphones, and some of these may contain images of balls.? Federated learning is a way to train an algorithm using all these images on individual devices in a way that does not involve actually sharing the images and creating a large central dataset.? Federated learning trains a global algorithm by training local versions of the global algorithm on local devices, such as smartphones, using the data held on those local devices. Parameters of the locally trained model are then shared with to update the global model.?

Federated learning has its own problems, one being the amount of data that needs to be sent back and forth during the process.? An example patent that mentions federated learning is EP3494522B1, which describes an efficient communication technique for transmission of model updates within a machine learning framework, involving local devices that may have unreliable network connections and low computational power.

G is for… generative AI.?

This is a type of AI that is capable of generating text, images, music, and so on.? You have probably heard of ChatGPT and DALL-E (which I’ve used to generate the image for this edition) – these are examples of generative AI.? Generative AI models learn patterns and characteristics in training data, and then use this learned information to generate data with similar patterns and characteristics.? US11049481B1 is a patent that describes measuring the quality of AI-generated music, to help improve the generative model.?

H is for… hallucination.?

If you have heard of ChatGPT and you work in the legal space, you’ve probably heard of the US lawyer who used ChatGPT to perform some case research and ended up including fake case law citations in their court submissions.? This is an example of a hallucination, which is a false fact output by a generative AI model like ChatGPT.? The problem arises because generative AI models have learned patterns and characteristics from training data (see G above), and can provide outputs based on that learned information that looks right but are not necessarily right.? In this example, it was able to output case law citations that have the right format, but were not real.?

I is for… inference.?

Inference is the process of inputting data into a trained AI algorithm to generate an output or prediction.?

I is also for… image processing.?

Also known as computer vision, this is a field of computer science that is concerned with using AI to identify and understand objects, people and actions in images and videos.? Example uses of computer vision include object classification, object detection, optical character recognition (OCR), gesture recognition. If you unlock your smartphone using your face, the smartphone may be using AI to recognise you.? Similarly, if you use a certain gesture to control the selfie camera on your smartphone, then the smartphone may be using AI to detect that gesture.?

J is for… joint learning.?

Also known as multi-task learning, this is a type of machine learning in which an algorithm learns how to perform multiple tasks.? This is based on the fact that when the tasks are related, learning how to perform one task helps the algorithm to learn how to perform another task.? Some model parameters may be shared between the tasks.? A way to think about this is how humans learn to identify objects – babies learn how to recognise faces, and this learning can be used to learn how to recognise animals or other objects.

K is for… knowledge graph.?

Here, graph does not mean a bar graph or pie chart, but a specific type of mathematical structure used to model relationships between objects.? A knowledge comprises nodes, which may represent objects, places, people, concepts, and so on.? A node may be connected to one or more other nodes by an edge.? The edge defines a relationship between nodes.? For example, one node may be the word “dog” and another node may be the word “animal”, and the edge connecting the two nodes defines the relationship, i.e. that a dog is a type of animal.? Knowledge graphs are commonly used by search engines and question-answering systems such as Siri.? I’ve talked about knowledge databases in an early version of this blog.

L is for… LLM.?

A large language model is a type of AI algorithm that is… very large!? The models are too big to run on normal computers or smartphones, and so are typically accessed as a web-based service or via an API.? Example LLMs include ChatGPT and the others in the GPT family, and Bard. LLMs may be used to perform natural language processing tasks, such as providing text in a certain style, performing question-answering, summarising text, and so on. LLMs are trained on vast quantities of data, typically scraped from the Internet.

M is for… machine learning.?

This is a general term for algorithms that enable machines to solve problems and make predictions by extracting insights from training data, and without needing to be explicitly told what to do.?

N is for… neural network.?

Neural networks are a type of machine learning and form the basis of many deep learning algorithms.? Neural networks are formed of nodes which are arranged in layers (picture them as columns), where the nodes of one column are connected to nodes of neighbouring columns.? Each node is connected to another node with an associated weight and threshold value.? If a node’s value is above the threshold value, the node is activated and sends data to one or more nodes to which it is connected in the next layer in the network.? Each neural network has an input layer and an output layer, and one or more hidden layers between the input and output layers.?

O is for… overfitting.?

This is a problem that occurs in machine learning when an algorithm is able to provide accurate outputs/predictions for data that closely resembles the data the algorithm was trained on, but is unable to do the same when the data is different.? ?This may happen when the training data contains noisy data.? For example, if a model is trained to identify dogs, but has only been trained on images of dogs in a park, the model may learn that features of the park, such as trees and grass and benches, are linked to dogs.? This results in a model not being able to accurately recognise a dog in an image when the dog is on a beach or curled up on a sofa.?

P is for... prediction.?

The output of AI systems are often referred to as predictions.? Predictions are not necessarily about what might happen in the future.? Instead the term is used more generally to mean the output of a trained machine learning model. ?See also I above.?

R is for… reinforcement learning.?

There are three general types of machine learning technique – supervised learning, unsupervised learning, and reinforcement learning.? Reinforcement learning is similar to supervised learning, except that successful outcomes are reinforced to maximise a reward.? It does not require labelled data.?A way to think about reinforcement learning is how you may reward a child or pet with a treat for good behaviour - the child/pet then learns what they have to do to keep getting those treats!

S is for… supervised learning.?

There are three general types of machine learning technique – supervised learning, unsupervised learning, and reinforcement learning.? Supervised learning requires the use of labelled training data to help the model to learn how to make predictions.? The model keeps adjusting its weights until it outputs the same label for a data item as the known label for that data item.? Often the training data is labelled by humans, which makes it expensive to obtain. ?Supervised learning may be used to detect dogs in images, by training an algorithm using labelled images of dogs.? Have a read of an earlier version of this blog in which I talk about object detection.

T is for… transformer.?

A transformer is a robot in disguise. (If you know, you know!)? It is also a type of deep learning architecture that is very powerful and useful.? So much so, I’ll be talking about the transformer in more detail in an upcoming edition of the blog.?

U is for… unsupervised learning.?

There are three general types of machine learning technique – supervised learning, unsupervised learning, and reinforcement learning.? Unsupervised learning looks for patterns in unlabelled data and without being provided with any guidance from a human operator. ?For example, unsupervised learning may be used to detect dogs in images, by allowing an algorithm to form its own concept of what a dog is.?

V is for… vector. ?

Vectors and matrices feature heavily in machine learning. ?Vector and matrix multiplication is at the heart of many algorithms, but when the matrices are very large, matrix multiplications are very computationally expensive.? So, working out ways to speed-up or simplify the matrix multiplications is key.? See also E above.

W is for… weights.?

Weights are the parameters of a neural network that transform input data into an output.? See also N above.? As an input enters a node of the neural network, it is multiplied by a weight, and the resulting value is either above the threshold value and passed to the next layer in the neural network or not.? Large weights more significantly change the output, while small weights may have little or no impact.? ?

Key Take-Aways

I hope this glossary is useful!? Of course, if you want to know more or want to know whether your software or AI invention could be protected using patents, please contact me via email ([email protected]) and I would be happy to have a confidential chat with you!