A Glossary of AI and Data Science Terms

It has been a long month already, with much news on the AI front. It has been useful to me to have a quick glossary to stay up to date with the acronyms and terms. It is getting to be as complex as IP Networking out there!

A

Artificial General Intelligence (AGI)

An AI system with human-like cognitive abilities, capable of learning, reasoning, and adapting across various domains and tasks.

Artificial Intelligence (AI)

The simulation of human intelligence in machines programmed to think and learn like humans.

Artificial Neural Networks

Computational models inspired by the human brain's neural structure, used to recognize patterns and make predictions.

C

Chain-of-Thought (CoT)

An approach in AI that enhances reasoning abilities by generating intermediate steps, providing transparency in decision-making processes.

Chain-of-Thought (CoT) Tokens

Internal, intermediate reasoning steps generated by a model during processing, representing the model's "thought process" and typically hidden from end users for safety and security.

CI/CD (Continuous Integration/Continuous Deployment)

Practices that automate testing and deployment of code changes, enabling regular updates and maintenance of AI models and systems.

D

Data Science

A scientific field that utilizes structured and unstructured data, manipulating it through various processes and algorithms to extract purpose-specific knowledge.

Data Wrangling

The process of cleaning, structuring, and enriching raw data into a desired format for better decision making in less time.

Deep Learning

A subset of machine learning that uses multi-layered neural networks to learn from vast amounts of data, enabling complex pattern recognition.

Dot Product

A mathematical operation multiplying corresponding elements of two vectors and summing the results, used in AI to measure similarity between vectors.

E

Embedding

The process of converting tokens into high-dimensional numerical vectors that capture semantic properties and relationships of words, serving as input to neural networks.

F

Feed-Forward Neural Networks

Layers within a model that apply transformations independently to each token's representation, refining and processing features further.

Fine-Tuning

The process of taking a pre-trained model and further training it on a specific dataset or task to adapt its behavior to specialized needs.

G

GPU/CUDA

GPU (Graphics Processing Unit) is specialized hardware for parallel processing, accelerating computation-heavy AI tasks. CUDA is NVIDIA's platform for harnessing GPU power for parallel computations.

I

Inference

The stage where a trained model processes new, unseen input data to generate predictions or outputs, with the model's parameters remaining fixed.

L

Layer Stacking

The practice of combining multiple transformer layers to build increasingly complex and abstract representations of the input.

M

Machine Learning

A branch of AI that involves programming systems to automatically learn and improve from experience without being explicitly programmed1.

Multi-Head Attention

An enhancement of the self-attention mechanism where multiple attention "heads" operate in parallel, each capturing different aspects of token relationships.

N

Natural Language Processing (NLP)

A field of AI focused on enabling computers to understand, interpret, and generate human language in a valuable way.

P

Positional Encoding

A technique used in transformers to inject information about token order or position within a sequence.

Prompt Engineering

The practice of designing and refining input prompts to elicit desired responses from language models.

Prompt Injection

A technique where the input prompt is crafted or manipulated to alter the model's behavior, potentially overriding default instructions.

Q

Query, Key, and Value Vectors

Components derived from token embeddings in a transformer, used in the self-attention mechanism to compute context-aware representations.

R

Reinforcement Learning

A type of machine learning where an agent learns to make decisions by taking actions in an environment to maximize a reward signal.

S

Self-Attention

A mechanism within transformers that enables each token to consider all other tokens in the sequence to build a context-aware representation.

Self-Supervised Learning

A training approach where the model learns from unlabeled data by predicting parts of the input, generating its own supervision signal.

T

Tokenization

The process of breaking raw text into smaller units (tokens) such as words, subwords, or characters for processing by a model.

Transfer Learning

A machine learning technique where a model developed for one task is reused as the starting point for a model on a second task.

Transformer

A neural network architecture using self-attention mechanisms to process entire sequences in parallel, capturing long-range dependencies efficiently.

V

Vectors

High-dimensional numerical representations encoding information about tokens or features, used in vector algebra operations within AI models.