50 Key Definitions in Machine Learning

1. Machine Learning: Machine Learning is a field of artificial intelligence (AI) that involves developing algorithms and models that enable computers to learn and make predictions or decisions without being explicitly programmed.
2. Supervised Learning: A type of machine learning where the model is trained on a labeled dataset, meaning the input data is paired with the correct output. The algorithm learns from these pairs to make predictions or classifications on new, unseen data.
3. Unsupervised Learning: This type of machine learning occurs when the model is trained on unlabeled data. The algorithm explores the data's structure or patterns without specific outputs, often used for clustering or dimensionality reduction.
4. Reinforcement Learning: A machine learning paradigm where an agent learns to make decisions by taking actions in an environment. The agent receives feedback in the form of rewards or penalties, allowing it to learn the best strategies or policies to achieve specific goals.
5. Algorithm: It's a step-by-step procedure or set of rules followed by a computer to solve a problem or perform a task. In machine learning, algorithms are used to train models on data and make predictions or decisions.
6. Model: A representation or abstraction of a system, process, or phenomenon. In machine learning, a model refers to the learned patterns or relationships from data that enable predictions or classifications on new input.
7. Classification: A task in supervised learning where models predict discrete class labels for input data, such as categorizing emails as spam or not spam.
8. Regression: A supervised learning task where models predict continuous numerical values, such as predicting house prices based on various features.
9. Clustering: An unsupervised learning task where models group similar data points together based on inherent patterns or similarities.
10. Dimensionality reduction: Dimensionality reduction is a process used in machine learning and data analysis to reduce the number of features or variables in a dataset while preserving as much relevant information as possible.
11. Data Points/Instances: In machine learning, data points refer to individual instances or observations within a dataset. Instances are also called ‘examples’ or ‘data points’ or 'observations'.
12. Label: The label represents the correct answer or expected output for a given input. For instance, in image classification, if the input is an image of a cat, the corresponding label would be the category or class "cat." Some prefer using terms like 'target,' 'output,' or 'response' interchangeably with 'label' to emphasize the concept of what the model is aiming to predict or produce based on the input.
13. Feature: In machine learning problems, a feature refers to an individual measurable property or characteristic of the data used for training a model. Features are the variables or attributes that the model learns from to make predictions, classifications, or decisions.
14. Feature Engineering: The process of selecting and transforming relevant features from raw data to improve model performance.
15. Feature Selection: Feature selection involves selecting a subset of the original features while discarding the irrelevant or redundant ones.
16. Feature Extraction: Feature extraction creates new features by combining or transforming the original features into a lower-dimensional space.
17. Normalization: Normalization is the process of scaling numerical features to a common scale, typically between 0 and 1. The goal is to adjust the values of different features to a uniform range.
18. Standardization: Standardization (also known as z-score normalization) is a technique that rescales features to have a mean of 0 and a standard deviation of 1. This process centers the data around 0 and scales it based on the variance. Standardization is effective when the features have different means and standard deviations.
19. Hyperparameters: Parameters that are set before training a machine learning model and affect its learning process, such as learning rate or number of hidden layers in a neural network.
20. Ensemble Learning: A technique that combines multiple machine learning models to improve overall performance and predictive accuracy.
21. Transfer Learning: The practice of leveraging knowledge gained from one task or domain to improve learning or performance in another related task or domain.
22. Deep Learning: A subset of machine learning involving artificial neural networks composed of multiple layers, allowing models to learn representations of data with multiple levels of abstraction.
23. Neural Network: A computational model inspired by the human brain composed of interconnected nodes (neurons) organized in layers to process complex data.
24. Gradient Descent: An optimization algorithm used to minimize the loss function by adjusting the model's parameters iteratively.
25. Loss Function: A measure of how well a model performs on the training data by calculating the difference between predicted and actual values.
26. Regularization: Techniques applied to prevent overfitting by adding penalties or constraints to the model's parameters during training.
27. Generalization: Generalization in the context of machine learning refers to the ability of a trained model to perform well on new, unseen data that it hasn't encountered during the training phase. When a machine learning model generalizes well, it demonstrates an ability to understand and infer patterns or relationships from the training data and apply that knowledge effectively to make accurate predictions or classifications on previously unseen examples.
28. Overfitting: It occurs when a machine learning model learns the training data too well, including noise and irrelevant details. As a result, the model performs well on the training data but fails to generalize to new, unseen data.
29. Underfitting: The opposite of overfitting, underfitting happens when a model is too simple to capture the underlying patterns in the training data. As a result, it performs poorly both on the training data and new data.
30. Cross-Validation: A technique used to assess a model's performance by splitting the data into subsets for training and validation to avoid overfitting.
31. Bias: The error introduced by approximating a real-world problem with a simplified model.
32. Variance: The variability of model predictions for a given input caused by sensitivity to fluctuations in the training data.
33. Bias-Variance Tradeoff: It's the balance between errors due to bias (underfitting) and variance (overfitting) in machine learning models. Finding an optimal point between bias and variance helps achieve better generalization on unseen data.
34. Speed-Accuracy Tradeoff: This tradeoff involves the relationship between the computational speed or efficiency of a model and its accuracy.
35. Exploration-Exploitation Tradeoff: This tradeoff is particularly relevant in reinforcement learning and optimization problems. It involves deciding between exploring new possibilities (exploration) and exploiting known strategies or actions (exploitation) to achieve the best overall outcome. Balancing exploration and exploitation is essential to find the optimal strategy while gathering sufficient information about the environment.
36. Validation Set: A separate portion of the dataset used to evaluate a model during training to tune hyperparameters and prevent overfitting.
37. Test Set: A portion of the data reserved to evaluate the final model's performance after training and validation, providing an assessment of its generalization capabilities.
38. Confusion matrix : A confusion matrix is a table used in machine learning to evaluate the performance of a classification model. It allows visualization of the performance of an algorithm by presenting a summary of the model's predictions compared to the actual ground-truth labels.
39. Accuracy: It measures the proportion of correctly classified instances out of the total instances in the dataset.
40. Precision: Precision calculates the ratio of correctly predicted positive observations to the total predicted positive observations. It focuses on the accuracy of the positive predictions.
41. Recall (Sensitivity): Recall computes the ratio of correctly predicted positive observations to the actual positives in the dataset. It focuses on how many of the actual positives were captured by the model.
42. F1 Score: The F1 score is the harmonic mean of precision and recall. It provides a balance between precision and recall.
43. ROC Curve (Receiver Operating Characteristic Curve) and AUC (Area Under the ROC Curve): ROC curve illustrates the true positive rate against the false positive rate at various threshold settings. AUC summarizes the ROC curve, providing a single value for model comparison.
44. Mean Absolute Error (MAE): It measures the average absolute differences between predicted and actual values.
45. Mean Squared Error (MSE): MSE calculates the average of the squares of the differences between predicted and actual values.
46. Root Mean Squared Error (RMSE): RMSE is the square root of the MSE, providing the measure in the same units as the target variable.
47. R-squared (Coefficient of Determination): R-squared represents the proportion of variance in the dependent variable that is predictable from the independent variables.
48. Silhouette Score: It measures how similar an object is to its own cluster compared to other clusters. A higher silhouette score indicates better-defined clusters.
49. Inertia: Inertia measures the compactness of clusters. It's the sum of squared distances of samples to their closest cluster centers.
50. Precision-Recall Curve: Similar to classification, it evaluates the tradeoff between precision and recall in identifying anomalies.

Please share any additional relevant terms that you believe should be defined further in the comment section.