The Optuna Project, Advanced Topics in Cryptography, Beyond Multiple Linear Regression

This week's agenda:

• Open Source of the Week - the Optuna project
• New learning resources - Advanced Topics in Cryptography, Intro to Machine Learning featuring Generative AI, Python lambda function
• Book of the week - Beyond Multiple Linear Regression by Prof. Paul Roback and Prof. Julie Legler

Daily updates on Telegram, WhatsApp, or Viber.

Are you interested in learning how to set up automation using GitHub Actions? If so, please check out my course on LinkedIn Learning:

Open Source of the Week

This week's focus is on the optuna project.

Optuna is a Pythonic framework for hyperparameter optimization and model tuning. It provides a set of tools and functions to automate the search for the optimal parameters with respect to a pre-defined loss function (or error metric). This framework is agnostic and it supports the core machine learning and deep learning frameworks in Python. This includes frameworks such as Scikit-Learn, XGboost, LightGbm, PyTorch, TensorFlow, and Keras.

The library key features:

• Flexible search spaces: It allows complex search spaces using Python conditionals and loops.
• Efficient optimization: Utilizes state-of-the-art algorithms and pruning to speed up the search.
• Easy parallelization: Supports parallel optimization across multiple threads or processes with minimal code changes.

The hyperparameter optimization is straightforward, and it includes the following steps:

• Set the objective function, which includes the model tuning parameters range, the model object, and the loss metric
• Set optuna object
• Execute the optimization

For example, this is the process of tuning a lightgbm model:

The full example is available on this link.

The library has a built-in functionality to track the model hyperparameter optimization history and visualize the results using a dashboard:

More details are available in the project documentation:

License: MIT

New Learning Resources

Here are some new learning resources that I came across this week.

Advanced Topics in Cryptography

MIT recently released a new course on cryptography. This course, by Prof. Yael T.Kalai, focuses on the evolution of proofs in computer science, and it provides some of the mathematical background beyond cryptography.

Intro to Machine Learning featuring Generative AI

This is a new course from freeCodeCamp and Rola Dali that focuses on the foundation of machine learning and generative AI.

Lambda Function in Python

This is a short and concise guide by Indently for Python's Lambda function and how to use it.

Book of the Week

This week's book focuses on applied statistics. The Beyond Multiple Linear Regression by Prof. Paul Roback and Prof. Julie Legler provides an introduction to applied generalized linear models (GLM) and multilevel models with practical examples using R. The book covers topics such as:

• Foundation of regression analysis
• Simple and multiple linear regression
• Maximum likelihood function
• Distribution theory
• Generalized linear models
• Logistic regression
• Multilevel models

The book uses R to demonstrate some of the theories and concepts taught in the book. Thanks to the authors, the book has an open online version:

A printed version is available for purchase through the publisher or via Amazon:

Have any questions? Please comment below!

See you next Tuesday!

Thanks,

Rami