Thermodynamic Computing: new hardwares for future AI algorithms

This is the 5th article of Beyond Entropy, a space where the chaos of the future, the speed of emerging technologies and the explosion of opportunities are slowed down, allowing us to turn (qu)bits into dreams.

What happens if computation, physics, and information theory converge together? New exciting, fast and energy-efficient computational approaches that take advantage of noise and stochastic fluctuations are emerging. They go under the name of Thermodynamic-based hardwares. This will be the focus of today’s post.

The outline will be:

• Why new hardware paradigms are needed?
• The startups that are leading Thermodynamic AI
• ?? Job & Research opportunities, talks, and events in AI.

For more references and details you can read the full version of Beyond Entropy here . Let’s start!

Thermodynamic Computing

To better understand why new hardware paradigms are needed we must go back on the software side, in particular on Thermodynamic AI algorithms. These are a family of models that heavily rely on randomness, energy-based approaches or ensembles. Among them, the most popular are Generative Diffusion Models, Monte Carlo Sampling, Bayesian Neural Networks, and Simulated Annealing. These algorithms are at the core of many Deep Learning architectures.

The problem is that today’s thermodynamic-based models are trained or run on digital hardware (CPUs or GPUs), thus limiting their scalability and overall potential (not to mention their power consumption!).

This motivates the search for novel, thermodynamic-inspired computing hardware to make Generative AI more capable, faster, and efficient. All together, these innovative approaches takes the name of Thermodynamic Computing.

Two emerging startups are challenging Nvidia's dominance, reinventing the computer chip entirely and paving the way for Thermodynamic Computing.

Normal Computing

Normal Computing is developing a physics-based thermodynamic hardware, named Stochastic Processing Units (SPU). A conventional processor performs calculations by processing information in binary form. A SPU, on the other hand, exploits the thermodynamic properties of oscillators to perform calculations using the random fluctuations that occur within circuits.

This process makes it possible to generate random samples useful for calculations or to solve linear algebra calculations, which are ubiquitous in science and engineering as well as in machine learning.

More referenced in the full version!

Extropic

Extropic mission is to merge generative AI with the physics of the world. Their motivation starts from the consideration that Moore's law is slowing down, given the physical limits of transistor silicon technology. Instead of fighting against nature, Extropic embraces it, drawing inspiration from biology's efficient computation to create hardware that thrives amid noise and randomness.

Digital computers are bad at generating random numbers. Extropic hardware accelerators aim to speed up sampling, a crucial operation in many AI models.

Their engineering challenge is clear:

How can we design a complete AI hardware and software system from the ground up that thrives in an intrinsically noisy environment?

The answer stands in probabilistic approaches, in particular relying on Energy-Based Models (EBMs). Extropic chips, modelled on the principles of Brownian motion, are programmable sources of randomness that leverage EBMs directly as parameterized stochastic analog circuits. Extropic claims that its accelerators will achieve many orders of magnitude of improvement over digital computers in terms of both runtime and energy efficiency for algorithms based on sampling from complex landscapes.

The number of alternative approaches to computing is an ever-increasing trend that will drastically transform the way, speed and efficiency with which we process information.

Opportunities, talks, and events

I share some opportunities from my network that you might find interesting:

?? Job opportunity:

• Atmosoar.io , an innovative startup in the weather data industry (with applications in the drone, energy, and insurance industry) is looking for a Data Science, Modeling, and Simulation Specialist (through Michele Travierso );
• 2024 Amazon Summer Research Science Internships in Quantum Computing (Through Simone Severini );
• New Research internship in the AI for Climate Impact team @ IBM;
• Zephyr AI, a healthcare technology company that reshapes traditional approaches to drug discovery and precision medicine, is looking for an ML Research Scientist ;
• Are you a Science Educator looking for a new challenge? Join the CERN Science Gateway education team by applying here (through Julia Woithe )

?? Research opportunities:

• The Human Technopole opened in Milan and at SISSA (Trieste) several PhD positions in AI and Data Science for Neuroscience, Genomics and Biology;
• A postdoc position related to Human-AI Interaction and Explainable AI, with a special focus on biomedical informatics and medical decision making is open at Bicocca University through Federico Cabitza ;
• Vincenzo Lomonaco, PhD (Pisa University) is looking for two research assistant (only Master Degree required) on Continual Learning!
• Junior Data Physicist @ INFN (Florence) for ML technologies applied to Particle Physics (through Matteo Barbetti )

?? Learning opportunities:

• Application are now open for Session #15 of the School of AI @ Pi School . Around 10 grants are available (apply here ) for a 8-week hands-on AI programme on real-world challenges (starting in May);
• Application are now open for several full fellowships at the Master in High-performance Computing (MHPC, SISSA/ICTP, Trieste), a 15-months-program for future experts in HPC, AI and Quantum Computing (Link to apply );
• Quantum Machine Learning Master at Ca’ Foscari Challenge School - Master e Alta Formazione (apply here );
• Master's Degree in Human-Centered AI @ Statale di Milano (Applications are open! )