AI in Industrial Physics: APS Webinar Notes

Last week, I had the opportunity to be a panelist on a webinar hosted by the American Physical Society (APS), where we explored the role of Artificial Intelligence in Industrial Physics. For those not familiar, APS is nonprofit on a mission to advance the knowledge of physics. Dating back to 1899, it represents over 50,000 members across academia, national labs and industry.

This webinar – artfully facilitated by Peter Fiske – was a wide-ranging exploration into how AI is influencing physics, along with advice on how to work at this intersection for those interested in doing so. For an in-depth download – and to hear the much more astute points made by my co-panelists from organizations such as Toyota Research Institute and Sandia National Labs –?I recommend watching the full recording embedded below. For a sneak peek, and as a complement to the recording, also including here an excerpt of my prep notes.

A big thank you to Alex Semendinger (Mathematics PhD student at Brandeis), David Shih (High Energy Theory Professor at Rutgers), and Gage DeZoort (Physics postdoctoral researcher at Princeton) for letting me pick their brains as I prepped, and Stephanie Hervey at APS for the opportunity ?? ??

What are some ways in which Artificial Intelligence/Machine Learning is having an impact in physics and physics-related industries?

• In advanced materials research, deep learning has been used to increase the speed and efficiency of discovery by predicting the stability of new materials.?Recently, a team was able to describe 2.2 million new crystals, 736 of which have been independently created by external researchers.?What's more, this process is being paired with autonomous labs to further accelerate the discovery process.?Where synthesis fails, this data is fed back into the learning model, which then improves follow-on techniques for materials screening and synthesis design. (Reference: Scaling Deep Learning for materials discovery; Nature, November 2023)
• In fusion energy, an overarching goal is to pull atoms together and capture the resulting energy. Our sun - a fusion energy plant –?accomplishes this through its sheer gravitational mass. On earth, scientists use powerful magnetic coils to confine the nuclear fusion reaction inside of donut-shaped "tokamak" vessels.?The Swiss Plasma Center used Reinforcement Learning to autonomously learn to command the full set of 19 magnetic control coils, a promising step towards a new approach to sustaining these reactions. This Neural Network was initially trained in a simulation. (Reference: Magnetic control of tokamak plasmas through deep reinforcement learning, Nature, February 2022)

• In high energy physics, many Machine Learning methods (e.g. graph networks, transformers, diffusion models, auto-encoders, normalizing flows) are having an impact. One theoretical physicist I spoke to shared how, in his experience, applying ML has enabled physicists to, for example, train models for anomaly detection, i.e. "look at anomalous events, reconstruct the invariant masses of the particles in the collision, and then decide if they can be described by a Standard-Model process." If they can't, perhaps it's a pointer to new physics. AI methods have also helped speed up existing simulations, and simulation-based inference, he continued.
• For accelerating the research process in general, platforms such as IBM's Deep Search and Google DeepMind's Gemini have been built to help researchers analyze a large corpus of existing knowledge and quickly make sense of it, an important part of the research process that can otherwise be painstakingly slow. IBM's approach involves creating "knowledge graphs," which uses Natural Language Processing to construct a comprehensive view of nodes, edges, and labels, enabling question answering and search systems to retrieve and reuse comprehensive answers to given queries. A preview of Gemini's approach, here:

What are some limits to applying AI/ML in physics?

• At facilities such as the Large Hadron Collider, one postdoc I spoke to shared how researchers there often face pre-defined constraints, e.g. a very stringent data-taking pipeline or little time to use compute resources. Access to talent and compute is a common theme I heard, especially in academia or smaller startups. This postdoc continued: "In academia if a group has 5-10 researchers working on the same thing, that's considered large."?A May 2023 MIT Sloan article (Study: Industry now dominates AI research) wrote: "Today, roughly 70% of individuals with a PhD in artificial intelligence get jobs in private industry, compared with 20% two decades ago.” Based on number of parameters, the largest AI models developed in any given year now come from industry 96% of the time. The number of published papers with industry co-authors has nearly doubled since 2000." That giant sucking sound? AI talent getting swooped up by industry!

• Silver Lining 1: Resource scarcity has led to innovations that enable smaller organizations and teams to do more with less. One postdoc I spoke to noted how equivariant architectures, which force neural networks to obey a certain symmetry, have enabled researchers in more resource constrained environments to work with less data, fewer parameters, and train and run models more efficiently. Such innovations benefit the research community as a whole, including in industry.
• Silver Lining 2: In the United States the government is stepping up to try and close the gap. In 2020, the National Science Foundation announced a $100 million investment in five NSF AI institutes, including the AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI) whose summer workshop I profiled earlier this year.?In 2023, the NSF also announced a $140 million investment to support AI research in seven areas of opportunity and risk associated with advances in AI.?The US Department of Energy, via national labs such as Argonne, is also driving strategic AI for Science initiatives as detailed in this 2023 report.

How can physics students/professionals best position themselves to transition to a data science or AI/ML career?

• The first step is to realize that as a physicist you have gained tools that are incredibly valuable in AI. Specifically you are well-versed in breaking down a complicated system into small parts; noticing something that works a certain way and building an intuition around it. Diffusion models – a breakthrough behind some of the photorealistic image generators that have become popular as of late – can find their antecedents in the physical process of diffusion from thermodynamics.
• Second: get practical experience. For example, learn how to code and build your first neural networks via micro-courses on Kaggle (free).?Coding opens up so many doors – just be sure to learn the latest frameworks, be it TensorFlow, PyTorch, etc. Once you've got some coding under your belt, find research papers that inspire you and, as one PhD student recommended: try and replicate their work on your own. And/or check out more specialized courses, such as this MITx course on Computational Data Science in Physics.
• Third: apply for an internship. Research Experiences for Undergraduates (REU) programs, funded by the National Science Foundation, support active participation by undergraduate students in research areas funded by the NSF. Google's Summer of Code enables you to pair up with mentor organizations such as CERN, who propose ideas for areas of research they need help with. Submit a proposal and, once accepted, spend 12+ weeks working on a project for your sponsor, receiving mentorship and building relationships throughout the process.

Full video recap of our webinar, below. Interested in attending a future APS webinar? Check them out and register here!

For more AI for science posts and my recommended resources, find me at solvereality.ai.