OpenAI Introduces Whisper, The Case for “Single Basin Theory”, and GitHub’s Arctic Vault Is Ready For Burial

The deep learning community is exploring the possibility of a single basin for neural network loss landscapes, and a new diffusion model is claiming to outperform its predecessors. Intel, Arm and Nvidia rally around standardizing the FP8 format and OpenAI introduces Whisper. Let’s dive in!

Research Highlights

• Researchers from the University of Washington demonstrated that it’s possible to mix and match trained models in neural network (NN) loss landscapes without the need for pre-training or fine-tuning. Their project, Git Re-Basin, introduces three algorithms that would allow researchers to produce more performant models by merging models trained on different datasets. The main argument is that models of a certain architecture share the same loss basins, and this single basin theory is of great interest to the deep learning community as it would have huge implications for collaborations and model reuse.

“Wow! If this turns out to work reliably in practice, this is going to be huge. Possibly a game changer for collaborations, reusing independently models to create better generalizable models, and more.” - Sebastian Raschka, Lead AI Educator | Lightning AI

• Researchers from Google and the University of Texas collaborated on defining a broader family of corruption processes that generalizes previously known diffusion models. Typically, diffusion models generate images by reversing a known corruption process that gradually adds noise. The proposed alternative incorporates the degradation process in the network and trains the model to predict a clean image that after corruption matches the diffused observation. This method is claimed to outperform all previous linear diffusion models and provides significant computational benefits compared to vanilla denoising diffusion.
• Learn how to code a simple neural network with PyTorch and Lightning! Josh Starmer PhD, the CEO of StatQuest and Lead AI Educator at Lightning, shows how Lightning makes writing the code easier and more portable. TRIPLE BAM!!!!

• Researchers from Salesforce introduced a “one-stop-shop” for language-and-vision research and applications called LAVIS. The typical process to train and evaluate language-vision models across tasks and datasets requires extensive domain knowledge that serves as an impediment to incoming researchers and practitioners. LAVIS is an open-source deep learning library designed to provide better accessibility to state-of-the-art image-language, video-language models and common datasets for training, evaluation and benchmarking on a rich variety of tasks.?

ML Engineering Highlights

• OpenAI introduced Whisper, an automatic speech recognition (ASR) system that approaches human-level robustness and accuracy on English speech recognition. The neural net was trained on 680,000 hours of multilingual and multitask supervised data collected from the web and is claimed to perform well even on diverse accents and technical language.
• Intel, Arm and Nvidia published a draft specification proposing an industry-wide adoption of the “8-bit floating point (FP8)” standard. The industry has moved from 32-bit precisions to 16-bit, and now even 8-bit precision formats to improve computational efficiency, reduce memory usage, and optimize for interconnected bandwidth. The FP8 format is being selected for its potential to “accelerate” AI development by optimizing hardware memory usage and work for both AI training and inference.
• The IdentiFlight bird detection system that blends AI with high-precision optical technology will be installed across Germany’s new onshore windfarms within the next few weeks. The German government announced the windfarm expansion as a strategy to compensate for the pending loss of nuclear power, coal plants and Russian gas, but the plan is receiving expected backlash from environmental groups concerned with the risk that wind turbines pose to endangered birds. Germany hopes that installing detection systems like IdentiFlight, which slows the rotation of blades when detecting birds, will prevent harm to endangered species and curb conservationist criticisms.

Open Source Highlights

• Comprised of hundreds of security analysts and researchers from across the world, the Trellix Advanced Research Center revealed that an estimated 350K open source projects are at risk due to the CVE-2007-4559 vulnerability. This vulnerability can be exploited by uploading a malicious file generated with two to three lines of simple code and has resided in Python systems for over 15 years. The researchers are urging developers to be educated on all layers of the technology stack to properly prevent the reintroduction of past attacks.
• GitHub's Arctic Code Vault, a giant open-source archive, is finally ready to be buried 250 meters deep within a mountain in Svalbard, Norway. The vault was created with a mission of preserving the world's open-source code for over 1,000 years and contains a 21 terabyte snapshot of all public GitHub repositories before Feb 2020. The nearly 1.5 ton steel box holding the code is covered in AI-generated etchings that “aim to entice future generations to explore it”.
• The Linux Foundation is planning to launch the Open Wallet Foundation (OWF), a collaborative effort to develop open source software to support interoperability for a wide range of digital wallet use cases including digital identity and payments.

Lightning AI Highlights

• You can now build custom physically-accurate synthetic data generation pipelines in the cloud with this Lightning App built with NVIDIA Omniverse Replicator. With it, you can generate physically accurate 3D datasets, complete with ground-truth annotations that can be used to train models – all from your browser. This gives you access to synthetic data much faster and easier than ever before!

Community Spotlight

Want your work featured? Contact us on Slack or email us at olya @lightning.ai

• Pranav Rajpurkar, an AI professor at Harvard, recommends reading code as an effective way to learn AI. He recently shared lecture notes from a computer science course, in which he walks you through the code used to fine-tune a vision transformer using Lightning.
• Bert Gollnick’s article teaches you how to create a neural network in just 100 lines of code, using Lightning. We love the way this article highlights how Lightning makes developing neural networks simpler than ever before.
• Check out Riccardo Musmeci’s repo covering Self-Supervised Learning Methods. This repo implements two of the most important SSL techniques using Lightning.

Don’t Miss the Submission Deadline

• ICLR 2023: 11th International Conference on Learning Representations. May 01-05, 2023 (Kigali, Rwanda). Paper Submission Deadline:?Thu Sep 29 2022 04:59:59 GMT-0700.
• CVPR 2023: The IEEE/CVF Conference on Computer Vision and Pattern Recognition. Jun 18-22, 2023. (Vancouver, Canada). Paper Submission Deadline:?Fri Nov 11 2022 23:59:59 GMT-0800

Upcoming Community Events

• Meetup: Speed up your Machine Learning Applications with Lightning AI at PyData Miami / Machine Learning Meetup September 22, 2022

Upcoming Conferences

• MLNLP 2022: 3rd International Conference on Machine Learning Techniques and NLP. Sep 24-25, 2022 (Toronto, Canada)
• ICIP 2022:?International Conference on Image Processing. International Conference on Image Processing. Oct 16-19, 2022 (Bordeaux, France)
• IROS 2022:?International Conference on Intelligent Robots and Systems. Oct 23-27, 2022 (Kyoto, Japan)
• PyTorch Conference: Brings together leading academics, researchers and developers from the Machine Learning community to learn more about software releases on PyTorch. Dec 2, 2022 (New Orleans, Louisiana)

Want to learn more from Lightning AI? “Subscribe” to make sure you don’t miss the latest flashes of inspiration, news, tutorials, educational courses, and other AI-driven resources from around the industry. Thanks for reading!