Artificial Intelligence #63

Hey, in this issue: CPU algorithm trains deep neural nets up to 15 times faster than top GPU trainers, a robot that taught itself to walk, deep learning model compression explained, researchers converted pre-trained transformers into highly-accurate and fast RNNs, machine learning operations (MLOps) explained, and more.

This issue is sponsored by Lambda Labs and iMerit.

• Researchers converted pre-trained transformers into RNNs, lowering memory cost while retaining high accuracy
• CPU algorithm trains deep neural networks up to 15 times faster than top GPU trainers (paper, code)
• Google open-sourced Lyra, a new audio codec using machine learning to produce high-quality voice calls (code)
• This robot taught itself to walk
• [Sponsored] Lambda GPU Cloud — Start Training in Minutes

Save up to 50% on GPU compute. RTX 6000 & V100 instances available. Get started quickly with TensorFlow, PyTorch, CUDA, and cuDNN pre-installed.

• Deep learning model compression (on quantization, pruning, DeepSpeed, ZeRO-Offload, and knowledge distillation)
• Shedding light on fairness in AI with a new data set
• [Sponsored] Noisy labels impact ML models: read iMerit’s latest post on minimizing the impact of labeling errors on your ML systems
• What is MLOps? Machine learning operations explained
• [Explained] Understanding BigBird's block sparse attention and how it can handle sequence lengths of up to 4096 through

The newsletter is now read by more than 350,000 subscribers. If you are building an AI or a data product or service, you are invited to join these great companies and sponsor one of the future newsletter issues. Feel free to reach out to [email protected] for more details on sponsorships.

Enjoy the newsletter? Help us make it bigger and better by sharing with your colleagues and friends.

Have a nice weekend! See you next week. — Andriy