The Robots Are Coming

While some would argue that we haven't yet reached the golden age of humanoid robots, this week gave us a glimpse into what the future could look like with new updates from Sanctuary AI's Phoenix and Tesla's Optimus. Sam Altman urged for AI regulation in a U.S. Senate hearing, Sebastian Raschka's latest course is about accelerating deep learning training, and we are gathering with +1K members of NYC's AI community tomorrow. Let's dive in!

Research Highlights:

• Salesforce researchers introduced InstructBLIP, a method aimed at enhancing the computer's comprehension of visual and textual information.The researchers assert that InstructBLIP models surpass previous counterparts, including BLIP-2 and Flamingo models, across various tasks using 26 diverse datasets. These models are also claimed to successfully integrate instruction-aware visual feature extraction to enhance their interpretative capabilities. The InstructBLIP models have been made open-source on GitHub for further exploration.
• Researchers in Germany developed HumanRF, a 4D dynamic neural scene representation, which claims to accurately capture the appearance and motion of human actors from multiple viewpoints. Their approach aims to allow for realistic playback from previously unseen angles, making it valuable in various applications like film production, video games, and videoconferencing. The team also introduced ActorsHQ, a multi-view dataset containing high-fidelity, per-frame mesh reconstructions at an impressive resolution of 12MP, highlighting the challenges and demonstrating the effectiveness of HumanRF in leveraging this high-resolution data for superior novel view synthesis.
• Stanford researchers conducted a comprehensive analysis of the costs associated with querying popular large language models (LLMs) such as GPT-4, ChatGPT, and J1-Jumbo. They discovered significant variations in pricing structures, with fees differing by orders of magnitude. In response, the researchers propose three strategies—prompt adaptation, LLM approximation, and LLM cascade—to mitigate the expense of using LLMs, ultimately presenting FrugalGPT as a flexible solution that reduces costs by up to 98% while maintaining performance comparable to state-of-the-art models like GPT-4. This research aims to provide valuable insights and techniques for the sustainable and efficient utilization of LLMs.

ML Engineering Highlights:

• Sanctuary AI, a Vancouver-based firm, unveiled Phoenix, a 5'7" humanoid robot weighing 155 pounds that aims to augment or replace humans. The robot is capable of lifting payloads up to 55 pounds and has complex hands with 20 degrees of freedom for fine manipulation. Sanctuary envisions a future where general-purpose robots like Phoenix are as ubiquitous as cars, assisting with various work tasks using its proprietary AI control system, Carbon.
• During Tesla's shareholder meeting, Elon Musk showcased new footage of the Tesla Bot, a humanoid robot that is now able to walk forward steadily and perform tasks like object recognition and picking up items. The video highlighted updates such as motor torque control, environment discovery, AI training from human movements, and object manipulation. The progress indicates that Tesla Bot is moving closer to becoming a marketable product, surpassing its initial prototype stage.
• Sam Altman, CEO of OpenAI, testified before a US Senate committee on the the potential risks of AI on the society at large. Altman emphasized the importance of congressional action, citing the missed opportunity to regulate social media, and called for new frameworks and regulations, including licensing, testing requirements, and independent audits for AI companies like OpenAI. Senators from both parties showed support for regulating the industry, although they also questioned the ability of an agency to keep up with the rapidly evolving technology.

Open Source Highlight

?TOMORROW!?

+1,000 members of the AI community of NYC are gathering to celebrate the power of open-source AI in our meetup with Stability AI !

Don't miss it ??https://partiful.com/e/AqsGTfRFmbCgIlMMnbEk

???May 19th | 6 PM

??NYC | Secret Location

?? Show off your AI demos, be inspired and join the movement to democratize AI!

Tutorial of the Week

Ready to accelerate your deep learning training? Start Unit 9 of Sebastian Raschka’s Deep Learning Fundamentals course.

Learn about:

• Mixed-precision training: using 16-bit & 32-bit floats to reduce memory & increase speed
• Multi-GPU training: data parallelism & model parallelism
• Performance tips: torch.compile, optimizing models for speed
• Batch size vs. training throughput: balancing efficiency & model performance

Don’t Miss the Submission Deadline

• ICCVS 2023: The 14th International Conference on Computer Vision Systems. Sep 27 - 29, 2023. (Vienna, Austria). Submission Deadline: Mon May 29 2023
• AI World Barcelona 2023: International Conference?dedicated to the field of?generative?AI?and?autonomous?agents. September 7 - 8, 2023. (Barcelona, Span). Submission Deadline: Wed Jun 07 2023 16:59:59 GMT-0700
• CoRL 2023: International conference focusing on the intersection of robotics and machine learning. Nov 6 - 9, 2023. (Atlanta, Georgia). Submission Deadline: Fri Jun 09 2023 04:59:00 GMT-0700
• ACML 2023: The 15th?Asian Conference on Machine Learning. Nov 11 - 14, 2023. (Istanbul, Turkey). Submission Deadline: Sat Jun 24 2023 04:59:00 GMT-0700
• ICMLA 2023L: The 22nd International Conference on Machine Learning and Applications. Dec 15 - 17, 2023. (Jacksonville, Florida). Submission Deadline: Sat Jul 15 2023

Want to learn more from Lightning AI? “Subscribe” to make sure you don’t miss the latest flashes of inspiration, news, tutorials, educational courses, and other AI-driven resources from around the industry. Thanks for reading!