A Closer Look at Etched and the World's First Transformer ASIC

Thank you for clicking on our newsletter, Arbisoft Next. Before we dive into the topic, if you haven't already subscribed, please do so to stay updated on the latest tech and Arbisoft news.

If you're interested in partnering with us, contact us here. Our team of over 900 members across five global offices specializes in Artificial Intelligence, Traveltech, and Edtech. Our partner platforms serve millions of users daily.

We’re always excited to connect with people who are changing the world. Get in touch!

Founded with a vision to revolutionize AI hardware, Etched has consistently pushed the boundaries of what's possible in the hardware field. Etched has created something amazing called Sohu, the first Transformer ASIC (Application-Specific Integrated Circuit). This new chip is made to improve the performance of AI models, especially those based on transformer architectures.?

What is Sohu?

The World's First Transformer ASIC - Sohu is an AI chip designed specifically for running transformer models, which are at the heart of many state-of-the-art AI applications, including natural language processing (NLP) and generative AI.?

Traditional general-purpose chips, such as GPUs (Graphics Processing Units), are versatile and capable of handling a wide range of tasks. However, this versatility comes at the cost of efficiency when dealing with specialized tasks. GPUs are designed to perform many types of computations reasonably well, but they are not optimized for the specific needs of transformer models, which involve complex operations like attention mechanisms and matrix multiplications that require high memory bandwidth and parallel processing capabilities.

Sohu, on the other hand, is tailored to the unique demands of transformer architectures. This specialization allows for significant performance improvements in terms of speed, power efficiency, and throughput. Sohu stands out due to its impressive performance metrics, such as:

• Speed and Efficiency: Sohu can perform up to 1,000 trillion operations per second while consuming only 10 watts of power. This makes it significantly faster and more energy-efficient than current leading GPUs, such as Nvidia's H100.
• Scalability: Sohu supports various AI models, including Llama and Stable Diffusion 3, and can handle over 500,000 Llama 70B tokens per second, and a single eight Sohu server will replace 160 H100s.
• Real-time Processing: The chip excels in real-time applications, such as voice agents and content generation, by leveraging multicast speculative decoding and tree search.

Strategic Importance

Etched's Sohu chip is positioned to challenge the AI hardware market, which Nvidia currently dominates. With a recent $120 million funding round led by Primary Venture Partners and Positive Sum Ventures, Etched aims to expand its operations and accelerate the development of its specialized AI chip. The chip has already garnered significant interest, with several unnamed customers placing orders worth tens of millions of dollars.

Market Context and Future Prospects

The AI chip market is ever-evolving and competitive, with many new and established companies trying to take the lead. Although Nvidia is the main player, the rise of specialized chips like Sohu shows a growing need for hardware made for specific AI tasks. Companies like Cerebras Systems and Taalas Inc. are also creating custom AI chips, increasing the competition.?

However, Etched faces significant challenges. The success of Sohu depends on the continued use of transformer models, which some experts think might be replaced by better architectures in the future. But for now, since transformers are widely used in AI research and industry applications, Sohu's launch is timely and potentially game-changing.

Conclusion

Etched's Sohu chip represents a significant leap in AI hardware technology, offering unmatched speed and efficiency for transformer-based models. With substantial funding and strategic market positioning, Etched is poised to challenge the dominance of traditional GPU manufacturers like Nvidia. As AI technology continues to evolve, specialized solutions like Sohu will play a crucial role in shaping the future of AI hardware.

Sources

• MarkTechPost
• Feed The AI
• Etched Transformer
• SiliconANGLE
• Bloomberg
• Analytics India Magazine
• Etched's official page
• Tom's Hardware?
• The Register