Speeding up Gen AI + New Models + Securing Data

Celestial AI, Astera Labs, Eliyan, NeuReality, Databricks, Jamba, Observe, BigID

Tons of new funding for AI infrastructure...

Celestial AI Secures $175 Million in Series C Funding for Photonic Fabric Technology

Celestial AI , creator of the Photonic Fabric? optical interconnect technology platform, has successfully raised $175 million in Series C funding led by the US Innovative Technology . The funding round saw participation from a mix of new and existing investors, including AMD Ventures, Koch Disruptive Technologies , and Samsung Catalyst Fund , among others. Thomas Tull 's US Innovative Technology has been aggressive in backing AI infrastructure - they most recently led Lambda 's $320 mm Series C.

The company's Photonic Fabric technology addresses the critical challenges faced by hyperscalers in the wake of the explosive growth in generative AI applications and next-generation data centers. These challenges include utility power availability, memory capacity, and high operational costs. Celestial AI's solution revolutionizes memory and compute fabrics, providing the foundation for advancements in AI while maintaining scalable, sustainable, and profitable business models.

The funding will enable Celestial AI to execute multiple large-scale customer collaborations focused on commercializing its Photonic Fabric technology platform. This platform is becoming the de facto standard for Accelerated Computing optical interconnectivity due to its unparalleled performance and efficiency.

Thomas Tull, Chairman of USIT, and Chase Koch, Founder of KDT and Executive Vice President at Koch Industries, both emphasized the importance of Celestial AI's work in maintaining the U.S.'s position as a global leader in AI innovation and addressing sustainability concerns in processing AI workloads.

Celestial AI's claims its Photonic Fabric is the industry's only optical connectivity solution that allows for the disaggregation of compute and memory, enabling more effective scaling of each component. The Databricks, a data intelligence company, recently announced the open-sourcing of DBRX, a general-purpose large language model (LLM) developed by its Mosaic Research team. DBRX outperforms other established open-source models on standard benchmarks and is designed to be easily customizable for enterprises to improve the quality of their AI applications.

Astera Labs Goes Public, IPO Soars 54%

Astera Labs , which pioneered connectivity hardware essential for cloud computing data centers, took advantage of public markets' massive appetite for AI hardware stories with a splashy IPO last week. The company's share prices leaped 54% on debut, giving the company a valuation of about $8.9 billion. The IPO, priced at $36 per share, outperformed expectations. Astera Labs reported a revenue growth of 45% in 2023 to $115.8 million, attributed largely to the AI-driven expansion in data centers.

Eliyan Raises $60 Million in Series B for Innovative Chiplet Interconnect Technology

Eliyan Corporation a semiconductor technology company building a high-performance chiplet interconnect announced a $60 million Series B funding round co-led by Samsung Catalyst Fund and Tiger Global Management . The round also saw participation from existing investors, including Intel Capital , SK hynix , Cleveland Avenue , and Mesh Ventures , among others. This latest funding follows the company's $40 million Series A round in 2022.

Eliyan plans to use the additional investment to focus on the challenges associated with designing and manufacturing advanced AI chips using multi-die architectures in advanced packaging or standard organic substrates. The company's chiplet interconnect technology enables chipmakers to achieve new levels of performance and power efficiency.

In addition to addressing die-to-die interconnects in chiplet-based designs, Eliyan's Universal Memory Interface (UMI?) targets the "memory wall" issue in large, multi-die designs. UMI offers a bandwidth-efficient connection to memory in both standard organic substrates and advanced packaging, providing increased aggregate memory bandwidth per AI chip and significant die area reduction for memory interfaces.

Eliyan's NuLink? PHY was recently taped out on TSMC's 3nm process, aiming for industry-leading performance of up to 64Gbs per link with unprecedented performance-to-power ratio.

Investors in Eliyan, such as Samsung, Intel, and others, have expressed confidence in the company's approach to multi-chip architectures and believe it will address the challenges of high costs, low yield, power consumption, manufacturing complexity, and size limitations. Eliyan's NuLink technology is now commercially ready and has achieved tape-outs in the most advanced processes, optimized for delivering high bandwidth, low latency, and low power capabilities.

NeuReality Secures $20 Million for AI-Centric Data Center Innovations

NeuReality , NeuReality, an AI inference and data center infrastructure startup based in Israel, has raised $20 million in a funding round. The round was supported by several investors including the European Innovation Council (EIC) Fund, Varana Capital , Cleveland Avenue , XT Venture Capital , and OurCrowd . This latest round brings NeuReality's total funding to $70 million.

The company produces the NR1-M AI Inference Module and NR1-S AI Inference Appliance, both of which feature the company's proprietary NR1 Network Addressable Processing Unit (NAPU) system-on-chip. The new funding will be used to accelerate the deployment of the NR1-M system to a wider customer base.

NeuReality was founded in 2019 by industry veterans Tzvika Shmueli, Yossi Kasus, and Moshe Tanach. The founders have extensive experience in the technology sector, having held senior engineering roles at companies like Marvell, Intel, Mellanox Technologies, and Habana Labs.

According to NeuReality CEO Moshe Tanach, the company's unique AI inference technology is not constrained by traditional CPUs, GPUs, and NICs. Instead, they have re-imagined the ideal AI inference system from the ground up to deliver improved performance, cost savings, and energy efficiency.

The company aims to provide an "express lane" for large AI pipelines, efficiently routing tasks to specialized AI devices and quickly delivering results to customers while conserving resources and capital.

...and two exciting new models

Databricks Launches DBRX: Setting a New Standard for Open Source Large Language Models

Databricks, announced the open-source launch of its latest general-purpose large language model (LLM) DBRX, developed by its Mosaic Research team. Databricks acquired MosaicML for $1.3 bn in June last year.

The Databricks team claims DBRX outperforms other established open-source models on standard benchmarks and is designed to be easily customizable for enterprises to improve the quality of their AI applications.

As has become standard for launches of this kind, the announcement includes several charts showing the superiority of DBRX over other open source models. I think the claims of superiority should be taken with a pinch of salt. I absolutely do not mean to disparage the quality of the model - I just believe current standardized benchmarks don't go far enough in capturing the true strengths and weaknesses of the current crop of SOTA models. There is a wide variability in performance dependent on use case, tuning, data and context.

DBRX claims significant advancements, outperforming models such as LLaMA2-70B, Mixtral, and Grok-1 on language understanding, programming, math, and logic. DBRX is meant to showcase the wide array of capabilities of the Databricks platform. Enterprises utilize its long context abilities in RAG systems, and create custom DBRX models with private data. The model's training process leveraged Databricks tools, including Unity Catalog, Apache Spark, Lilac AI, Mosaic AI Training service, and Inference Tables.

A few days before the launch of DBRX, Databricks also announced its acquisition Lilac, a data integration tool designed to help data scientists search, cluster, and analyze text datasets with a focus on generative AI. Lilac aims to streamline data exploration and understanding in the age of GenAI, making it easier for users to interact with unstructured text data. Designed with an intuitive interface and AI-augmented features, Lilac enables data scientists and researchers to explore data clusters, derive new data categories, and tailor datasets based on insights gained.

The open-source project has gained popularity within the data science and AI research communities, including the Mosaic AI team at Databricks. The founders of Lilac, Daniel Smilkov and Nikhil Thorat, bring a wealth of experience in developing enterprise-scale data quality solutions, having spent a decade at Google.

The integration of Lilac into Databricks is expected to enable customers to accelerate the development of production-quality generative AI applications using their own enterprise data. Databricks Mosaic AI aims to provide end-to-end tooling to develop high-quality GenAI applications, and Lilac's technology will simplify the evaluation and monitoring of LLM outputs, as well as prepare datasets for RAG, fine-tuning, and pre-training within a unified platform.

AI21 Introduces Jamba

AI21 Labs announced Jamba, a production-grade model that combines Mamba Structured State Space model (SSM) technology with elements of the traditional Transformer architecture. Jamba addresses the limitations of pure SSM models and offers a 256K context window, resulting in significant improvements in throughput and efficiency. The model outperforms or matches other state-of-the-art models in its size class on various benchmarks.

Jamba's release marks two milestones in Large Language Model (LLM) innovation: the successful integration of Mamba alongside the Transformer architecture and the advancement of the hybrid SSM-Transformer model to production-grade scale and quality.

Conventional Transformer architecture, while powerful, presents drawbacks such as a large memory footprint and slow inference as context grows. Mamba, proposed by researchers at Carnegie Mellon and Princeton Universities, addresses these issues but struggles to match the output quality of the best existing models on recall-related tasks.

Jamba's Joint Attention and Mamba (Jamba) architecture combines Transformer, Mamba, and mixture-of-experts (MoE) layers to optimize memory, throughput, and performance simultaneously. Jamba is released with open weights under Apache 2.0 and is available on Hugging Face and the NVIDIA API catalog.

Jamba's hybrid structure required several architectural innovations to scale successfully. Its blocks-and-layers approach allows for the integration of two architectures, with each Jamba block containing either an attention or Mamba layer followed by a multi-layer perceptron (MLP), resulting in a ratio of one Transformer layer out of every eight total layers.

The use of a mixture-of-experts (MoE) increases the total number of model parameters while maintaining efficient inference by streamlining the number of active parameters. This results in higher model capacity without significantly increasing compute requirements. The number of MoE layers and experts were optimized to maximize quality and throughput on a single 80GB GPU, leaving enough memory for common inference workloads.

Jamba demonstrates impressive performance in throughput and efficiency, outperforming Transformer-based models like Mixtral 8x7B with 3x throughput on long contexts. Its ability to fit 140K context on a single GPU enables more accessible opportunities for deployment and experimentation compared to other open-source models of similar size.

Future optimizations, such as better MoE parallelism, faster Mamba implementations, and other improvements, are expected to further enhance Jamba's performance.

Observe Secures $115 Million in Series B Funding to Enhance Data Observability Solutions

Observe, a data observability platform, announced the successful completion of a $115 mm Series B funding round. This round was led by Sutter Hill Ventures and saw participation from existing investors Capital One Ventures and Madrona with Snowflake Ventures joining as a new investor. The investment positions Observe's valuation between $400 million and $500 million, according to sources reported by TechCrunch, although the company has not officially disclosed the valuation figure.

Observe's ARR increased 171%, TCV increased 194%, and NRR, which indicates product stickiness, increased to 174% in FY2024. The company's headcount grew by over 50%, and it is expanding its sales organization amid a tech industry slowdown. Observe's modern architecture, featuring a data lake and powerful data graph, helps businesses troubleshoot distributed applications more efficiently and at a lower cost.

Investors are enthusiastic about Observe's innovative approach and rapid growth in the fast-growing observability market. The funding will enable Observe to expand its market presence in North America and potentially more than double its business size. Recent product releases like Project Hubble and Trace Explorer position Observe for continued momentum in the observability space.

BigID Closes $60 Million Growth Round to Expand AI Data Security and Compliance Solutions

BigID , a leader in AI-enhanced data security, compliance, and privacy for modern enterprises, has successfully secured $60 mm in a growth funding round. The round was led by Riverwood Capital , with participation from Silver Lake Waterman and Advent .

BigID has made significant strides since its market debut in 2018, quickly approaching $100 million in recurring revenue. The company's innovative approach in addressing complex data security challenges across various data types and environments has garnered significant customer ROI and positioned BigID as a key player in enabling responsible and trusted AI applications for enterprises. With this latest round of funding, BigID aims to accelerate its growth trajectory, leveraging both organic and inorganic expansion strategies to further solidify its stance as a category leader amidst an evolving regulatory landscape and increasing cybersecurity risks influenced by the adoption of AI and cloud technologies.

Other Stuff

DevOps platform GitLab announced the acquisition of Oxeye (Part of GitLab) , a SAST provider.

微软 announced the acquisition, err, hiring of the founders of Inflection AI . Microsoft is also a big backer of Inflection AI and its announcement was made in conjunction with a post from Inflection suggesting things had changed, but not quite? It is all very confusing. Check out TechCrunch's entertaining take here.

If you found this useful, please share, comment and like the newsletter - it helps us grow!

To continue receiving updates, please click here to subscribe to InfraRead.

My day job is advising growing companies on fundraising and M&A. If you are an investor or entrepreneur I'd love to connect.