LLM Pulse - Jan 02, 2025

New Releases & Updates

• Infinigence AI Releases Megrez-3B-Omni: A 3B On-Device Open-Source Multimodal Large Language Model MLLM

The integration of artificial intelligence into everyday life faces notable hurdles, particularly in multimodal understanding—the ability to process and analyze inputs across text, audio, and visual modalities. Many models require significant computational resources, often relying on cloud-based infrastructures. Read More

• Deepseek unveils Deepseek V3 AI LLM with free chatbot access

DeepSeek, a Chinese AI start-up, has made headlines with the release of its advanced large language model, DeepSeek V3. The model, boasting 671 billion parameters, outperformed prominent AI models like Meta’s Llama 3.1 and OpenAI’s GPT-4o in benchmark tests evaluating text understanding, coding, and problem-solving. This achievement is a major step for China's AI industry. Read more?

• China's GenAI market continues to heat up as Beijing records more? LLM? filings

The latest batch of GenAI services registered in Beijing included large language models from Zhipu AI and Xiaomi affiliate Rigo Design. Beijing added 11 new generative artificial intelligence (GenAI) services set for public release, as the nation’s capital continues to burnish its reputation as the country’s leading hub for the technology ’s development. Read More

• Buyer beware: OpenAI’s o1 large language model is an entirely different beast

Subscribe to GeekWire Newsletters today!GeekWire’s startup coverage documents the Pacific Northwest entrepreneurial scene. Sign up for our weekly startup newsletter, and check out the GeekWire funding tracker and venture capital directory. Read More

• Quantum: An AI Stock Making LLM Training Cheaper

Quantum helps enterprises manage their unstructured data in today's GenAI world. Quantum fulfils this task by designing robust and cost-efficient solutions for business for the entire data lifecycle. Furthermore, Quantum helps its clients to extract key and valuable pieces of data from a pool of raw and unstructured data, giving its clients a competitive advantage. Read More

• IBM Wants to Be the Enterprise LLM King With Its New Open-Source Granite 3.1 Models

IBM is staking its claim at the top of the open-source AI leaderboard with its new Granite 3.1 series out today. The Granite 3.1 large language models (LLMs) offer enterprise users extended context length of 128K tokens, new embedding models, integrated hallucination detection and improved performance. Read More

• Writer launches new Palmyra Creative LLM

Generative AI platform provider Writer on Tuesday introduced Palmyra Creative, the latest addition to its family of large language models. Palmyra Creative is designed to help teams brainstorm fresh ideas, according to Writer. Read More

• Google DeepMind Introduces ‘SALT’: A Machine Learning Approach to Efficiently Train High-Performing Large Language Models using SLMs

Google Research and Google DeepMind researchers introduced a novel approach called Small model Aided Large model Training (SALT) to address the above challenges. This method innovatively employs smaller language models (SLMs) to improve the efficiency of LLM training. Read More

• Meet Moxin LLM 7B: A Fully Open-Source Language Model Developed in Accordance with the Model Openness Framework (MOF)

The rapid development of Large Language Models (LLMs) has transformed natural language processing (NLP). Proprietary models like GPT-4 and Claude 3 have set high standards in terms of performance but often come with drawbacks such as high costs, limited accessibility, and opaque methodologies. Read More

• IBM's new enterprise AI models are more powerful than anything from OpenAI or Google

IBM is zooming along with new open-source Granite Large Language Models (LLM) releases every few months. Granite 3.1 is the latest generation model, building upon the success of Granite 3.0. The model offers enhanced capabilities and performance optimized for business applications. Read More

Research and Technology

• An introduction to preparing your own dataset for LLM training - AWS Machine Learning Blog

Large language models (LLMs) have demonstrated remarkable capabilities in a wide range of linguistic tasks. However, the performance of these models is heavily influenced by the data used during the training process. Read More

• Slim-Llama is an? LLM? ASIC processor that can tackle 3-billion parameters while sipping only 4.69mW - and we'll find out more on this potential AI game changer very soon

Slim-Llama is an LLM ASIC processor that can tackle 3-billion parameters while consuming only 4.69mW- - and we'll find out more on this potential AI game changer very soon.? Read More

• Microsoft's search engine Bing transitions from Transformer to a combination of LLM and SLM & announces integration of TensorRT-LLM

Microsoft has been using the machine learning model ' Transformer ' developed by Google for its search engine Bing. However, Microsoft has announced that it will move to a combination of large language models (LLM) and small language models (SLM) as Transformer has reached its limits. In addition, Microsoft has announced that it will optimize searches by integrating ' TensorRT-LLM ' developed by NVIDIA into its workflow. Read More

• AI language model runs on a Windows 98 system with Pentium II and 128MB of RAM — Open-source AI flagbearers demonstrate Llama 2? LLM? in extreme conditions

EXO Labs has demonstrated a modern LLM running on a 26-year-old Windows 98 PC. EXO Labs has penned a detailed blog post about running Llama on Windows 98 and demonstrated a rather powerful AI large language model (LLM) running on a 26-year-old Windows 98 Pentium II PC in a brief video on social media.? Read More

• ReDrafter: Apple and NVIDIA collaborate on artificial intelligence

Apple and Nvidia have officially announced that they are working hand in hand to advance large language model (LLM) technology. The first results revealed already show significant progress, highlighting the effectiveness of this strategic collaboration. Read More

• Data extraction from polymer literature using large language models

Automated data extraction from materials science literature at scale using artificial intelligence and natural language processing techniques is critical to advance materials discovery. However, this process for large spans of text continues to be a challenge due to the specific nature and styles of scientific manuscripts. Read More

• Accelerating LLM Inference on NVIDIA GPUs with ReDrafter

Accelerating LLM inference is an important ML research problem, as auto-regressive token generation is computationally expensive and relatively slow, and improving inference efficiency can reduce latency for users. In addition to ongoing efforts to accelerate inference on Apple silicon, we have recently made significant progress in accelerating LLM inference for the NVIDIA GPUs widely used for production applications across the industry. Read More

• The Role of Specifications in Modularizing Large Language Models

Software has been a critical catalyst for economic growth over the past several decades, a phenomenon prominently articulated by Andreessen in his influential blog post, “Why software is eating the world.” Read More

• Memory Challenges in LLM Serving: The Obstacles to Overcome

Although fine-grained batching reduces the waste of computing and enables requests to be batched in a more flexible way, the number of requests that can be batched together is still constrained by GPU memory capacity, particularly the space allocated to store the KV cache. Read More

• How Fastweb fine-tuned the Mistral model using Amazon SageMaker HyperPod as a first step to build an Italian large language model

AI’s transformative impact extends throughout the modern business landscape, with telecommunications emerging as a key area of innovation. Fastweb, one of Italy’s leading telecommunications operators, recognized the immense potential of AI technologies early on and began investing in this area in 2019. With a vision to build a large language model (LLM) trained on Italian data, Fastweb embarked on a journey to make this powerful AI capability available to third parties. Read More

Other News?

• Google to use LLM to identify fraudulent websites in Chrome

Google plans to use artificial intelligence (AI) to identify fraudulent web pages in Chrome by analyzing their content and intent, similar to Microsoft's scareware blocker. The Chrome browser has a new experimental feature similar to Microsoft's scareware blocker. In this case, it seeks to combat a threat that occupies the entire browser screen and generates a sense of urgency in the user to give remote access to their computer to the cybercriminal. Read More

• Apple Is in Talks to Use Zhipu AI's? LLM? on iPhones in China

Apple is in discussions with Zhipu AI on using the Chinese artificial intelligence developer's large language model in its iPhones in China, Yicai learned. This development comes on the heels of a Reuters report yesterday stating that Apple is in very early-stage talks with Tencent Holdings and TikTok owner ByteDance about integrating their AI models into iPhones sold in China. Read More

• Rewind 2024: Top enterprise AI trends in vogue

In 2024, the realm of artificial intelligence (AI) witnessed a significant surge with global technology players like?Nvidia,?Google,?Microsoft, and?AWS, among several others consolidating their position in the AI space. Throughout the year, AI and generative AI (GenAI) dominated the technological landscape, with virtually every IT conference, product unveiling, and news event being associated with AI, with experts calling 2024 “a critical year for AI” with organizations exploring how this technological leap can be integrated into daily life and work. Read More

• Alibaba Cloud announces aggressive LLM price cuts in bid to dominate China's AI market

Alibaba Cloud said Tuesday that it’s slashing the price of access to its most advanced large language models by up to 85% in a bid to generate more interest from Chinese businesses. Read More

• Chinese LLM unicorn Zhipu AI pockets over $411m

Beijing-based Zhipu AI, one of China’s top large language model (LLM) startups, has pocketed 3 billion yuan ($411.8 million) in its latest funding. Read More

• Shanghai Sets Up Embodied Intelligence, LLM Funds With USD140.8 Million Each

Shanghai has formed two funds, each with an initial CNY1 billion (USD140.8 million), to invest in embodied intelligence and large language models with the aim of bolstering the city’s position as a global leader in artificial intelligence and other cutting-edge industries. It will invest in related supply chains, including robotics, core components, autonomous driving applications, sensor technology, general robotics, and other areas. Read More

• SemiKong is the world's first open-source semiconductor-focused? LLM? — it claims to bring new chips to market 30% faster

Meta, Aitomatic, and other members of the AI Alliance have released the world's first large language model specifically trained on the needs of the semiconductor industry. SemiKong, a new LLM trained by Aitomatic and its partners in the "AI Alliance", is the world's first large language model specifically crafted to serve the semiconductor industry's needs. Read More

• Meet DeepSeek: the Chinese start-up that is changing how AI models are trained

Chinese start-up DeepSeek has emerged as “the biggest dark horse” in the open-source large language model (LLM) arena in 2025, just days after the firm made waves in the global artificial intelligence (AI) community with its latest release. Read More