The Key to Edge-Side Large Model Iteration: ABUP's Differential Upgrade Technology Empowers DeepSeek's Efficient Deployment
The application scenarios for large models are rapidly expanding, transitioning from cloud-based environments to edge devices—a trend that is becoming increasingly prominent.
The rise of large models on edge devices addresses critical needs such as real-time processing, privacy protection, and offline capabilities. This shift is propelled by advancements in hardware performance, the development of lightweight models, and supportive policies, unveiling substantial market potential.
Take Deepseek as an example: within its first month of release, it gained widespread attention across various industries. In the automotive sector, over 20 car manufacturers announced integrations with Deepseek, applying it in areas like in-cabin experiences, operations, and private model optimization. Companies are leveraging post-training and domain-specific data to customize private models, enhancing competitiveness and value.
This phenomenon showcases the formidable potential of large models and prompts industry reflections on model iteration and upgrades. However, as the training of large models accelerates, upgrade challenges are emerging.
Considering compressed packages, attempting to reduce package size through compression is largely futile. This is because large models have complex file structures and massive data volumes, and they are already highly compressed (quantized), making further compression extremely difficult and time-consuming. Consequently, complete OTA (Over-The-Air) upgrade packages are substantial in size, leading to slow transmission and updating processes.
Analyzing model usage and upgrade packages reveals that, compared to the vast amount of model data, iterative data involves only fine-tuning, yet the upgrade requires the entire dataset—a natural contradiction.
Differential upgrades have become an optimal solution for efficient updates within the limited resources of edge devices.
In the context of the Internet of Vehicles (IoV), differential upgrades not only reduce data traffic costs but also enable seamless updates, significantly enhancing user experience. For car manufacturers, this translates to lower operational costs and faster product iteration. More importantly, in environments with no or unstable network connections, differential upgrades can still be efficiently executed, ensuring continuous model optimization.
The value of differential upgrade technology extends beyond the IoV scenario. With the widespread adoption of large models on edge devices, more applications are relying on locally deployed AI models. For instance, voice assistants on smartphones and control systems in smart homes can achieve rapid iteration through differential upgrades.
ABUP's Breakthrough: Differential Upgrade Algorithm Tailored for Large Model Files
Differential upgrades are not a novel concept; they have been widely applied in mobile applications, operating systems, automotive, and other fields. As a leader in the differential upgrade domain, ABUP?shares the optimal outcomes of its latest differential upgrade algorithm tailored for large model files.
By comparing file sizes and differential compression effects under original floating-point and quantized formats, ABUP?has achieved dual breakthroughs in both differential package size and processing time.
Compared to certain open-source algorithms, ABUP's ABDiff-M technology utilizes an intelligent differential algorithm framework capable of precisely identifying differences between large model versions, generating minimal differential packages, and significantly shortening upgrade times, offering a novel solution for the industry.
We selected some of the most popular models in the market to compare an open-source differential algorithm with ABUP's ABDiff-M, yielding the following results:
Differential Package Size: For example, with DeepSeek's 1.5B 4-bit model, ABDiff-M generated a differential package of only 90MB, whereas the open-source algorithm required 400MB. ABUP's differential package size is merely 5.7% of the original package and 22.5% of the open-source algorithm's package. For the 8B 4-bit model, ABDiff-M's differential package size is 0.27GB, while the open-source algorithm needs 1.35GB; ABUP's package is 5.2% of the original and 19.9% of the open-source algorithm's package.
Differential Processing Time: Taking DeepSeek's 1.5B 4-bit model as an example, ABDiff-M's upgrade time is 50% of that of the open-source algorithm. For the 8B 2-bit model, the upgrade time is 51% of the open-source algorithm's time.
With the extensive application of large models on edge devices, the importance of differential upgrade technology is becoming increasingly prominent. ABUP's ABDiff-M algorithm, through its intelligent differential framework and efficient compression technology, not only substantially reduces differential package sizes and upgrade times but also supports multi-threaded restoration and streaming upgrades, significantly enhancing the upgrade efficiency of edge devices.
In the future, with further improvements in hardware performance and continuous advancements in model optimization technologies, the application scenarios for large models on edge devices will further expand. As a key to the iteration of large models on edge devices, differential upgrade technology will play an increasingly important role.
Open Source Data Information:
?