Microsoft’s New Phi-3 Small Language Models Pack A Mighty Punch, & More
Want more research from the ARK Team? Have feedback on our publications? Click here to help inform our content creation .
1. Microsoft’s New Phi-3 Small Language Models Pack A Mighty Punch
By: Jozef Soja
Last week, Microsoft released Phi-3 ,[1] the third generation in their family of small language models (SLMs). Trained on a mix of curated educational materials and synthetic data, Phi-3's models are setting a new standard on general knowledge benchmarks like MMLU (Massive Multitask Language Understanding) relative to similarly sized models like Meta's Llama 3 8B and Alphabet's Gemma 7B. Phi-3 Mini, Phi-3's smallest 3.8 billion parameter model, nearly matched the MMLU scores of much larger flagship models from 2022 and 2023, scoring 68.8% compared to Llama 2 70B's 68.9% [2] and GPT-3.5's 70.0% .[3]
Amid the broad-based and rapid improvement of language models, the size of SLMs is addressing an increasingly distinctive and important requirement. Phi-3's variants, for example, have only 3.8–14 billion parameters, while large language models like GPT-4 and Llama 3 400B have more than 1.7 trillion [4] and 400 billion [5] parameters, respectively. In other words, relative to Phi- Mini, they are 400x and over 100x larger, respectively. Smaller sizes allow SLMs to be inferenced and fine-tuned less expensively and on a wider range of devices, including consumer laptops and smartphones .[6] In our view, as their performance improves, the low-cost and portability of SLMs should help AI run locally on everyday smart electronics.
That said, small models lack some planning and reasoning capabilities that are important for more complex tasks. As a result, hybrid deployments leveraging both large and small models should prevail in the fierce competition ahead.
2. Robotaxis Took Center Stage In Tesla's Recent Earnings Call
During its earnings call last week, Tesla sounded [7] more confident than ever about its autonomous technology. Ashok Elluswamy, head of Tesla’s Autopilot effort, explained that the latest version of Tesla’s Full Self Driving (FSD) software—including a shift from manually-tuned models to more robust neural networks that learn from video data—will enable the company to predict its progress in scaling model architecture, sizes, data, and compute with increasing accuracy.
Management also seemed confident that Tesla will update its vehicle fleet to offer distributed inference-as-a-service. While integrating distributed inference-as-a-service into personally owned vehicles could present challenges—like unstable Wi-Fi connectivity and bottlenecks in scheduling during owner downtime—a robotaxi fleet could address some of the issues. Dedicated infrastructure for charging with stable internet access and organized vehicle driving hours could result in an organizational structure supporting inference services.
ARK’s analyses suggest that robotaxi revenues could scale from very low levels now to $8-10 trillion in the next five to ten years. In our open-source Tesla model, we estimated [8] that robotaxis will contribute more than two-thirds of Tesla’s future enterprise value.
Consider the potential transformation: while the Model 3 typically sells for ~$40,000, a one-time prepayment for transportation, a Model 3 integrated into Tesla’s ride-hail service could generate $100,000 in recurring revenue per year. If each robotaxi were to transport passengers more than 100,000 miles per year, increased vehicle utilization at scale should lower vehicle operation costs per mile. As a result, Tesla could charge just $1 per mile, significantly less than Uber charges, expanding the market for ride-hail services significantly. While Waymo has led the way toward autonomous ride-hail, data advantages and vertical integration could help Tesla scale more quickly in this winner-take-most market. For more analysis, stay tuned for ARK’s updated Tesla valuation model.
领英推荐
3. Runes Gave Miners A Valuable Lifeline After The Bitcoin Halving
By: Lorenzo Valente
A plethora of experiments aims to leverage the Bitcoin blockchain beyond customary transactions. Some enthusiasts hope to enhance usability of the Bitcoin blockchain with Ordinals, or BRC-20s, and Bitcoin Layer 2s. Launched in tandem with Bitcoin’s halving last week, Runes are enabling efficient ways to create fungible tokens on Bitcoin. Developed by Casey Rodarmor, the founder of Ordinals, Runes created a post-halving frenzy that prompted people to pay exorbitant fees to mint and buy the new fungible tokens. For several days, Runes comprised the majority of transactions on the Bitcoin blockchain, as shown below.
The Runes initiative was an unexpected blessing for Bitcoin miners, generating $135 million USD in miner fees ,[10] which more than offset the 50% cut in traditional block subsidies during the first week post-halving. While creating excitement in the short term, memecoin activity, the primary use case of Runes, is unlikely to sustain the momentum but it does give miners the hope and reassurance of new revenue streams ahead.
Want more research from the ARK Team? Have feedback on our publications? Click here to help inform our content creation .
[1] Beatty, S. 2024. “Tiny but mighty: The Phi-3 small language models with big potential.” Microsoft.
[2] Touvron, H. et al. 2023. “Llama 2: Open Foundation and Fine-Tuned Chat Models.” arXiv.
[3] OpenAI et al. 2024. “GPT-4 Technical Report.” arXiv.
[4] Patel, D. and Wong, G. “GPT-4 Architecture, Infrastructure, Training Dataset, Costs, Vision, MoE.” Semianalysis.
[6] Abdin, M. et al. 2024. “Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone.” arXiv.
[7] Seeking Alpha Transcripts. 2024. “Tesla, Inc. 2024 Q1 – Results – Earnings Call Presentation.”
[8] Keeney, T. 2023. “ARK’s Expected Value For Tesla In 2027: $2,000 Per Share.” ARK Investment Management LLC.
[9] Dune. 2024. “Runes Ecosystem [Work In Progress...]”
[10] Chawla, V. 2024. “Runes generate over $135 million in fees in first week on Bitcoin network.” The Block.