Chiplet and Tier 1 IC Supplier to enable AIoT Era

By JingYang JY Zhang, MooreElite Founder & CEO

2019.3.9 San Jose, CASPA 2019 Spring Symposium: AIoT, AI of Things

Abstract: Semiconductor industry has high expectation for AIoT, wish these new applications evolve exactly like Smart Phone, generate high volume, unified IC demand eventually. However, the reality today is all the demands are fragmented, low volume, and hard to justify a full mask cost. Should we just sit back and wait for things to change? Or we should explore ways to solve this mismatch now? In this talk, we will discuss new options that Chiplet and Tier-1 IC suppliers bring to the table and give some thought how this may change IC industry and enable AIoT.

Artificial Intelligence of Things

AIoT stands for Artificial Intelligence of Things, it is a Fusion of AI and IoT, also the next stage of IoT - from simply connected devices to the advanced interaction of smart devices. The fusion of AI and IoT is natural, they need each other, AI needs the data collected by IoT to train algorithms, and IoT needs the thinking model established by AI to make decisions.

There are a lot of smart hardware IoT device out there today, I am a big fan myself, so I almost brought and tried all the smart devices I could get my hands on, Google glass, HoloLens, MagicLeap, smart TV, camera, doorbell, light bulb, robot, and many more. With so many smart hardware at my home, so how is the user experience?

While, the DIY smart home system works fine for me, demo kind of future life style, however, the problem is this system works fine for me ONLY. My nightmare started when my parents moved in , early morning things work fine, but the moment I left home, smart home became a troubled home, the system is so unstable, one tiny disturbance can destroy the whole system. To my parents, one minute early, everything is ok, one minute later, nothing works, they cannot open light, cannot use electric cooker, cannot open the door, Sound Box and robot turn on unexpectedly …

So every night when I return home, I have to reconnect everything and teach my parent again and again; weekend is even worse, because when I stay at home, neither my parents nor my wife want to explore usage of these smart hardware anymore, they use me as a central control unit, as a remote controller, anything they need, they just ask me to do through App on my cellphone or operate for them.

The AIoT today is way too complicated for people to use, if every device needs human intervention to connect, to login, to config, it will be a disaster, and never come to high volume and commercialize. Each IC and IoT devices today need to be redesigned with AI function embedded.

Currently all the AIoT are actually AI Challenged IoT

Currently, all the AIoT are actually AI Challenged IoT, even industry most proud hardware, smart phone fail for this simple scenario, “recommend a restaurant but no Chinese food”, here is what I got with latest model of some $1500 USD smart phone, in next few years, we will still face such awkward scenes.

Why? We can build powerful AI like IBM Watson, Google AlphaGo, why cannot realize this simple chat bot? Early last year, the“AI phone call” demonstrated at Google I/O Conf was amazing, why google assistant in your phone perform so pool even with cloud computing help?

Today AI hardware performance, cost and power consumption all have big gap for everyday use, even if AIoT application can afford AlphaGo level HW cost, power consumption will still too expensive, it costs Google ~ $20K USD of electric fee for that famous game, with Korea GO Champion, how much are you willing to pay for a restaurant recommendation?

There is a joke about How to found an AI startup, there are 3 steps:

1. Hire a bunch of minimum wage humans to pretend to be AI pretending to be human;
2. Tell investors and users you have developed a scalable AI technology while secretly relying on human intelligence;
3. Wait for workable and affordable AI to be invented, and most importantly, keep fund-raising.

There is something wrong

Today, the latest SoC can have over 8 billion transistors, 8,000 times of a bee’s brain neurons number, while this small insect can see, can fly, can find food, find wife, have baby, and the brain power consumption is extremely low; on the other hand, our super computer, expensive, complicated, very high-power consumption.

Biological brain is so efficient because it is application specific, it can only do one thing. But today’s CPU have too many functions, too much flexibility, all these cost too much performance. AIoT won’t be truly commercialized, unless we can achieve enough performance at a fairly low cost and power level.

Need ASIC to Maximize Performance

We all know this chart, in terms of Performance-flexibility trade-off, General Purpose Processors like CPU enjoy the maximum flexibility, but the least power efficiency; ASIC dominants the efficiency and performance. Most of the IC today, even many so-called ASIC, are still multi-Purpose. We are always trying to group as many functions as possible in one IC, so we can cover more market with one tape-out, we also prefer to use commercial IP and foundry stdcell directly, so we can have quicker time to market. But all these short cuts are at cost of performance.

If we push ASIC design to the extreme, only support one function, serve one purpose, you may be surprised for result achieved, take Cryptocurrency Mining as an example, at the same cost level, performance benchmark shows, dedicated ASIC is 733,000X better than CPU, Efficiency is 344,000X. To achieve this, these Cryptocurrency ASIC customize every stdcell, do place & routing by hands, violate foundry DRC rules boldly, use SRAM cell’s way to redesign stdcell and ASIC, anything you can think of, to squeeze all the juice out. some people might say this is a design methodology throwback, waste a lot of time to do no sense work, but Cryptocurrency ASIC do get the amazing performance out, which before been considered as impossible by traditional IC society.

A large number of applications in the past can be solved by general-purpose computing, but in the era of AIoT, the hardware performance has no redundancy and will be bottleneck for a long time, we will always need to drive it to the extreme. Cryptocurrency ASIC story teach us an important lesson, dedicated ASIC can achieve miracle. 

Is ASIC Affordable for all IoT Segment?

While, is dedicated ASIC affordable? Designing a 28nm chip costs about $50 million, 7nm, $300 million. At 3nm, could be incredible $1.5 billion. Very few players in the world can afford this expensive game. Large companies will request a 10 billion-scale market forecast, before entering any specific segment. Even for product like iPhone, with 200M shipment yearly, and highest margin, also has to use many general-purpose ICs to balance between performance and dedicated ASIC cost.

In the other side, although AIoT market in total is huge, each segment is too fragmented to be covered by traditional SoC approaches, all the demands are low volume, hard to justify full mask cost. The wording of “IoT” is very misleading, which group tens of thousands fragmented segment into one word, together in total, it is gold mine, but each segment is barren and poor. Without ASIC to improve performance and user experience, this segment will never grow to hundred million shipment.

More categories, smaller shipments, shorter time-to-market, higher performance, lower cost, how can we achieve this Mission Impossible？

Moore`s Law includes clue - Heterogeneous Integration

When Gordon Moore published his legendary paper - Cramming More Components onto Integrated Circuits, in 1965, he did not only address his prophecy ['pr?fisi], later known as “Moore’s Law”, but also gave us a clue, how to quickly and economically, build large systems out of smaller functions.

Moore wrote: It may prove to be more economical to build large systems out of smaller functions, which are separately packaged and interconnected. Heterogeneous Integration could be a promising way out, for performance-thirsty, fragmented AIoT market.

Chiplet: Lego-like Hardened-IP

Recently, the concept of Chiplets is heating up, From DARPA's CHIPS project to Intel's Foveros, Chiplet is regarded as an important base technology for future semiconductor industry.

To put it simply, Chiplet technology is like building LEGO blocks, by integrating some pre-produced die into a system chip through advanced packaging (such as 3D 2.5D packaging). And these pre-produced dies are called Chiplet. In this sense, Chiplet is a new hardened IP reuse model. 

This idea was first been seriously introduced by Marvell’s founder Sehat at ISSCC 2015, the name MoChi stands for modular chip. In 2017 DARPA lunched new CHIPS program, focused on developing mix-and-match functional chiplets that may be combined as needed. The basic idea is that you have a collection of modular chips, or Chiplets. Then, you assemble the Chiplets into a package and connect them using a die-to-die interconnect scheme. In theory, the Chiplet approach is a fast and less expensive way to assemble various types of third-party chips, such as I/Os, memory and processor cores together.

Pros & Cons of SoCs, Chiplets and PCB

From this chart, we can learn that Chiplet is a compromise between SoC and PCB, the R&D investment and TAT has been greatly reduced. Most importantly, as hardened IP, designers can spend more time to customize stdcell, handmade place & routing, squeeze foundry DRC rules for size reduction, tune process corner for sweet point, achieve amazing performance.

Specially for AIoT market, Chiplets mode offers flexibility of process selection. We can integrate high performance computing realized in advanced nodes and analog/RF in mainstream technologies in one packaging cost-effectively.  

Marvell Intel AMD zGlue explore Chiplet

Since Marvell’s MoChi architecture being introduced 4 years ago, they have built a variety of Virtual SoCs Mochi family. It can be a baseline architecture for a host of applications including IoT, smart TVs, smartphones, servers, notebooks, storages, etc.

In 2017, Intel presented EMIB packaging technology to assemble Chiplets into single packages, together with Intel's AIB standard, Advanced Interface Bus, that enables a modular approach to system design. 3 Months ago, Intel revealed its latest FOVEROS 3D face-face chip stacking, which is said to be in volume production in 2nd half this year.

AMD Unveils its Chiplet Design Approach 2018, as an evolution of the company’s modular design EPYC processors, featuring its Zen micro architecture. The integration involve CPU Chiplets made using 7 nm as well as an I/O die made using a mature 14 nm process. 

Startup zGlue announced a similar effort, as a commercial offering, targeting SoCs for the IoT. zGlue enables affordability and quick development of chips, by utilizing chiplets, 2.5D packaging and a cloud-based design software. Similar to FPGAs’ programmability, zGlue allow programming of inter-chiplet connectivity, chiplet power delivery, security, self-test, and repair function.

If Chiplet comes true, then what next?

In Today’s semi industry value chain, IC design company plays a vital ['vait?l] part, they purchase IP,EDA, wafer from vendors, then design and sell chip to system house, directly or through distributors. If Chiplet becomes a mainstream option, when chiplet standards established, with hundreds of thousands Chiplets ready for pick, what may happen next?

New Structure of SEMI Industry

If we take a look at Automotive Industry, after over one century, now this mature industry has Car OEM, Tier-1, Tier-2, Tier-3 suppliers. The most interesting part is collaboration relationship of Tier-1 and Tier-2 suppliers. Tier-1 suppliers highly rely on Tier-2 technical skills, purchase components from Tier-2 and package it into systems, Tier-2 suppliers make parts and are happy to sell them to Tier-1. Car OEM trust Tier-1 supplier more, and source components from Tier-1 mostly.

Today, these is no Tier-1 Tier-2 structure in semiconductor, traditional IC design company play both roles, they design both whole chip and parts, sell final chip products. Chiplet offering may drive industry to evolve, new species may emerge, we may call it “Tier-1 IC Suppliers” for now, just like automotive industry, Tier-1 IC Suppliers can purchase components from Tier 2 chiplet suppliers and package it into SIP, Tier 2 make chiplet and could be happy to sell them to Tier-1 only, system house will trust Tier-1 IC suppliers more, and source components from Tier-1 mostly. 

Chiplet & Tier-1 IC Supplier Candidates

Who has the best chance to win this new game? Small IC design houses, IP houses, even Foundry, assembling & testing vendors all have the opportunity to become Chiplet suppliers. Eventually, the boundaries between IC design house and IP house will gone, they will all transform into hardened IP companies, and provide Chiplets to market.

If a chip design company can easily profit through Chiplets, without worrying about supply chain, sales and marketing, this will be as light as a working studio, if engineers can make big money as independent design studio, more and more chiplet start-ups will emerge.

On the other hand, which parties will evolve to Tier-1 IC suppliers？Big IC design house, Distributors, module manufacturers may all have a say. In fact, all of them will race to be Tier-1 IC suppliers in order to gain more profit. Take Distributors as an example, they normally earn only 3-5% of chip sales in traditional industry structure; but as Tier-1 Suppliers, buy Chiplet and then integrate them into SIPs, the gross margin can rocket to 50% like IC Design company! Distributors are very sharp on market trend and requirement, once they got powerful weapon like chiplet, they can be very strong competitors to today’s IC Design companies.

New IC Eco-System w/ Tier-1 IC Supplier

After years of commercial practice, the leading IC Design companies have developed efficient internal collaboration model. That is, using Consolidated Manufacture and Operation, supporting hundreds of R&D Departments, then all the products sold using Consolidated Sales & Brand. Each R&D Department is like special force, move fast, share resource, get timely support from Consolidated operation and manufacture, grow $$$ quickly leverage Consolidated sales and brand. Perfect candidate of chiplet supplier.

With the emergence of Tier-1 IC Suppliers and chiplet suppliers, there could be a new IC Eco-system, more open, more dynamic. All the resources and technical capabilities will not necessary belong to one big company but could be Eco-system with many small companies combined. Chiplet Studios will be smaller than today’s IC Design houses, leverage powerful support of Eco-system’s consolidated Manufacture & Operation. Tier-1 IC suppliers’ well-established supply chain, vendor relationships, sales channels and brand are the perfect amplifiers of chiplet Design Studio. Tier-1 IC suppliers and many chiplet studios may can serve AIoT fragmented market better and last longer.

MooreElite: To be Alibaba of Semiconductor

At MooreElite, we have explored a little on the road of building new IC eco-system.

With a goal of making IC design easy & efficient, we are building the future semiconductor infrastructure to enable IC design companies, just like what Alibaba built for e-commerce Industry, Alibaba provide logistic, payment, traffic generation service to enable many small and middle size e-commerce companies to success.

Founded in 2015, now MooreElite has over 250 fulltime employees globally, most of our employees are IC designers, supply chain experts, tape-out, packaging & testing engineers. We provide Design Services, Supply Chain Operation, HR and Enterprise services to over 1,500 Fabless IC clients globally, we are trying to be an Alibaba-like platform to enable small and middle size IC design companies to success.

One-stop Platform for IC Design Studio

Most of the IC startups, especially these in China, are actually a R&D dept or Design studio at most, they are key competence is technology, it is hard and make no sense for these CTO background entrepreneurs to run a full set company, learn how to be sales, how to do operation, how to deal with supply chain, and also it is not economic to hire full function team for a small shipment product.

MooreElite provide one-stop service platform to free these CTO from company sales, operation and supply chain workload. We want to help these IC Design Studio to focus on developing their key advantage, to become top 3 worldwide in their niche market.

Summary

AIoT today is still AI challenged, need ASIC to boost its performance, increase user experience. The AIoT market as a whole is huge, however each segment is too small to use traditional SoC approaches. Chiplet, Tier 1 IC Suppliers and other biz model innovations might pave a new way out. MooreElite is trying to build a one stop platform to enable small and middle size IC design companies to success. 

If we think IC market as gold mine, the first batch gold diggers encountered the PC market, they took the biggest gold nuggets. The second batch diggers came across the mobile phone market and took the rest gold nuggets. Now the 3rd batch diggers meet AIoT and there are no more exposed nuggets, old ways cannot get you gold any more. They have to dig in deep and grind the whole mountain in order to get the gold powder. But guess in gold rush history before, who got the most gold at the end? The 3rd batch diggers!

Today, the semiconductor industry contributes only 0.6% of Global GDP, and there are many AIoT segment markets to be discovered and to be worked on. It will not be easy, old method will not work, you may have to grind a mountain for gold powder, but this could be the best chance to gather the most gold in history.