New Trajectories for Memory and Storage

Radical new solutions in memory and storage technologies will be needed for future information and communication technologies (ICT.)

The Global DataSphere

As we discussed in the previous blog of this series, data is “eating the world.” In fact, data is more than a megatrend—it can be viewed as a measure of the progress of humanity, cutting across all walks of life, public and private organizations, and every vertical of the economy.

The amount of data we create as a society is rising in an exponential manner. During 2020 alone, more than 59 zettabytes of data was created and consumed. This was further increased by the COVID-19 pandemic, which caused an upsurge in remote workers that now rely heavily on internet-based programs and video communication, as well as video recording and downloading. Over the next three years, the amount of data created will be more than the data created over the past 30 years combined. And, over the next five years, the world will create more than three times the data than it did in the previous five. These are, of course, enormous numbers as data is the fuel powering high-performance computing, social media platforms (and their reliance on data centers), edge computing (including autonomous driving), Internet of Things, and so many more.

How many bits?

Any attempt to capture the world’s data production requires an understanding of the prefix “zetta-”, which denotes a factor of 10^21, or a “billion trillion.” According to the total bit inventory, mankind processed 1 zettabyte in 2010, somewhere between 10 and 100 zettabytes in 2020, and the world will store around 100,000 zettabytes in 2040. Of course, this is good news for the memory and storage industry. However, this huge number also represents a problematic challenge; global demand for data storage is growing so fast that, in the near future, today’s technologies will not be sustainable due to the sheer mass of material resources needed to support the ongoing data explosion. The fact is, each bit has a weight—meaning, a weight basis in memory device silicon—and storing zillions of bits will necessarily consume zillions of grams of different materials.

Cost per bit: A little bit of history

If we look back into history, magnetic storage devices—both magnetic hard-disc drive (HDD) and magnetic tape—were first to the game, invented ~65 years ago within a year of one another. In 1956, the first HDD featured a capacity of 5 megabyte and a concurrent price of $10,000/megabyte. Thus, the cost of information was 1￠/byte or about 0.1￠/bit (1 byte (B) = 8 bits). Today, HDDs can store 18–20TB at a price as low as 0.2 nano￠/bit. Moving forward, HDD capacities will reach 100TB at a price of ~75 pico￠. Tape has scaled to 12TB per cartridge at a price of 0.1 nano￠/bit and is projected to reach several hundred TB at price points as low as 40 pico￠. Electron-based NAND solid-state drives (SSDs) were introduced in 1991 at an initial capacity of 20MB and a price point of 0.6 mega￠/bit, and are now scaling up to 16TB capacities with a price point of $130/TB with projections reaching 60–80TB capacities in 2030, yielding price points as low as ~0.4 nano￠/bit. While the scaling across HDD, SSD and Tape continues to be remarkable, it is still projected to fall short of expected data storage demand growth.

How much does a bit weigh?

As mentioned previously, it is important to remember that each bit of information is created in a materials system so there is a mass associated with it. As with all man-made things, there is a material price or associated cost.

A bit of NAND Flash storage is among the smallest memory units and can weigh as little as one picogram (10^-12 g)—a typical mass of a bacterium cell! Now, if we multiply this tiny number by 100,000 zettabytes—the total data we expect to store in 2040—it follows that the total mass of silicon wafers required would be approximately 10^10 kg. This, however, significantly exceeds the world’s total available silicon supply! Obviously, we want to make bits as small as possible, but there are fundamental limits to scaling. For example, in electron-based devices, such as flash, quantum mechanics dictates a limit on material size—if the ‘box’ is too small, the electrons will too easily escape, and the information will be lost. This effect is known as quantum-mechanical tunneling and it forbids us from making a silicon bit less than about 10^-12 g. Therefore, we need to use other materials and other mechanisms to make still smaller bits. Thus, it could also help in this data-storage exercise to revisit the scaling limits of magnetic bits, both for HDD and tape. Additionally, optical storage may offer new surprises beyond disc drive technologies like the DVD (e.g., Microsoft’s Project Silica aims to use femtosecond lasers to store data in silica glass.) Perhaps the most intriguing opportunity is nature's solution to the data storage problem, nucleic acids, the building-blocks of DNA and RNA. DNA has an information storage density that is several orders of magnitude higher than any other known storage technology. In fact, a few tens of kilograms of DNA could, in theory, meet all the world’s storage needs for centuries to come.?Accessing data in this format, however, presents its own unique set of challenges and opportunities.

Memory and storage

A simplistic way to view the difference between memory and storage is to understand that memory is an integral part of compute operations while storage serves as a highly resilient data repository. These lines get somewhat blurred when we consider “far” memory and “fast" storage. Memory is a cornerstone of modern computing systems, be it DRAM or extremely fast and expensive SRAM cache and scratchpad memory on, or tightly coupled to, processor silicon. Similar to the way DRAM is used for tiering and buffering data to avoid storage accesses, SRAM-based cache designs are used to avert costly accesses to main memory. SRAM collocated on processor ASICs has gone from 25% of total transistor count in 1990 to nearly 80% in 2020.?In short, with the advent of multicore processors and domain specific accelerators, memory subsystems are a dominant consideration in the design of modern computing systems.

The cost of data movement can no longer be overlooked in modern systems and applications design.?There is a hierarchy of memory and storage types ranging from processor registers and caches to main memory to storage and data movement through this hierarchy to the computing elements consumes far more energy than is required to complete the computation itself.?As such, technologies such as in-memory-compute and byte-addressable storage are becoming increasingly attractive because they circumvent the need to perform costly data moves.

Don’t’ forget interfaces!

Memory and storage are playing increasingly important roles in modern ICT and efforts to increase memory/storage capacity are underway. However, the device technology innovations cannot progress without simultaneous development of new energy-efficient, high-bandwidth interfaces and protocols that are conducive to in-memory-compute. In this next decade, many technology interfaces are in flux, and it is important to define a new set of interfaces to avoid the need for every vertically integrated computing solution be a custom one-off special purpose design.

As a final remark, while the memory and storage industries are experiencing unprecedented growth, the risks of vertical market failure have recently emerged. This is due in large part to increasing technical asset specificity where there are few buyers outside of hyperscale customers. Moreover, specialization often creates niche markets that are sub-scale in size, making significant R&D investments difficult to justify. This scenario creates a dilemma for manufacturers as they have historically carried both the burden of product research and development, as well as the subsequent commercial risk. The prospect of vertical market failure can be mitigated by private sector market participants through risk-share agreements between customers and suppliers, as well as increased vertical integration.?Moreover, given the strategic nature of data management and storage, it would also be beneficial for the government to lend public-sector market leverage to help ensure that memory and storage innovation is supported to an extent necessary to spur needed innovation, and ensure America has a hand in the market.

SRC's Decadal Plan for Semiconductors outlines research priorities that can help us meet the needs of future generations. Developed by leaders across academia, government and industry, the report identifies five seismic shifts shaping the future of semiconductor technologies and calls for an annual $3.4 billion federal investment over the next decade to fund research and development across these five areas. Read the report at: src.org/decadalplan

This article was authored by Sean Eilert (Micron Technology), Steffen Hellmold (Western Digital), Steve Kramer (Micron Technology), and Victor Zhirnov (SRC).