Introduction to NVMe and NVMe/TCP

NVMe stands for Non-Volatile Memory access through the PCIe (PCI Express) bus. This standard defines how to access solid-state disk drives (SSDs).

The previous generations of disk drives were based on stacks of magnetic media disks spinning at high speed. The protocols (like SATA) used to interface with those electromechanical disks were defined so as to exploit their advantages and cope with their limitations.

With the massive adoption of SSDs, it became apparent that SATA couldn't realize the full capabilities (high speed, parallelism) of the new generation of solid-state storage media.

NVMe came to bridge this gap. This protocol has lower latency, offers multiple, deep command queues, and can support thousands of interrupts to take advantage of the unique features of SSDs as compared to electromechanical disk drives. The multiple command queues are easily mapped to multiple-cores, making for a more scalable solution.

NVMe also has a rich set of features to support flexible power management solutions, needed to extend the battery life of portable devices.

NVMe over Fabrics

The traditional solution for storage was to include one or several SSDs for each computing box. This solution has some obvious disadvantages for data centers. The SSDs of some of the CPUs can be overloaded, while others are underused. To make data available from one CPU to others, the data may have to be copied from one local drive to another local drive.

These scalability and utilization pitfalls prompted the adoption of centralized SSD banks to be accessed by all the CPUs on the data center. This organization approach of the data center, which includes function-specific racks (CPU racks, SSD racks) is usually called a disaggregated data center.

The NVMe-of (NVMe over Fabrics) set of protocols provided the backbone to accessing the SSD data racks for each of the CPU racks on the data center. There are several flavors of NVMe-of. One of them is NVMe/TCP, which bases itself on the well-known and worldwide adopted TCP-IP protocol.

If you want to know more, you can download a ten-page whitepaper about NVMe/TCP here