Designing a NVMe compliant PCIe SSD Jérôme Gaysse IP-Maker

Explore IP-Maker IP here

Introduction

Flash Memory Summit 2012, Santa Clara, California, is over. It was a great success about NandFlash memory technologies with 50% increase in term of attendees. Forecast for the next year is on the same trend. One key word of this event was "NVMe" with: 1) a full conference session and a panel on this topic, 2) some companies where demonstrating running hardware on the exhibition area, 3) many companies claim to come to the PCIe SSD market. NVM Express is definitely the key technology solution to enable the broad adoption of SSD using the PCIe interface. The remaining question is: how to design a PCIe SSD compliant to the specification?

Reminder: why designing a PCIe SSD with NVM Express features?

Drivers for PCIe SSDs are not standardized like for USB with the mass storage class. That means that each SSD provider needs to develop its own software driver. That participates to the high cost of a PCIe SSD. In addition, most of current PCIe SSDs are based on the integration of a PCIe-to-SATA bridge and SATA-based NandFlash controllers. This solution is efficient for a PCIe Gen 1 interface. That becomes not viable with Gen 2 and Gen 3. To sustain a so important throughput, too many SATA channel in parallel would be required. The last point is about the data transfer control. With this current solution, the management is done by the host processor; therefore wasting time when waiting for the end of NandFlash read or write.

For more details, read the Tech Talk "Software driver standardization and performance increase for PCIe SSDs"

A workgroup composed of storage industry leaders has defined the NVM Express specification which aims to eliminate all these limitations and to provide a standard interface. The NVM Express specification, developed cooperatively by more than 80 companies from across the industry, was released on March 1, 2011 by the NVMHCI Work Group, now more commonly known as the NVMe Work Group. The NVM Express 1.0c specification defines an optimized register interface, command set and feature set for PCI Express Solid-State Drives (SSDs). The goal is to help to enable the broad adoption of solid-state drives (SSDs) using the PCI Express (PCIe) interface.

For more details on the specification: www.nvmexpress.org.

Controller design tools

This is the beginning of a new era for SSD controllers. Today, the trend is definitely to design its proprietary SSD controller in order to provide the optimized features and performances.

The base of such controller (figure 1) is to integrate a PCIe controller and a NandFlash Controller. How to make it NVMe compliant? Since NVMe is a set of command and features, why not implementing it in software? The hardware layer remains the PCIe bus and the use of an embedded processor seems to be a solution for the NVMe protocol decoding. But, the NVM Express specification has been established to optimize the performances, so there is no way to implement it in software. The ideal solution is to have a full hardware implementation.

Figure 1 - PCIe NVMe SSD architecture

The IPM-NVMe IP core (figure 2) from IP-Maker is a full RTL solution compliant to the NVMe 1.0c specification. The complete IPM-NVMe system is managed by a submission and completion administration queue. The system is very flexible, and allows user to attribute dedicated queue to a specific action. To add service quality, logs are used such as error and healthcare. A set of arbitration is available: either a simple round robin or a weighted round robin. There is also the possibility to execute a configurable amount of command on a queue before requested a new arbitration.

Figure 2 - IPM-NVMe architecture

Validation and certification

The first basic test is to connect the SSD in a host system with the NVMe driver installed. NVMe drivers are available from the nvmexpress.org website. If the SSD embeds NVMe features, it will be recognized by the host driver.

The last step for having the "NVM Express" label is to pass the official test suite managed by the University of New Hampshire Interoperability Lab (UNH-IOL). It provides the NVM Express Conformance and Interoperability Tests. The hardware platform used for this new conformance test suite is the Teledyne LeCroy Summit^TM Z3 Exerciser. The conformance tests are based on the UNH-IOL NVM Express Conformance Test Suite Document release by the NVM Express work group. UNH-IOL will provide NVM Express conformance testing services in the near future to ensure proper standardization in the rapidly growing NVM Express solid-state storage device (SSD) market.

At Flash Memory Summit, IP-Maker has demonstrated NVM Express capabilities thanks to its NVMe IP core integrated in a FPGA. The board was connected to a host system with the use of the Teledyne Lecroy Summit T3-8 analyzer (figure 3), showing NVMe protocol.

Figure 3 - Teledyne Lecroy Summit T3-8

For more details on the test suite: https://www.iol.unh.edu/services/testing/NVMe/

Conclusion

It seems that all the tools needed for the design of PCIe SSD compliant to the NVM Express specification are available, including the key element of the PCIe SSD controller: the NVMe IP core. Such SSDs are expected on the market in 2013. PCIe SSDs have a small market share today. Let's see in the coming years how it will move. Forecast numbers show a drastic increase of this share. That will measure the success of the NVM Express standard.

Explore IP-Maker IP here

• BCH Encoder/Decoder IP Core
• IPM-NVMe - NVM Express IP Core
• Universal Nand Flash Controller

About the Author

Jérôme Gaysse
IP-Maker

Jérôme Gaysse is responsible for business development for IP-Maker, with a strong focus on SSDs. He held positions from hardware design at General Electric Healthcare to product marketing at Atmel. He used to present papers at Embedded System and IEEE international conferences.