²ÝÁñÉçÇø

When the CXL 1.0 spec was released, it brought the concept of FLITs (FLow-control unITs) into the PCIe world.  Using the Alternate Protocol Negotiation feature introduced with the PCIe 5.0 spec, two PCIe link partners proceed through an otherwise normal PCIe link negotiation sequence, but instead of ending up in the PCIe ¡°L0¡± state, they enter CXL mode and begin exchanging CXL FLITs instead.  With its focus on a host-centric asymmetric cache coherency protocol, latency is absolutely critical to CXL¡¯s effectiveness, and the entire CXL specification is built around minimizing latency.  A discussion of the details and tradeoffs made to achieve that goal is outside the scope of this article, but for the 64-byte transfers typical of many CPUs¡¯ cachelines, CXL offers several times lower latency than PCIe equivalents.  CXL actually defines three different protocols: CXL.io, CXL.cache, and CXL.mem.  CXL.io transports PCIe for backwards compatibility, bulk data transfers, and system configuration purposes ¨C with no gain in performance over a native PCIe link.  CXL.cache and CXL.mem are the real substance of the CXL spec ¨C supporting only 64-byte transfers but offering the amazingly low latency that¡¯s become CXL¡¯s hallmark. 

Originally it seemed like the CXL.mem protocol was defined simply to allow a host CPU to access an accelerator¡¯s local memory, but the spec was wisely written generically enough to enable that host to access arbitrary memory in the CXL fabric.  Initially, there was interest in using CXL.mem as a way to add non-volatile memory to systems.  Traditional storage interfaces are block-oriented and difficult to adapt to the random access patterns typical of CPUs.  While it¡¯s always been possible to map memory on interfaces such as PCI Express, the I/O focus of those interfaces combined with legacy software means that memory attached there cannot effectively be cached by the host CPU.  CXL.mem¡¯s ability to access memory from the CPU¡¯s existing cache architecture enables designers to easily attach NAND flash and other non-volatile memories in a way that appears to the host CPU as if it is simply more system memory.  CXL is really the first interface to offer widespread access to non-volatile system memory, and that opens up a new frontier for software architecture which blurs the lines between files and memory.  

It didn¡¯t take long for system architects to expand this same vision of CXL-attached memory to traditional volatile memory technologies.  By moving system memory away from the CPU, it becomes much more flexible.  Where traditional architectures can leave unused memory connected to one specific CPU inaccessible to other CPUs, memory residing on a CXL interface can be allocated to CPU A for some period of time, and CPU B at some other time.  As CXL fabrics extend to box-to-box environments, one can imagine a near future in which memory from multiple physical servers can be pooled to service a memory-intensive job.  Many segments of the datacenter market found this flexibility in memory placement to be extremely attractive, and this market has been exploding over the last couple of years.  Such CXL.mem devices are dominating discussion at such industry events as Flash Memory Summit, Open Compute Project Summits, and more.

CXL.mem offers both system and SoC designers significant flexibility in implementing low latency memory solutions.  SoC designers need to focus on reducing their overall latency, and plan early for specification-required application logic when implementing CXL.mem Devices.  ²ÝÁñÉçÇø offers a wide range of CXL and PCIe controller and PHY IP including Dual-Mode (both CXL Host and CXL Device run-time selectable in a single controller) and Switch port support.  ²ÝÁñÉçÇø further has the expertise to help guide SoC architecture and design choices to achieve the best possible latency and performance.