草榴社区

草榴社区 AMBA CHI C2C System Verification 草榴社区

Venkatesh Kudumula

Feb 22, 2024 / 5 min read

Overview

Arm has recently introduced the . This is a chip(let)-to-chip(let) extension of the AMBA CHI architecture and is referred as AMBA CHI C2C protocol. This article delves into the details of the AMBA CHI Chip-to-Chip protocol and the associated 草榴社区' verification solutions tailored to meet industry demands. 

.AMBA CHI C2C protocol also scales to the upcoming major revisions of AMBA CHI standard, beyond CHI Issue F (CHI-F). Please refer to our blog on 草榴社区 AMBA CHI-F VIP for more details on Industry leading AMBA CHI verification IP solutions.

The CHI C2C protocol enables building a system with multiple CPU, accelerator, or other device chips or chiplets using CHI protocol. In this blog, the terms "chip" and "chiplet" are used interchangeably unless explicitly stated otherwise.

Use Cases

The two primary use cases covered by the CHI C2C protocol are:

  1. Multi-chip symmetric multiprocessor (SMP) topology: A small number of functionally similar & tightly connected chips. Typically, each chip is a SoC with several processing cores and attached memory, and such a memory is coherently shared by processing cores on all the chips.
  2. Multi-chip coherent accelerator attach topology: One or more fully coherent or I/O coherent accelerators connected to host chips. id est laborum.
a) SMP Topology, b) Accelerator Attach Topology

Concept

The conceptual diagram below illustrates the connection between two chiplets via UCIe. Each chiplet demonstrates a logical flow from on-chip CHI to the corresponding CHI C2C layers, functioning as the UCIe streaming protocol layer. C2C interface such as AMBA CXS, establishes a connection between the streaming protocol layer and the UCIe transport.

CHI C2C Over UCIe: Conceptual Diagram

The functional logic between the on-chip CHI interface and the chip pins that the message traverses through is composed with multiple functional layers. Any of these below layers can be a 草榴社区 Verification IP component, depending on the target DUT and the required verification topologies. For eg: on ‘chiplet 0’, the UCIe Streaming Protocol Layer, C2C I/F TxRx, UCIe transport can be Verification IPs from 草榴社区. Whereas on-chip CHI logic on ‘chiplet 0’, and entire ‘chiplet 1’ can be DUT components.

  • CHI C2C Protocol Layer
    • Accepts CHI flits from on-chip CHI logic, transforms them into appropriate outgoing C2C messages, and dispatches these C2C messages to the C2C Packetization Layer.
    • Receives incoming C2C messages from the C2C Packetization Layer, converts them into CHI flits, and transmits these CHI flits to the on-chip CHI logic.
    • For each on-chip channel, there exists a corresponding C2C message class, namely REQ, RSP, DAT, SNP. Additionally, a miscellaneous C2C message class manages various other aspects of the C2C protocol, such as initialization, flow control, property exchange, and more.
  • CHI C2C Packetization Layer
    • Accepts outgoing C2C messages from the C2C Protocol layer, encapsulates them into fixed-size outbound containers, and dispatches these containers to the Link Layer via the C2C interface.
    • Receives incoming fixed-size containers from the Link Layer through the C2C interface, extracts C2C messages from them, and transfers these C2C messages to the C2C Protocol Layer.
  • C2C Interface
    • The combination of the Link layer and Physical layer is referred to as the 'Transport Layer' throughout this blog. When Transport Layers are available, it establishes a connection between the CHI C2C layers and the Link Layer of the Transport.
    • In Arm-based systems, the commonly employed C2C Interface is the AMBA CXS interface.
    • In cases where Transport Layers are absent, it connects the local CHI C2C layers to to the C2C layers of the remote C2C component.
  • Transport Layers: Link Layer and Physical Layer
    • The Link layer is responsible for transporting message at the FLIT granularity, ensuring data integrity and error detection, and facilitating recovery through a retry mechanism. The Physical layer is responsible for ensuring reliable electrical connectivity between two interconnected chips or chiplets.
    • For chiplet-to-chiplet connections, the designated transport is UCIe.
    • For chip-to-chip or socket-to-socket connections, the designated transport is CXL

In the presence of transport layers, the C2C packetization layer accommodates two container formats for packing and unpacking C2C messages, depending on the specific Transport layer:

  • Format X: Utilized with UCIe transport for chiplet-to-chiplet topologies, is compatible with UCIe 256B Latency Optimized Mode, featuring optional bytes.
  • Format Y: Employed with CXL transport for chip-to-chip topologies, is compatible with the CXL 256B latency-optimized flit format.

C2C Protocol Initialization

In the diagram below, the two chiplets are interconnected through UCIe, within a SiP (System in Package). This diagram illustrates the initialization flow of the C2C protocol, with AMBA CXS as the C2C interface. 

C2C Initialization Flow

C2C Interface Initialization

The C2C interface initialization is required to enable sending and receiving C2C protocol messages. This includes:

  • Coordinate with the Link Layer to bring up the C2C interface and to check if the remote link partner C2C protocol layer is up.
  • Exchange interface properties using appropriate C2C messages, to setup the interface to start exchanging message credits followed by protocol messages. 

C2C Flow Control

The local and remote C2C protocol layers incorporate a credit exchange mechanism to control the flow of C2C messages. This mechanism is essential for a local transmitter, managing the associated remote receiver buffer size, similar to the on-chip CHI per-channel credit mechanism.

The C2C protocol also introduces the concept of 'Resource Planes' to further categorize messages within a message class. Each resource plan (RP) is assigned an independent message class credit pool, allowing messages from one RP to make forward progress while those from another RP may not advance. This approach aids in meeting system requirements for Quality of Service (QoS), resembling the TC-VC mapping found in the PCIe protocol.

Following the completion of interface initialization, it is essential to exchange interface properties bidirectionally for each logical link prior to sending any protocol messages. These properties encompass:

  • On-chip CHI payload fields presence/width, for eg: MEC_Suport_Rx, MEC_Support_Tx, RSVDC_REQ_Rx, RSVDC_REQ_Tx
  • On-chip CHI opcode/flow, for eg: RME_Support_Rx, RME_Support_Tx
  • C2C specific, for eg: Container_Format, NUM_RP_REQ_Rx, NUM_RP_REQ_Tx

Other Key Features

  • The C2C protocol provides HW flows to manage the connectivity state of the interface and thus the link for the following flows: Fully coherent messages, DVM messages, All protocol messages.
  • RME-DA, RME-CDA:  In the multi-chip coherent accelerator attach configurations, I/O coherent and fully coherent device accelerators are connected to a host. RME-DA, RME-CDA is part of Realm Management Extension (RME) architecture, this allows secure assignment of such assignable device interfaces. This is one of the features that is aligned with upcoming CHI protocol versions, to support CHI interface-based accelerators.
  • On-chip CHI DVM transactions are supported over C2C interface, with optimized flows to reduce the number of C2C messages. 

Summary

Arm and 草榴社区 have continued to collaborate close on this newest extension of the AMBA protocol family to ensure joint customers can experience the benefits from using the AMBA CHI C2C protocol, while ensuring their chiplets and multi-die designs are adherent to the protocol standard.

Several HPC and datacentre customers already achieved tape out success using 草榴社区 AMBA Verification IPs along with multi-chip verification solutions catering to CHI Host based SMP topologies, as well as CHI host to CXL Cache/Mem device topologies.

Please contact 草榴社区 for more details on Verification solutions for AMBA CHI C2C, AMBA AXI C2C, AMBA CHI, AMBA AXI, UCIe and CXL protocols catering to the simulation and emulation needs for pre-silicon verification/validation of chiplet and multi-chip design topologies.

Continue Reading