草榴社区

ASIPs have established themselves as an implementation option next to standard processor IP and fixed-function RTL. They combine hardware specialization with flexibility through software programmability.? This talk will provide an introduction into 草榴社区' ASIP Designer tool-suite, targeted markets, and how 草榴社区 collaborates with university partners in this domain.

In one of our funded projects, we have adopted an experimental approach, where we employ ASIP Designer for crafting a compiler tailored for a specific RISC-V host processor in conjunction with a dedicated vector co-processor for AI and sensor fusion workloads. One main aspect is that we have integrated the entire framework into a robust CI/CD pipeline, enabling us to conduct regular comparisons with GCC for enhanced efficiency and accuracy in our development process. We will share insights into the hardware foundation and the anticipated hurdles. Additionally, we will give a sneak peek into our newly developed RISC-V instruction set explorer RISE that utilizes profiled simulation to determine the most suitable RISC-V configurations and standard extensions for given applications.

Edge AI applications often need a high computational efficiency solution. Lower power and area efficiency architectures are key, whereas they still must be able to deliver a high performance and provide a high flexibility to support many graphs and other functions.

Getting the data efficiently from/to the main data structures to the actual compute units is often the most important problem. A memory hierarchy bridges the gap between the main memory and the registers connected to the computation units. We must limit the size of each memory layer whereas we must maximize the reuse of data to avoid unneeded data movements to the next layer. 

SmarT is an ASIP with specialized vector units for convolutions and quantization with 32 MACs. The 4-lane vector address generation unit allows to read gather/scatter vector data from 4 different locations in a single access. This flexibility enables to accurately load the data that can be reused in registers. The circular postmodify addressing modes allows to continuously load data in parallel with computations and have efficient data memory management (circular buffers). Finally, specialized DMA instructions can move data between a large remote memory and the local memory. These DMA instructions have very low overhead and can therefore be efficiently used for fetching/storing relatively small blocks to a relatively small local memory.  

We have mapped various CNN kernels to prove the effectiveness of the architecture. For a specific kernel different optimizations are required, depending on its parameters such as kernel-size, input-width and depth. Kernels must be optimized to balance the computations, memory accesses, and their reuse in registers. By computing multiple outputs simultaneously, we can improve the register reuse, and reduce the memory bandwidth.

When the kernels of a graph are executed sequentially, the intermediate data must be stored in remote memory.  By merging kernels, data can be kept in local memory, using circular buffers.  Data movement to remote memory can be avoided.

We use tensorFlow Light for Microcontrollers (TFLM) to map the CNN graph onto the smart processor. We have optimized the relevant TFLM kernels to execute efficiently on the smarT vector units.

Our smarT is derived from a RISC-V example. Although we are not instruction compatible, it is still a RISC core that can execute generic computations.

Layer-wise mixed-precision quantization (MPQ) has become prevailing for edge inference since it strikes a better balance between accuracy and efficiency compared to the uniform quantization scheme. As the pursuit of an optimal mixed-precision quantization approach for models continues to intensify, there is a growing demand for corresponding hardware support to enhance the effectiveness of MPQ. The open-source ISA RISC-V offers the flexibility to tailor the instruction set architecture (ISA) to specific requirements. In our ASIP project, we propose to tightly integrate Single-Instruction-Multiple-Data (SIMD) Vector-Matrix Multiply (VMM) units that support various quantization bit-width as part of the RISC-V processor pipeline. These units are designed to support a range of quantization bit-widths, achieved through micro-architecture and Instruction Set Architecture (ISA) co-design. We also extend the RISC-V ISA with custom instructions that follow the “R type” base instruction format and encode the quantization precision in the funct7 bits. Evaluation results show that for a General Matrix Multiply (GEMM) kernel, the tightly integrated MPQ units achieve a speedup of 15.13x-29.65x compared to a baseline RISC-V processor.

The rise of quantum computers poses a threat to security because they are predicted to break some of the cryptographic algorithms used today. New post-quantum algorithms were developed to replace the vulnerable, existing standards. The new standard for Public Key Encryption (PKE), Kyber, was selected in July 2022. In this work, we design an Application Specific Integrated Processor (ASIP) optimized for Kyber. We start from a previous RISC-V-based processor design and use 草榴社区' ASIP Designer tool to extend it. The processor features zero-overhead hardware loops, post-modify incrementing, and 2-way Instruction-Level Parallelism (ILP). We extend it with a Keccak accelerator that computes a whole permutation round in a single clock cycle. The accelerator holds 50 separate 32-bit registers to store the 1600-bit state. To accelerate arithmetic operations, the ASIP processes all polynomial coefficients in a 2-way Single Instruction Multiple Data (SIMD) manner. The Number Theoretic Transform (NTT) is also accelerated using SIMD by means of special butterfly units. The ASIP is synthesized in a 28nm high-performance technology at 1 GHz. We reach execution times of 33, 193 and 249 ?s for KeyGen, Encaps and Decaps and an area of 34 705 ?m?. We achieve a speedup of 5,0x with respect to a similar work that implements a Post-Quantum Cryptography (PQC) ASIP, at the cost of an area increase of 3,0x.

With the current complexity of hardware designs, verification is a tedious task. ASIP Designer has many features that support a simulation-based approach to verification. However, in this presentation we reveal a new feature of ASIP Designer for Formal ISA Verification. Formal verification is a powerful method to aim at bug-free designs that complements traditional simulation. In contrast to simulation, formal verification exhaustively explores the complete design space and results in a mathematical proof or disproof that the implementation exhibits expected behavior. ASIP Designer automatically generates SystemVerilog Assertions (SVA) for each rule specified in nML. Formally proving these assertions guarantees the expected behavior of these rules in the ASIP’s RTL implementation under all kind of contexts such as, for example, overlapping instructions that exhibit pipeline hazards, within parallel instructions, or when transitioning to/from on-chip debug mode. The assertions are complemented by constraints (assumptions) to remove invalid states, that is, states the compiler would not produce. The generated assertions and assumptions form an easy starting point, already available during design exploration, and can be extended by the user to verify specific aspects of the ASIP and its peripherals. Additionally, several strategies are applied to ensure that proofs converge in reasonable time, hereby especially looking at using 草榴社区’ VC Formal FPV app.

The next generation of wireless communication systems (beyond 5G) will feature advanced, key technologies such as massive multiple-input multiple-output (MIMO) composed of a large number of antenna elements. Massive MIMO enables spectral-efficient communication and accurate positioning, but the concomitant tremendous data rate requirements, low processing latency needs, and demands for flexibility necessitate careful algorithm-architecture co-design. Additionally, deep convolutional neural networks (CNN) have by now proven their mettle in tackling unwieldy problems. To address these challenges, in this work we have developed an application specific instruction set processor (ASIP) that is both capable of supporting communication algorithms and positioning. The positioning is based on the fingerprinting method and utilizes the wireless channel state information (CSI). The ASIP is geared with vector processing capabilities in the form of single instruction multiple data (SIMD) which are utilized to implement the zero-forcing (ZF) algorithm. Furthermore, a systolic array is tightly-coupled to the processor which is tuned to speed up matrix calculations. A parallel memory subsystem is also integrated into the ASIP architecture in order to expedite access to matrix elements.

We present research conducted at TU Dresden utilizing the ASIP designer tool to design and assess the complexity of baseband kernels within modems implementing the 5G IoT wireless standard NR-Redcap. ASIP designer proves to be a valuable asset in exploring the generated assembly code for the multi-instruction issue vector processor (vDSP) prototype, facilitating an in-depth analysis of baseband kernel performance. This analysis aids programmers in enhancing both code density and clock cycle performance of the DSP engine. Over the past couple of years, extensive research has been dedicated to the Synchronization kernel for the RedCap modem. Leveraging the ASIP designer tool, the study identified system implementation bottlenecks and devised optimizations resulting in remarkable improvements, with vectorized kernels achieving up to a 4X speedup compared to conventional counterparts. These optimizations demonstrate the cost-effectiveness of ASIP-designer-based kernel implementations for emerging IoT baseband modems. Furthermore, this research underscores the potential for minimizing hardware costs by employing optimized vDSP-based solutions, eliminating the need for additional hardware accelerators to perform tasks now efficiently executable on vector DSPs. In sum, this study highlights the significance of ASIP design in enhancing the performance and cost-effectiveness of 5G IoT wireless modem implementations.