草榴社区

Making AI Adoption Easier with the World's First Network Addressable Processing Unit

Everyone is talking about AI these days. Indeed, with the rise of ChatGPT and its generative AI cohorts, we are experiencing an infusion of intelligence in our everyday lives. According to , the global AI market of US$150.2 billion in 2023 is anticipated to grow at a compounded annual growth rate (CAGR) of 36.8% from 2023 to 2030.

For NeuReality, democratizing AI adoption comes by making AI Inference systems the most efficient and affordable possible in the world’s data centers. By adopting AI-centric system architecture, governments and businesses of all sizes can achieve greater AI data processing speed, cost savings, and a lower AI carbon footprint. The company was founded in 2019 by a team of system engineers who wanted to make AI easy to deploy, manage, and scale – with the foresight that AI would soon explode while the right kind of system and silicon would take a minimum of three years to become reality. That moment is now. NeuReality launched its next-gen NR1? AI Inference Solution with the world’s first NAPU? (network addressable processing unit) and complete NeuReality software and APIs in late 2023.

“We provide a holistic silicon-to-software solution for AI inference,” said Yossi Kasus, co-founder and VP of VLSI at NeuReality. “Multiple types of compilers are typically required for the full flow of an efficient AI pipeline, along with a very complex set of AI accelerators on a device that handles voice, audio, images, and time series data. Our NR1 solution removes all the complexity of pre- and post-processing, so customers can get up and running very quickly. Our hardware runs multiple types of AI workloads and, as a network device, provides a simple way to scale.”

A key component of the NR1 NAPU is a multi-billion-gate 7nm chip, developed with the 草榴社区 Digital Design Family (which includes the 草榴社区 Fusion Compiler physical implementation solution), 草榴社区 Interface IP, and the 草榴社区 Verification Family. The two choices of data center products – the NR1-S? AI Inference Appliance and NR1-M? AI Inference Module – feature an AI-centric system architecture with complex IP, such as a hardware-based AI-Hypervisor? and AI-over-Fabric? network engine, developed alongside many other programmable compute engines to process high-variety, high-volume AI requests and responses. Lots of software and firmware run off these compute engines. The NR1-M is a full-height PCI Express card containing an NAPU and operates independently from the CPU while providing connectivity to deep learning accelerators (DLAs). Given these parameters, the team knew it would be very challenging to simulate the hardware together with the software and verify the full device. 

Scaling for Greater Chip Design Emulation Capacity on Demand

Adopting a cloud-based solution for emulation made perfect sense for NeuReality. As Kasus explained, “When you build your own server infrastructure, you must find more budget for new servers whenever you need to scale. Scaling is easier on the cloud as your project needs fluctuate.”

After an evaluation of available solutions, NeuReality selected ZeBu Cloud, noting the effective level of field support they’d receive from 草榴社区 based on team members’ previous experiences. ZeBu Cloud provides flexible, secure, turnkey emulation to accelerate software bring-up, performance validation, power analysis, and system validation for IP and SoCs. ZeBu emulation capacity is hosted in the 草榴社区 data center. The NeuReality team started using ZeBu Server 4 before transitioning to ZeBu Server 5, the industry’s highest capacity emulator for up to 30 billion gates.

“When we wanted to shift to a stronger emulation machine, the flexibility of ZeBu Cloud helped us to easily do it and benefit from the speed and the number of gates supported,” said Kasus. “And the 草榴社区 support team can log in to easily see what is happening and work in parallel with us when we need the assistance.”

The ramp up was quick, taking only a couple of weeks. As they developed their 7nm chip, the team easily and quickly expanded the capacity of their emulation resources a few times. Without a cloud-based solution, it would have been difficult to meet an aggressive time-to-market target. “Scaling to more emulation servers on the cloud takes a couple of weeks, but had we needed to do this with on-prem resources, it would take months to get new machines in,” said Kasus. “Your needs change as you progress with a project, especially for a new device. You don’t exactly know how much capacity you’ll require, so having the flexibility as you progress is immensely helpful.”

With a comprehensive suite of virtual transactors available, using ZeBu Cloud also meant the team could use all the high-speed interfaces needed for its NR1 AI Inference Solution without any extra hardware. This allowed the team to build a complete system-level test environment for the NR1 NAPU  to be emulated on the ZeBu system.