草榴社区 Cloud
Cloud native EDA tools & pre-optimized hardware platforms
Request a Free Trial →
Table of Contents
From video doorbells to modern vehicles, the types of applications that rely on cameras are growing. For many of these systems, having access to real-time, high-resolution imagery is integral to their effective operation. After all, if the doorbell can’t clearly identify a package thief or the car isn’t able to accurately detect a roadway obstruction, their value declines tremendously. Increasingly, deep-learning models are being implemented to enhance the vision-processing capabilities that these products need.
Increasingly, vision-processing applications are applying artificial intelligence (AI) in their designs to enhance the quality of the images and/or to take advantage of advanced AI features like object detection. Computer vision applications that relied exclusively on the highest level of digital signal processing operations (DSP) are now mixing DSP and AI. Now, there’s a neural processor IP solution available that delivers the industry’s smallest power and area footprint to support up to two trillion operations per second (TOPS) of the latest AI networks, including transformers. Read on to learn more about how the new 草榴社区 ARC? NPX6-1K NPU Processor IP with 1,024 MACs provides an ideal option for designers ready to introduce greater intelligence into their chip designs.
AI Transformers + Neural Networks = Amazing Vision Processing
Demand for AI capabilities in a variety of products is growing, and nowhere is this more evident than in computational imaging. The ubiquity of cameras integrated into smartphones (and delivery of high-quality images) has paved the way for cameras to be included in everything from doorbells to medical devices. There are dual branches in computational imaging:
- Computational photography, which involves the capture and processing of images using digital computation
- And computer vision, which involves the creation of digital systems that can process, analyze, and make sense of visual data as humans can
AI-driven computer vision technology is proving to be essential for delivering accurate, real-time, high-resolution imagery. Traditionally, convolutional neural networks (CNNs) were the dominant algorithm in computer vision applications. Neural networks have been commonly used for object detections and are now playing an important role in image quality improvement, taking over tasks that were once done by digital signal processors. For example, because they can embed knowledge of what a good image should look like, neural networks can upscale video streams to 4x the resolution via super resolution networks, and they can be used to reduce noise and enhance low light performance. Features such as blur reduction, high dynamic range, and wide dynamic range are also within the realm of AI-driven computer vision capabilities.
Now, AI transformers—originally developed for natural language processing such as translation and question answering—are emerging as the highest accuracy option. Transformers, based on a self-attention mechanism, are better at learning complex patterns for accurate object detection than CNNs, and therefore can better understand context. Used together with CNNs, the combination of the two deep-learning models can significantly enhance computer vision and image processing accuracy.
Integrating AI-Driven Vision Processing to a Wider Array of Applications
With 1,024 MACs, the new ARC NPX6-1K NPU IP provides a good entry point for designers to add support for the latest neural networks—especially transformers—to power- and area-constrained vision-processing designs. Another version, the ARC NPX-1KFS NPU IP, provides state-of-the-art hardware safety features to accelerate ISO 26262 certification for automotive designs. The NPX6-1K and 1KFS processors can be tightly integrated with the 草榴社区 ARC VPX2 DSP Processor IP, to produce the market’s most area- and power-efficient AI + DSP solution for DSP and neural network transformers. The combined NPX6 and VPX solutions can scale upwards and configurations can be mixed and matched to support a large DSP with small AI, large AI, small DSP, etc.
There are a variety of use cases for a 1,024 MAC neural network processor. Here are a few examples:
- Processing raw data ahead of the image signal processor (ISP) in a camera application for better noise reduction and, therefore, enhanced image quality, rather than applying AI during post-processing and having the ISP handle the raw data
- Running neural network algorithms to upscale a display by 4x, with minimum power and area
- Processing streaming data directly from the image sensor in a vision application, minimizing area and power in a DRAM-free architecture
- Utilizing a lower cost camera lens and relying on neural network algorithms to enhance image quality or, in an automotive application, to improve the accuracy of functions such as object detection
- Integrating AI into automotive radar and LiDAR designs to support applications such as object detection and automated braking
- Providing support for virtual machines in an automotive environment, helping to isolate safety-critical from non-safety-critical tasks and simplifying certification requirements
The list goes on, as the availability of a processor such as the ARC NPX6-1K opens up possibilities to integrate AI into a new array of applications.
Scalable, Award-Winning NPU IP
With the newest additions, the ARC NPX NPU IP family now scales from 1K to 96K MACs and is capable of delivering up to 3,500 TOPS performance on a single SoC. The 草榴社区 ARC MetaWare MX Development Toolkit provides a single toolchain to accelerate application development with the IP and automatically partition algorithms across MAC resources for efficient processing. Earlier this spring, the IP was named the “Best Edge AI Processor” in the 2023 Edge AI and Vision Product of the Year Awards presented by the Edge AI and Vision Alliance.
Learn More
For more insights about the ARC NPX6-1K, read our white paper, “Computational Imaging Craves System-Level Design and Simulation Tools to Leverage Disruptive AI in Embedded Vision.”