草榴社区

What is a Virtual Sensor?

A virtual sensor has its roots in the concept of digital twins. As the name suggests, where a physical sensor generates data based on what it “sees” in its immediate environment, a virtual sensor computes — or extrapolates — based on third-party information. This data acts as a representation of the environment and can be based on one, two, or three dimensions, which the virtual sensor processes and converts into a digital image when the sensor is a camera.

There are multiple scenarios in which a virtual sensor can act as both the chip and ed-product designer’s friend. Let’s take the variables of a camera as an example. The principles could apply to any physical sensor, such as those found in LiDAR and radar.

In one instance, a virtual camera can help determine the specifications of the physical camera for a particular autonomous vehicle model: Should it be black and white or color? How many pixels should it have? What would be the optimum depth of field? Where should this camera be positioned on the car to get the best information? Experimenting with a virtual camera — adjusting the pixel count, color balance, signal processing, and so forth — can help yield the answer to these questions and more without spending money on expensive prototypes.

In the design phase, a camera may consist of three or more components: the lens, the sensor, and the image signal processor. A virtual camera can simulate the resulting image quality to test the interplay of these components and establish the right mix.

Virtual Sensors in Testing and AI Model Development

Post-design, it’s time to test the camera in situ. A virtual installment behind the windscreen or on a side mirror will provide an accurate understanding of how the final product will look at the environment in which it operates and ultimately how well it will function when integrated into a vehicle.

A further aspect where virtual sensors come into play is in developing the AI models that interpret the environments that the camera senses. Virtual images of elements such as road signs and pedestrians enable the model’s training and development, removing the need for extensive physical information collection. Once fully formed, the AI model can be utilized by the physical camera that allows the autonomous vehicle to operate as it should.

In short, a virtual sensor has the potential to generate scenarios that a real autonomous equivalent will need to respond to, as opposed to seeking them out or waiting for them to happen (such as how to react to a child darting out in the middle of the road in search of a ball or how to give right of way to an emergency vehicle). In the field of automotive chip design, we can expect digital twin models and virtual sensors to play an increasing role in architecture exploration to meet the growing performance demands of OEM workloads; software development, hardware and software integration, and system-on-chip (SoC) validation before silicon availability; and the testing and validation of semiconductor models.

At the same time, it bears noting that autonomous vehicles have yet to become prevalent on our roads despite the presence of highly advanced chips, cameras, and automotive technology. Among other concerns, the volume of data that the system as a whole must process remains a barrier. Currently, autonomous cars are designed on a component-by-component basis rather than as a complete system that can streamline that data. Virtualization in terms of cameras, electronic control units (ECUs), and environments can help enable better teamwork and system optimization by providing a view of the components’ interdependencies and breaking down the silos in which they take shape.