草榴社区

OpenVX for Embedded Vision

Embedded vision applications are typically written as OpenVX graphs. OpenVX is an open standard for the acceleration of computer vision applications and has many embedded and real-time use cases. This includes applications such as face, body, and gesture tracking, smart video surveillance, advanced driver assistance systems (ADAS), object and scene reconstruction, augmented reality, visual inspection, and robotics.

An OpenVX graph is constructed from one or more kernels. Each kernel performs a vision function and may be one of the standard OpenVX kernels, a custom supplied kernel, or a user-defined kernel (Figure 1).

Trace View

The trace view displays a timeline of the OpenVX graph execution so that you can study the scheduling, pipelining, and timing in detail. The trace view can be adapted in many ways and supports both horizontal and vertical display.

As an example, we are using the demo trace provided with Tracealyzer for OpenVX (“demo_openvx.xml”). This has been recorded from an OpenVX demo application together with a screenshot from the trace view (Figure 3).

You can see how the runtime software schedules the graph using two cores. Core 0 reads the input frames and feeds it to the sobel3x3 node. The result is then processed further on Core 1, using the magnitude and convert_depth filters. The processing is run-to-completion, so each rectangle (fragment) in the trace is a separate job that runs without preemptions. The short fragments of the filter functions are precondition checks, while the long fragments show the actual filter processing.

The magnitude node starts before the sobel3x3 node is completed, which is possible because the OpenVX implementation in this example divides each frame into tiles. One node may output multiple tiles that are written to the output buffer one by one, as soon as completed. Thus, the following node (e.g., magnitude) does not need to wait for a full frame, but can start as soon as the first tile is available, assuming the nodes run on different cores. This allows for a pipelined processing that utilizes the cores efficiently.

CPU Load Graph

To get an overview of how your OpenVX application utilizes your CPU cores, look at the CPU load graph, shown twice (one for each core) together with the trace view in vertical mode (Figure 5). The CPU load graph allows you to see the overall load on your CPU cores, as well as how the load varies over time and the contribution of each node.

The CPU load graph also works as an overview where you can spot anomalies, for instance, the two spikes in the “sobel3x3” node (shown in red) where the utilization is around 80-90%. You can see what causes these spikes by double-clicking in the CPU load graph to show the corresponding section in the trace view.

Actor Instance Graph

In Figure 6, we added two instances of the Actor Instance Graph, where the Y-axis shows the execution times of the graph nodes. This way, you can see where nodes execute longer than normal and inspect the trace view to see the details. Note that “Actor” is a Tracealyzer term meaning “execution context”, corresponding to nodes in OpenVX.

Actor Statistics Report

The Actor Statistics Report gives a statistical summary of the trace, including the highest, lowest, and average values observed for timing properties such as execution time. All extreme values in this report are links and can be clicked on to find the corresponding location in the trace view (Figure 7).

With the Actor Statistics Report, you can find the extreme values and see what was going on in the system at that time. Although all details are not recorded you can still get valuable clues about what caused these values. For instance, using the Actor Instance Graph you can find other cases with similarly high execution time and check for correlations in the trace. Perhaps the high execution time only occurs under particular circumstances, e.g., due to intense activity on other CPU cores saturating the bus.

Note that you can export and save the statistics reports, either as formatted HTML files (like above) or as tabbed text files (Figure 8). The latter allows for easy data import into other tools and is done by checking the option “Data Export” in the Actor Statistics Report dialog. This allows you to run measurements on alternative designs and compare the resulting performance metrics systematically. For instance, the statistics report shows that IDLE0 is running 40.7% of the time, meaning that Core 0 is only 59.7% loaded, while IDLE1 only runs for 17% of the time, meaning that Core 1 is 83% loaded.

User Events

Tracealyzer allows you to add your own user events, i.e., custom events logged from the application code. User events allows you to visualize just about anything in your application, like diagnostic messages, variable values, and states.

Figure 9 shows an example where user events have been logged on two user event channels, “MyVariable” showing values of an integer variable and “MyState” showing state names. Tracealyzer can display such user events in several ways, e.g., as event labels in the trace view (1) and as entries in the Event Log (2). The User Event Signal Plot (3) allows for plotting numerical data from user events.

Moreover, if you have important state variables in your system, you can log the state changes as user events and define a State Machine in Tracealyzer to see the states in the trace view timeline (4). You can also see a summary of the state changes as a state machine graph (5). You can even get statistics on the time spent in each state, or the time between any two events by defining a “custom interval”.