GPU Compute/Media Hotspots Analysis (Preview)

Analyze the most time-consuming GPU kernels, characterize GPU usage based on GPU hardware metrics, identify performance issues caused by memory latency or inefficient kernel algorithms, and analyze GPU instruction frequency per certain instruction types.

Note

This is a PREVIEW FEATURE. A preview feature may or may not appear in a future production release. It is available for your use in the hopes that you will provide feedback on its usefulness and help determine its future. Data collected with a preview feature is not guaranteed to be backward compatible with future releases.

Use the GPU Compute/Media Hotspots analysis to:

The GPU Compute/Media Hotspots analysis is a good next step if you have already run the GPU Offload analysis and identified:

How It Works: Intel Graphics Render Engine and Hardware Metrics

A GPU is a highly parallel machine where an array of small cores, or execution units (EUs), do graphical or computational work. Each EU simultaneously runs several lightweight threads. When one of these threads is picked up for an execution, it can hide stalls in the other threads if the other threads are stalled waiting for data from memory or other units.

To use a full potential of the GPU, applications should enable the scheduling of as many threads as possible and minimize idle cycles. Minimizing stalls is also very important for graphics and general purpose computing GPU applications.

VTune Profiler can monitor Intel Graphics hardware events and display metrics about integral GPU resource usage over a sampled period, for example, ratio of cycles when EUs were idle, stalled, or active as well as statistics on memory accesses and other functional units. If the VTune Profiler traces GPU kernel execution, it annotates each kernel with GPU metrics.

The scheme below displays metrics collected by the VTune Profiler across different parts of the Intel® Processor Graphics Gen9:

GPU metrics help identify how efficiently GPU hardware resources are used and whether any performance improvements are possible. Many metrics are represented as a ratio of cycles when the GPU functional unit(s) is in a specific state over all the cycles available for a sampling period.

Configure the Analysis

Run the Analysis

  1. In the VTune Profiler GUI, select the Configure Analysis button.

  2. In the HOW pane, from the Analysis Tree, select GPU Compute/Media Hotspots (Preview) analysis in the Accelerators group.

    Note

    If you have multiple Intel GPUs connected to your system, run the analysis on the GPU of your choice or on all connected devices. For more information, see Analyze Multiple GPUs.

  3. Choose and configure one of these analysis modes:

  4. Optionally, narrow down the analysis to specific kernels you identified as performance-critical (stalled or time-consuming) in the GPU Offload analysis, and specify them as Computing tasks of interest to profile. If required, modify the Instance step for each kernel, which is a sampling interval (in the number of kernels). This option helps reduce profiling overhead.
  5. (Optional) To collect data on energy consumption, check the Analyze power usage option. This feature is available when you profile applications in a Linux environment and use an Intel® Iris® X e MAX graphics discrete GPU.
  6. For GPUs with interconnect (Xe Link) connections, use the Analyze Xe Link Usage option to examine the traffic between GPU interconnects (or Xe links). This information can help you assess data flow between GPUs and the usage of their interconnects.

  7. Click Start to run the analysis.

Run from Command Line

To run the GPU Compute/Media Hotspots analysis from the command line, type:

vtune -collect gpu-hotspots [-knob <knob_name=knob_option>] -- <target> [target_options]

Note

To generate the command line for this configuration, use the Command Line... button at the bottom.

Analyze Multiple GPUs

If you connect multiple Intel GPUs to your system, VTune Profiler identifies all of these adapters in the Target GPU pulldown menu. Follow these guidelines:

Once the analysis completes, VTune Profiler displays summary results per GPU including tile information in the Summary window.

Analysis Results

Once the GPU Compute/Media Hotspots Analysis completes data collection, the Summary window displays metrics that describe:

Note

Families of Intel® Xe graphics products starting with Intel® Arc™ Alchemist (formerly DG2) and newer generations feature GPU architecture terminology that shifts from legacy terms. For more information on the terminology changes and to understand their mapping with legacy content, see GPU Architecture Terminology for Intel® Xe Graphics.
Analysis Results for Multiple GPUs

When you profile an application running on multiple Intel® GPUs, the Summary window of the GPU Compute/Media Hotspots Analysis displays results by grouping GPUs of the same Intel microarchitecture. Each architecture group then contains metric information for that group.

For each architecture group, you can see the values for these metrics:

For every adapter in an architecture group, you can also see the values for the XVE Array Active/Stalled/Idle metrics.

Naming Convention for GPU Adapters

The results of GPU profiling analyses use aliases to refer to GPU adapters. .

The GPU Topology Diagram

When you run a GPU analysis across multiple Intel GPUs (or multi-stack GPUs) connected to your system, the Summary window displays interconnections between these GPUs in the GPU Topology diagram. This diagram contains cross-GPU information for a maximum of 2 sockets and 6 GPUs connected to the system.

The GPU Topology diagram displays topological information about the sockets (available for GPU connection) as well as interconnect (Xe Link) connections between GPUs. You can identify GPUs in the GPU Topology diagram by their Bus Device Function (BDF) numbers.

Hover over a GPU stack to see actively utilized links (highlighted in green) and corresponding bandwidth metrics.

Use the information presented here to see average data transferred:

Analyze Xe Link Usage

For GPUs with Xe Link connections, when you check the option (before running the analysis) to analyze interconnect (Xe Link) usage, the Summary window includes a section that displays the aggregated bandwidth and traffic data through GPU interconnects (Xe Link). Use this information with the GPU Topology diagram to detect any imbalances in the distribution of traffic between GPUs. See if some links are used more frequently than others, and understand why this is happening.

Analyze Xe Link Usage
Analyze Bandwidth Data Over Time

Along with the Xe Link usage information in the Summary window, the Platform window displays bandwidth data over time.

Use this information to:

Configure Characterization Analysis

Use the Characterization configuration option to:

When you select the Characterization radio button, you can select platform-specific presets of GPU metrics. With the exception of the Dynamic Instruction Count preset, all other presets collect the following data about the activity of Execution Units (EU):

Each preset introduces additional metrics:

Note

You can run the GPU Compute/Media Hotspots analysis in Characterization mode for Windows*and Linux* targets. However, for all presets (with the exception of the Dynamic Instruction Count preset), you must have root/administrative privileges to run the GPU Compute/Media Hotspots analysis in Characterization mode.

Alternatively, on Linux* systems, you can configure the system to allow further collections for non-privileged users. To do this, in the bin64 folder of your installation directory, run the prepare-debugfs-and-gpu-environment.sh script with root privileges.

Additional Data for Characterization Analysis

Use the Trace GPU programming APIs option to analyze SYCL, OpenCL™, or Intel Media SDK programs running on Intel Processor Graphics. This option may affect the performance of your application on the CPU side.

For SYCL or OpenCL applications, identify the hottest kernels and the GPU architecture block that contains a performance issue for a particular kernel.

For Intel Media SDK programs, examine the execution of Intel Media SDK tasks on the timeline. Correlate this data with GPU usage at each instant.

Usage Considerations:

Configure Source Analysis

In the Source Analysis, VTune Profiler helps you identify performance-critical basic blocks, issues caused by memory accesses in the GPU kernels.

When you select the Source Analysis radio button, the configuration pane expands a drop-down menu where you can select a profiling mode to specify a type of issues you want to analyze:

In the Basic Block Latency or Memory Latency profiling modes, the GPU Compute/Media Hotspots analysis uses these metrics:

If you enable the Instruction count profiling mode, VTune Profiler shows a breakdown of instructions executed by the kernel in the following groups:

Instruction count profiling mode

Control Flow group

if, else, endif, while, break, cont, call, calla, ret, goto, jmpi, brd, brc, join, halt and mov, add instructions that explicitly change the ip register.

Send & Wait group

send, sends, sendc, sendsc, wait

Int16 & HP Float | Int32 & SP Float | Int64 & DP Float groups

Bit operations (only for integer types): and, or, xor, and others.

Arithmetic operations: mul, sub, and others; avg, frc, mac, mach, mad, madm.

Vector arithmetic operations: line, dp2, dp4, and others.

Extended math operations.

Other group

Contains all other operations including nop.

In the Instruction count mode, the VTune Profiler also provides Operations per second metrics calculated as a weighted sum of the following executed instructions:

Note

The type of an operation is determined by the type of a destination operand.

View Data

VTune Profiler runs the analysis and opens the data in the GPU Compute/Media Hotspots viewpoint providing various platform data in the following windows:

Support for SYCL* Applications using oneAPI Level Zero API

This section describes support in the GPU Compute/Media Hotspots analysis for SYCL applications that run OpenCL or oneAPI Level Zero API in the back end. VTune Profiler supports version 0.91.10 of the oneAPI Level Zero API.

Support Aspect

SYCL application with OpenCL as back end

SYCL application with Level Zero as back end

Operating System

Linux OS

Windows OS

Linux OS

Windows OS

Data collection

VTune Profiler collects and shows GPU computing tasks and the GPU computing queue.

VTune Profiler collects and shows GPU computing tasks and the GPU computing queue.

Data display

VTune Profiler maps the collected GPU HW metrics to specific kernels and displays them on a diagram.

VTune Profiler maps the collected GPU HW metrics to specific kernels and displays them on a diagram.

Display Host side API calls

Yes

Yes

Source Assembler for computing tasks

Yes

Yes

Instrumentation for GPU code (Source Analysis option or Dynamic Instruction Count characterization option)

Yes

Yes

Note

For a use case on profiling a SYCL application running on an Intel GPU, see Profiling a SYCL App Running on a GPU in the Intel® VTune Profiler Performance Analysis Cookbook .

Support for DirectX Applications

This section describes support available in the GPU analysis to trace Microsoft® DirectX* applications running on the CPU host. This support is available in the Launch Application mode only.

Support Aspect DirectX Application

Operating system

Windows OS

API version

DXGI, Direct3D 11, Direct3D 12, Direct3D 11 on 12

Display host side API calls

Yes

Direct Machine Learning (DirectML) API

Yes

Device side computing tasks

No

Source Assembler for computing tasks

No

See Also