DPC++ L0 Switch

Level Zero (L0)

Data Parallel C++ (DPC++) is just one of the many components of the oneAPI project. The oneAPI Level Zero (L0) API provides low-level direct-to-metal interfaces that are tailored to the devices on a oneAPI project. While heavily influenced by other low-level APIs, such as OpenCL™, L0 is designed to evolve independently.

More information on L0 is available in the specification.

Packages to Install

The packages you must install are intel-level-zero-gpu and level-zero.

L0 Loader

L0 is supportable across different oneAPI compute device architectures. The L0 loader discovers all L0 drivers in the system. In addition, the L0 loader is also the L0 SDK: It carries the L0 headers and libraries where you build L0 programs.

Notes

The L0 loader does not come with DPC++ and therefore must be installed independently.

More information about the L0 loader is available at Loader section of the oneAPI Level Zero Specification.

L0 GPU Driver

The first L0 driver was created to support an Intel GPU: Gen90+. The driver is open-source and regular public releases are maintained. It does not come with DPC++ and therefore must be installed independently. The L0 driver and OpenCL driver come in the same package. More info about the L0 driver is available at GitHub.

DPC++ Plugins

DPC++ targets a variety of devices: CPU, GPU, and FPGA. Different devices can be operated through different low-level drivers, such as OpenCL for FPGA. The Plugin Interface (PI) is a unified DPC++ API for working with different devices in a unified way. Plugins of DPC++ implement specific translations of the PI API into low-level runtime. The L0 PI Plugin was created in DPC++ to enable devices supported through the L0 system. More information on PI is available at GitHub.

Scenario Information
DPC++ Device Selection

The PI for DPC++ performs device discovery of all available devices through all available PI plugins. The same physical hardware device can be seen as multiple different DPC++ devices if multiple plugins support it (for example, OpenCL Gen90 and L0 Gen90). The DPC++ runtime performs device selection from the available devices based on device selectors. The device selectors can be user-defined or built in (for example, gpu_selector).

Discovery of Multiple PI Plugins

The implication of support for the discovery of multiple plugins is that the same GPU card can be seen as multiple different GPU devices available under different PI plugins.

Note

Corresponding runtimes (OpenCL and/or L0) must be installed correctly and independently for PI to see their devices. The DPC++/SYCL* specification does not define which device will be used if there are multiple devices that match criteria (for example, is_gpu()).
Default Preference is Given to an L0 GPU on Linux*

By default, if no special action is taken and the L0 runtime reports support for the installed GPU, then the DPC++ runtime uses the installed GPU. This is true for standard built-in device selectors and custom device selectors, where no action is taken to change the default behavior.

Currently, on Windows*, the preference is given to an OpenCL GPU.

Devices that are not supported with the L0 runtime (CPU/FPGA) continue to run with OpenCL.

How to Change the Default Preference

Use the SYCL_BE environment variable to change the default preference. The valid values are PI_OPENCL and PI_LEVEL0.

For example, if you specify SYCL_BE=PI_OPENCL and the PI OpenCL plugin reports the availability of the device of the required type, then that device is used. It overrides the default preference that is given to the L0 GPU, if the GPU is supported by the installed version of OpenCL.

Note

The SYCL_BE setting only works when there are multiple choices.

Recommendation

If your code does not work out of the box, try running it with SYCL_BE=PI_OPENCL to see if the problem is related to L0.
How to See Where the Code is Running

Use the SYCL_PI_TRACE=1 environment variable to see where your code is running. It reports the choice made by the built-in device selectors, if they are used.

Use SYCL_PI_TRACE=-1 to enable verbose tracing of the PI. This shows all devices detected by the PI discovery process.

How to Load all SYCL Plugins Discovered in the System

Use the sycl-ls utility to load all the SYCL plugins on your system. This utility also queries all the platforms and devices available through the plugins.

There is a verbose output available with $ sycl-ls --verbose. This gives you the same choice that would be made by a standard built-in device selector.