AMD OpenCL

Jump to: navigation, search

OpenCL (GPGPU) on AMD (ATI) GPUs

Warning about support level

Support of OpenCL/GPGPU on consumer AMD video cards with Linux is rather poor (Exceptions are cards with the same GPU chips as used in professional hardware. For instance - discrete Vega (GCN5) - Radeon Rx Vega 56/64 or Radeon VII). Support of OpenCL/GPGPU on AMD APU (iGPU) from AMD is near zero. Use another solutions if you need OpenCL/GPGPU for professional usage. Right now this is Nvidia only. Intel consumer graphics lacks FP64 support (link1, link2). Or use AMD GCN5/CDNA Pro products. For notebooks with AMD CPU + iGPU: if notebook has Nvidia discrete video card, then it is usable for OpenCL/GPGPU. Decent support level for AMD will be achieved only after maturing of Mesa Rusticl driver. PAL driver is good (but doesn't tested with RDNA2 & newer). Comparison of available drivers and support level with LuxMark you can check here (may include errors).


Preface

Using GPU for computing can give you huge speed-up – 3x, 10x, 100x, or even more. We can describe modern GPU as a set of many (100 – 10000) rather simple CPUs, connected to very fast dedicated RAM. To use this power you need drivers, which support some compute standards – OpenCL, CUDA, Vulkan, or something else. AMD supports OpenCL, CUDA via HIP translator, Vulkan, and other standards.

You don’t need OpenCL for desktop acceleration, web browsers and other ordinary tasks, you need only 3D drivers for that: OpenGL, sometimes Vulkan. For hardware acceleration Firefox Webrender requires OpenGL 3.0, KDE requires OpenGL 3.1.

Check hardware specifications with AMD GPU list here.

You can get some acceleration in some applications with your GPU(s) after installing needed drivers. For instance, you can use GPU acceleration in Darktable and Blender after changing appropriate settings. Also you can use GPU to accelerate Ark or Dolphin after installing middleware drivers (see chapter "Using GPU for ordinary applications (GPGPU)" for details). For some applications you need additional tuning, read chapter Special Cases for details.

Volume of available video RAM is critical: often 1-2 GiB is a bare minimum, for complex tasks you may need 6 GiB or more. You can adjust available video RAM for built-in graphics via BIOS settings.

With OpenCL you can easily get 100% GPU load, so your desktop windows can stutter, web browser may hang. To avoid this behaviour dedicate one GPU (the simplest one, such as built-in) for desktop usage, and use other(s) for OpenCL acceleration. Also this solution will free some video RAM in accelerator for calculations.

Check supported OpenCL extensions with clinfo utility (clinfo package), measure speed with clpeak utility (clpeak package). Use package radeontop to watch GPU utilization.


Available drivers


ATI FireGL – for Terascale 2 & 3 chips

OpenCL 1.2 support, incompatible with modern distributions.

Mesa 3D Clover - for different hardware

OpenCL 1.2 support without image support, often hangs. Mesa 3D will deprecate it in favour of Rusticl driver.

Mesa 3D Rusticl - for different hardware.

Under development. May not work right now with your hardware. Was introduced with Mesa 22.3. Radeonsi support (GCN1 & newer) is expected in Mesa 23.1, feature support (OpenCL tests conformance) - in Mesa 23.2. Support with r600g (Terascale 1 & 2) - in plans.
Page: https://docs.mesa3d.org/rusticl
Certified OpenCL 3.0 support with image support, OpenCL 2.x - under development.

AMD ORCA – for AMD GCN 1 – GCN 4.

OpenCL 1.2 support.

AMD PAL – for AMD GCN 2 – GCN 5 and RDNA 1.

PAL = Platform Abstraction Library. The latest AMD driver package with PAL OpenCL inside is 20.40. Link to package is here. Newer drivers have ROCm-based OpenCL drivers for GCN5 and newer chips, and ORCA drivers for GCN1 - GCN4. Works with Leap 15.2 & 15.3 (kernel 5.3), and with Leap 15.4 (kernel 5.14.21) with manual install.

Pros:

  1. OpenCL 2.0 support with image support.
  2. Supports discrete GPU and iGPU (APU).
  3. Doesn't require PCIe atomics support - works with old hardware.
  4. Doesn't require DKMS.

Cons:

  1. Old and unsupported. Dropped by AMD in late 2020.
  2. Closed-sourced. No community support.
  3. HIP doesn't work right now with this driver.

AMD ROCm drivers for OpenCL – for AMD GCN 5 and RDNA 1–3.

OpenCL 2.x support. For discrete video cards only (unofficial support for Ryzen APUs - GCN5 iGPU and newer).


Combining drivers

You can combine drivers. For instance, you can use 3D drivers from one source, and OpenCL - from another one. Drivers for AMD hardware consist from four parts: kernel driver (amdgpu for GCN 1 & newer), OpenGL drivers, Vulkan drivers, OpenCL drivers. For Vulkan and OpenCL drivers you can install more than one set of drivers, and activate needed one with editing icd files.
Method of using OpenCL PAL AMD driver, which is described beneath, combines amdgpu + Mesa 3D drivers for OpenGL & Vulkan for Leap 15.x with OpenCL PAL AMD driver for Leap 15.2.

Sources of amdgpu driver

AMD repository for amdgpu, Mesa 3D stuff and etc. is here. Choose appropriate subfolder.
For Leap you can use amdgpu driver from AMD packages or from distribution (xf86-video-amdgpu package). To use amdgpu driver from AMD packages you need working DKMS. AMD tests ROCm with amdgpu driver from AMD packages. Amdgpu driver from distribution works OK and is easily maintained for end user. It is better to use one source of amdgpu driver, otherwise system will hop from one to another.
With Tumbleweed you have to use amdgpu driver from distribution.

Sources of 3D (OpenGL & Vulkan) drivers

You can use up to four repositories for openSUSE Leap (SUSE SLE) to get video drivers:

  1. Default 3D drivers (Mesa 3D, included in distribution)
  2. The newest drivers Mesa 3D drivers from so-called experimental X1:XOrg repository
  3. Mesa 3D drivers from AMD drivers package (repository, hosted by AMD)
  4. AMD proprietary drivers from AMD drivers package (repository, hosted by AMD)

Vulkan driver from Mesa 3D bears name RADV, Vulkan driver from AMD has name AMDVLK. Mesa 3D drivers can be faster for games, while AMD proprietary drivers may support more OpenGL/Vulkan extensions.

Sources of OpenCL drivers

We have two options: AMD or Mesa 3D.


Installing drivers


ATI FireGL

You may install it with downgrading X and other core packages. Instead of that use old & compatible distribution. You can use virtual machine for that.

Mesa 3D Clover

Available from Main repository. This driver hangs too much. Simply do not use it.

Mesa 3D Rusticl

Install Rusticl package. Use the latest available Mesa 3D drivers.
For Tumbleweed simply install Rusticl package.
(Not available right now for Leap 15.4 & 15.5). For openSUSE Leap 15.5 add Mesa 3D experimental repository (X11:XOrg). Then install Rusticl package from it. Supposedly you will need to perform vendor change upgrade using this repo.

AMD ORCA

For Leap 15.4 use installation script with "--opencl=legacy" option.
For Leap 15.3 install AMD proprietary drivers – all packages or OpenCL only (‘headless’ option). With installing headless AMD drivers you will use Mesa 3D drivers for 3D and AMD OpenCL drivers for OpenCL.
For GCN 1 extract ORCA drivers from 20.10 drivers.
You need amdgpu kernel driver to use AMD OpenCL drivers. For GCN 1 & 2 amdgpu driver is still in experimental phase, so you need kernel parameters to use amdgpu instead of radeon one. Add to kernel parameters “radeon.si_support=0 amdgpu.si_support=1” for GCN1, and “radeon.cik_support=0 amdgpu.cik_support=1” for GCN2. See here for details.
With GCN 1 & 2 and amdgpu driver VGA output in video card may not work. For GCN 2 – 4 with AMD PAL drivers you will get OpenCL 2.0 instead of OpenCL 1.2.

AMD PAL

For Leap 15.3 use this instruction. With installing headless AMD drivers you will use Mesa 3D drivers for 3D and AMD OpenCL drivers for OpenCL.
Usage with Leap 15.4 is under tests.

AMD ROCm

Documentation is here.
Doesn't work with Leap 15.4 - bug filed.

Foreword

ROCm is a fully open source solution from AMD for calculations on AMD GPUs. AMD creates ROCm mainly for supercomputer market, for CDNA product line and its predecessor GCN5 (a.k.a. Vega graphics). Only PRO series video cards support ROCm by default, and only discrete video cards. You can use ROCm with consumer video cards with adjusted environment variable - see chapter "ROCm - Running on unsupported hardware" for details. GCN3 & GCN4 may work, possibly after recompiling of ROCm with changed parameters, GCN1 - GCN2 generations need tests & additional work. OpenCL on iGPUs with GCN5 graphics works with some old ROCm releases with reduced speed (compared to PAL driver). Special edition of ROCm 3.10 (and 3.3.0) for APUs is available from Bruhnspace, but it may bear some limitations.

Requirements

Reference Guide about supported hardware is here, it is usually outdated and has mistakes. For AMD ROCm drivers with pre-Vega (i.e. GCN 4 and older ones) GPUs you need support for PCIe atomics feature (PCIe 3.0 and newer). Some old CPUs and motherboards do not support PCIe atomics. AMD Ryzen family (including Athlon 200GE/300GE/3000G and newer) supports PCIe atomics. AMD AM3+ and older sockets lacks PCIe atomics.
To check compatibility use command (with GPU being installed) 

sudo dmesg | grep -i kfd

If you find something like 

kfd kfd: added device 1002:15d8

, then your system supports PCIe atomics. Without PCIe atomics you will get something like 

kfd: skipped device 1002:7300, PCI rejects atomics

Installation

Add user to "video" group. Install AMD drivers using installation script "amdgpu-install" after installing package "amdgpu-install" or add one of ROCm repository and install needed packages from it. Some ROCm (https://repo.radeon.com/rocm/zyp/) repos (use one of them):

https://repo.radeon.com/rocm/zyp/latest/
https://repo.radeon.com/rocm/zyp/zypper/
https://repo.radeon.com/rocm/zyp/5.3.2/

Sometimes you need full re-installation of ROCm packages, not just simple upgrade.

ROCm - Running on unsupported hardware

You can force ROCm (HIP) to run on unsupported hardware with HSA_OVERRIDE_GFX_VERSION environment variable (link 1, link 2, link 3). For instance, you may use 

export HSA_OVERRIDE_GFX_VERSION=10.3.0

This tricks ROCm into treating your card as a Navi 21 card (link). It is better to use GFX number from the same generation. For GCN use GCN substitutes, not RDNA ones. To emulate Vega (GCN5) try this one: 

export HSA_OVERRIDE_GFX_VERSION=9.0.0


Using GPU for ordinary applications (GPGPU)

It is possible to use OpenCL acceleration with ordinary applications with working appropriate drivers. Tests were conducted using Ark compression and Dolphin for calculating checksums. No acceleration and no additional GPU load were observed with Leap 15.4 and installed libdrm-amdgpu package from AMD repository with uninstalled libdrm_amdgpu1 counterpart from Mesa 3D.


OpenCL on openSUSE Tumbleweed

While AMD supports OpenCL on SUSE SLE (= openSUSE Leap) only, you can use AMD OpenCL drivers with Tumbleweed by manually choosing which packages to install. Or you can compile drivers by yourself.
Of course, you can use Mesa 3D OpenCL drivers with Tumbleweed.


OpenCL on APU (iGPU)

AMD provides no official support for OpenCL on integrated GPUs. ROCm doesn't work with Leap 15.4 right now - bug filed. Some old ROCm versions support iGPU, but they work much slower than expected (i.e. with PAL driver). Driver installer supports legacy option, but this legacy driver is ORCA driver, supports GCN1 - GCN4, and has no support for GCN5 (Vega) and newer GPUs.
PAL driver works. But AMD drops PAL driver in favour of ROCm, so no new versions are available since 20.40 driver. That hints on absence of support for RDNA2 and newer chips with it.
Good option is using Mesa Rusticl OpenCL driver.


Special Cases

Blender

Download archive from blender.org, extract, and use it as a portable install. Blender doesn’t support Terascale chips at all, needs GCN 1 and newer just to run (however, may work with software emulation). Blender 2.x works OK with OpenCL. Blender 3.x doesn't support OpenCL, emits CUDA commands, requires working AMD HIP translator from AMD ROCm. For GCN5 use HIP 5.3 and newer. GPUs older than GCN5 are not supported with Blender 3.x + HIP.

DaVinci Resolve

DaVinci Resolve requires unconventional OpenCL extensions cl_khr_gl_sharing and cl_khr_image2d_from_buffer, which may not be available in some drivers. Read here for details: link1, link2, link3. Also you may need to install Workstation (Proprietary) (previously AMDGPU-Pro) OpenGL drivers. Support with Rusticl is expected due to this patch.


Hints

  1. You can install and use more than one OpenCL driver after configuring installed drivers with icd files located in "/etc/OpenCL/vendors/".
  2. Old AMD drivers contents installer and all packages in it. Script creates subfolder in system folder containing all packages and then installs packages from it. New AMD drivers installer uses AMD repositories at https://repo.radeon.com.
  3. You need special tuning to use amdgpu and radeon drivers simultaneously when system has GCN 1 or GCN 2 GPUs.
  4. Installation of ROCm OpenCL drivers blacklists "radeon" video driver by default.
  5. Terascale chips lacks some commands for addressing memory, so you need some tuning to use OpenCL in Darktable via settings in file - read Darktable manual for details.
  6. OpenCL on Terascale 1 and older AMD/ATI chips is not supported (well, it is possible, but meaningless).
  7. AMD officially supports OpenCL only with discrete video cards. However, OpenCL may work on APU.
  8. You can use CPU(s) as OpenCL device after installing PoCL package or with LLVMpipe + Rusticl.
  9. You can run calculations on GPU and on CPU together.
Retrieved from "https://en.opensuse.org/index.php?title=AMD_OpenCL&oldid=174662"