SDB:AMD GPGPU

Jump to: navigation, search

GPGPU on AMD (ATI) GPUs

General info about AMD (ATI) hardware: https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units

For general GPGPU info see SDB:GPGPU.

GPGPU = General-purpose computing on graphics processing units, means using GPUs for common computations.


Introduction

Warning about support level

ATI in 1990s was famous for good hardware and buggy drivers. After acquisition ATI in 2006 AMD carefully preserves this tradition. Even if AMD creates good driver - PAL OpenCL, it rapidly drops it and substitutes it with semi-working ROCm. OpenGL and Vulkan support is good due to open drivers contributed by Mesa 3D and Valve.

AMD had quit GPGPU consumer market on Linux in 2020 after dropping PAL OpenCL driver. ROCm, which substitutes PAL, works on a small part of hardware, and is supported on even smaller number of GPUs. Support of GPGPU on AMD APU (iGPU) from AMD is near zero and requires manual settings. You can use Nvidia and Intel GPUs. But Intel consumer graphics Arc A-series lack FP64 support (link1, link2). For notebooks with AMD CPU + iGPU: if notebook has Nvidia or Intel discrete video card, then it is usable for GPGPU. Decent support level of GPGPU via OpenCL for AMD GPUs will be achieved only after maturing of Mesa's Rusticl driver. PAL driver is good. But Rusticl and PAL drivers provide OpenCL support only.

Comparison of available drivers and support level with LuxMark you can check here (might include errors).


Support period

Support period for AMD products from AMD is about 5 years. After 5-6 years since first release AMD drops support for its products. Some exclusions: GCN5 (GCN5.1) might get longer support due to using it for iGPUs in APUs (desktop & mobile).


Using CUDA code on AMD hardware


Using CUDA code with ZLUDA and ROCm

You can use CUDA applications after installing (running) ZLUDA code translator. ZLUDA needs working ROCm installation (currently ROCm 5.7.x). Read more at https://github.com/vosen/ZLUDA and Phoronix article.
To run ROCm 5.7 on openSUSE with RDNA3 chip use environment variable 

export HSA_OVERRIDE_GFX_VERSION=10.3.0

Read more here about compatibility options.


Using CUDA code with "SCALE" Software and ROCm

You can use CUDA applications after installing (running) "SCALE" code translator (closed source). "SCALE" needs working ROCm installation (ROCm 6.x). Announce is here. Read more at https://docs.scale-lang.com/ and Phoronix article.


Converting CUDA and HIP to OpenCL and Level Zero - chipStar

ChipStar is a tool for compiling and running HIP/CUDA on SPIR-V via OpenCL or Level Zero APIs. ChipStar enables porting HIP and CUDA applications to platforms which support SPIR-V as the device intermediate representation. It supports OpenCL and Level Zero as the low-level runtime alternatives. More at https://github.com/CHIP-SPV/chipStar.
In future it will be possible to use chipStar with Mesa 3D Rusticl to run CUDA code: CUDA code -> convert by chipStar into OpenCL code -> run this code with Rusticl OpenCL driver (using cl_ext_buffer_device_address extension, Rusticl commit).


Legal issues

Regulation differs in different states, common case is: if you have source code written in CUDA language, then you can convert it with software (mainly non-Nvidia) to something that you can use - OpenCL, ROCm HIP, oneAPI, any other. Then you can execute that code on non-Nvidia devices. But you cannot use Nvidia proprietary software in that chain.
So, chipStar is legal, but ZLUDA is not.


AdaptiveCpp - heterogeneous C++ compiler

AdaptiveCpp is an open-source C++ compiler formerly known as hypSYCL and Open SYCL. AdaptiveCpp is the independent, community-driven modern platform for C++ -based heterogeneous programming models targeting CPUs and GPUs from all major vendors.
Page: https://adaptivecpp.github.io/AdaptiveCpp/, Github: https://github.com/AdaptiveCpp/AdaptiveCpp
How to install: https://adaptivecpp.github.io/AdaptiveCpp/installing/

  1. For ROCm - https://adaptivecpp.github.io/AdaptiveCpp/install-rocm/
  2. For SPIR-V/OpenCL - https://adaptivecpp.github.io/AdaptiveCpp/install-ocl/


Available drivers

ATI FireGL – for Terascale 2 & Terascale 3 chips

OpenCL 1.2 support, incompatible with modern distributions.


Mesa 3D Clover - for different hardware

OpenCL 1.2 support without image support, often hangs. Mesa 3D is dropping it in favour of Rusticl driver (news). Clover will be removed in Mesa 25.2 (news, merge request 1, merge request 2).


Mesa 3D Rusticl - for different hardware

It provides OpenCL 3.0 support with image support, some OpenCL 2.x features (under development, partly implemented). Available for Tumbleweed, Leap 15.5 and newer.
Under development. Use the newest version available (X11:XOrg or devel:gcc:next:testing or Community repos). Was introduced with Mesa 22.3. Radeonsi support (GCN1 & newer) came with Mesa 23.1, feature support (OpenCL tests conformance) - with Mesa 23.2. It works good enough since Mesa 23.3.
Page: https://docs.mesa3d.org/rusticl. Supported extensions: https://mesamatrix.net/.

AMD Terascale architecture issues:

  • Terascale 1 is not supported.
  • Image support is not available for Rusticl driver on Terascale 2 & 3.
  • Mesa 3D is dropping OpenCL support for Terascale 2 & 3 (with r600g driver).


AMD ORCA – for AMD GCN1 – GCN4

OpenCL 1.2 support. Lacks some extensions.
Driver 5.7.1 (23.20) is the last with ORCA OpenCL (a.k.a. "legacy OpenCL") packages.


AMD PAL – for AMD GCN2 – GCN5 and RDNA1

PAL = Platform Abstraction Library. The latest AMD driver package with PAL OpenCL inside is 20.40. Link to package is here. Newer drivers have ROCm-based OpenCL drivers for GCN5 (if not dropped) and newer chips, and ORCA drivers for GCN1 - GCN4. Works with Leap 15.2 & 15.3 (kernel 5.3), and with Leap 15.4 & 15.5 (kernel 5.14) after manual install. It might work with RDNA2 & later - not tested yet.

Pros:

  1. OpenCL 2.0/2.1 support with image support.
  2. Supports discrete GPU and iGPU (APU).
  3. Doesn't require PCIe atomics support - works with old hardware.
  4. Doesn't require DKMS.

Cons:

  1. Unsupported. Dropped by AMD in late 2020.
  2. Closed-sourced. No community support.
  3. Possibly HIP doesn't work right now with this driver.


AMD ROCm drivers – for selected AMD GCN and RDNA

OpenCL 2.x & HIP support. For discrete video cards only (unofficial support for Ryzen APUs - GCN5 iGPU and newer). For unsupported hardware use parameters or patches. ROCm packaged by AMD is available for openSUSE Leap (SUSE SLE), for Tumbleweed compile it by yourself. Some parts of ROCm are available - read here for details.


Support history:

  1. GCN3: AMD mentions community-supported ROCm.
  2. GCN4: ROCm 4.5 had dropped Polaris (GCN4, gfx803) support, so the last ROCm version for it is 4.3.1.
  3. GCN5.0: The last ROCm with support for GCN 5.0 (Vega 10, Raven, Picasso, gfx900) is 4.5.2.
  4. GCN5.1:The last ROCm series with full support for GCN 5.1 (Vega 20, Radeon VII, Renoir, Cezanne, gfx906) is 5.7.
  5. RDNA1: Some enhancements with ROCm 6.2 - link. No official support, although might work. Use "export HSA_OVERRIDE_GFX_VERSION=10.3.0" parameter. Read more here.
  6. RDNA2: Official support with ROCm 5.x and newer.
  7. RDNA3: Support starts with ROCm 5.7.1 for Ubuntu 22.04 and Windows, for openSUSE Leap (SUSE SLE) use ROCm 6.0 and newer. You can use Distrobox to get it running.


Incompatibility of AMD ROCm with iGPU from supported architectures

ROCm (v. 6.x right now) is not supported on discrete AMD GPU if system has integrated GPU (iGPU) with architecture supported by ROCm (link).
Citation from https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/amdgpu-install.html:

Warning

ROCm doesn’t currently support integrated graphics. Should your system have an AMD IGP installed, disable it in the BIOS prior to using ROCm. If the driver can enumerate the IGP, the ROCm runtime may crash the system, even if told to omit it via HIP_VISIBLE_DEVICES.

It is rather difficult to use ROCm 6.x with {AMD GCN5.1 or AMD RDNA2 or RDNA3} iGPU + {AMD GCN5.1 or AMD RDNA2 or RDNA3} dGPU. You can disable iGPU on AM5 Ryzen 7000 desktop series. Also that means all new laptops are semi-incompatible with ROCm 6.x. Possibly it works OK with GCN5.0 iGPU - ROCm just ignores it. Otherwise we have troubles in usage ROCm 6.x with AMD iGPU + AMD dGPU on any modern laptop.
It is possible to overcome this issue by patching AMD KFD (amdkfd) driver - link1, link2. But these patches are out of tree, i.e. no support is provided by Linux kernel developers and AMD both. Other options: use virtual machine or Distrobox or Windows.
Integrated AMD GPU without discrete AMD GPU possibly works somehow.


Vulkan for GPGPU

It is possible to use Vulkan drivers for computations. It works, but possibly slower than more direct approaches. Another feature: no warranties for arithmetic precision, because graphics output requires speed and can allow ignoring accuracy for results. So, Vulkan works well for AI tasks, but usage for precision computations is not recommended. Nevertheless, some software is using Vulkan for GPGPU.
To get support you need working Vulkan drivers. You can use AMDVLK driver from AMD or RADV driver from Mesa 3D. Vulkan drivers are available for GCN3+ by default. For GCN1 & GCN2 use kernel parameters to get drm/amdgpu driver instead of drm/radeon, read more here. For Terascale 2 & 3 driver named as "Terakan" is under development (Vulkan 1.0?).
You can install AMDVLK and RADV both and switch between them on-the-fly. You can load needed driver with LD_LIBRARY_PATH or LD_PRELOAD prefix.

RADV driver Vulkan API support (source):

  • GCN1 & GCN2 (GFX6 & GFX7): Vulkan 1.3
  • GCN3 (GFX8) and newer: Vulkan 1.4

More info about supported extensions you can find at https://mesamatrix.net/.


Ray Tracing

Some programs can use Ray Tracing (raytracing, RT) hardware blocks to accelerate their performance. For instance, Blender can use AMD HIP RT library (a part of AMD ROCm). Hardware Ray Tracing is available since AMD RDNA2 architecture. Big improvements were made in AMD RDNA4 architecture. For Linux you can use Vulkan RT API with AMD AMDVLK or MESA 3D RADV drivers, often AMDVLK works faster with RT.


Combining drivers

You can combine drivers. For instance, you can use 3D drivers from one source, and OpenCL - from another one. Drivers for AMD hardware consist of four parts: kernel driver (amdgpu for GCN1 & newer), OpenGL drivers, Vulkan drivers, OpenCL drivers. For Vulkan and OpenCL drivers you can install more than one set of drivers, and activate needed one by editing icd files.
Method of using OpenCL PAL AMD driver, which is described beneath, combines amdgpu + Mesa 3D drivers for OpenGL & Vulkan for Leap 15.x with OpenCL PAL AMD driver for Leap 15.2.


Sources of amdgpu driver

AMD repository for amdgpu, Mesa 3D stuff and etc. is here. Choose appropriate subfolder.
For Leap you can use amdgpu driver from AMD packages or from distribution (xf86-video-amdgpu package). To use amdgpu driver from AMD packages AMD recommends using DKMS. AMD tests ROCm with amdgpu driver + DKMS from AMD packages. Amdgpu driver from distribution works OK and is easily maintained for end user. It is better to use one source of amdgpu driver, otherwise system will hop from one to another.
With Tumbleweed you have to use amdgpu driver from distribution. It is possible to compile AMD drivers for Tumbleweed.


Sources of 3D (OpenGL & Vulkan) drivers

You can use up to four repositories for openSUSE Leap (SUSE SLE) to get video drivers:

  1. Default 3D drivers (Mesa 3D, included in distribution, Main repository)
  2. The newest drivers Mesa 3D drivers from so-called experimental X11:XOrg repository
  3. Mesa 3D drivers from AMD drivers package (https://repo.radeon.com/amdgpu/ repository, hosted by AMD)
  4. AMD proprietary drivers from AMD drivers package (https://repo.radeon.com/amdgpu/ repository, hosted by AMD)

Vulkan driver from Mesa 3D bears name RADV, Vulkan driver from AMD has name AMDVLK. Mesa 3D drivers can be faster for games, while AMD proprietary drivers might support more OpenGL/Vulkan extensions.
AMD has dropped support for GCN4 & GCN5. The latest proprietary drivers for them are 23.20 (23.30?) and 5.7.x. Open drivers work OK.


Sources of OpenCL drivers

We have two options: AMD or Mesa 3D.


Installing drivers

ATI FireGL

You may install it with downgrading X11 and other core packages. Instead of that you may use old & compatible distribution. You can use virtual machine for that.


Mesa 3D Clover

Available from Main repository. This driver hangs too much. Simply do not use it.


Mesa 3D Rusticl

Install Rusticl package. Use the latest available Mesa 3D drivers.
For Tumbleweed simply install Rusticl package.
Not available for Leap with default repos, not available with experimental repository even after discussion.
For openSUSE Leap you can add compatible Mesa 3D Community repository (not supported by openSUSE developers - use at your own risk). Then install Mesa 3D and Rusticl packages from it. You may perform vendor change update using this repo to get upgrade for all graphics drivers.

If Mesa 3D from X11:XOrg repo for openSUSE Leap is available, then you can use these commands (change version number (here it is "15.6") according to your needs): 

zypper addrepo https://download.opensuse.org/repositories/X11:XOrg/openSUSE_Leap_15.6/X11:XOrg.repo
zypper refresh
zypper install Mesa
zypper install Mesa-libRusticlOpenCL


For community repo you can use Home:videoregataos. Change version number (here it is "15.6") according to your needs. 

zypper addrepo https://download.opensuse.org/repositories/home:videoregataos/15.6/home:videoregataos.repo
zypper refresh
zypper install Mesa
zypper install Mesa-libRusticlOpenCL


Enabling Rusticl

For Mesa 3D 24.3+ you may not need this parameter.
In order to use Rusticl on any platform the environment variable RUSTICL_ENABLE has to be used. Rusticl does not advertise devices for any driver by default yet as doing so can impact system stability until remaining core issues are ironed out.
To enable all radeonsi devices use 

RUSTICL_ENABLE=radeonsi

Enabling second iris and first and third radeonsi device: 

RUSTICL_ENABLE=iris:1,radeonsi:0,2


Enabling Rusticl experimental features

Use RUSTICL_FEATURES environment variable to enable experimental features. Those are disabled by default as they might not be stable enough or break OpenCL conformance. Add comma-separated list of features to enable.

  • fp16 enables OpenCL half precision support
  • fp64 enables OpenCL double precision support

FP64 (double precision) format is available since GCN1 for all GPUs, FP16 (half precision) is available since GCN5 (Vega).


AMD ORCA

For Leap 15.5:

  1. Download amdgpu-install rpm package from https://repo.radeon.com/amdgpu-install/ into some folder.
  2. Add this folder as a repo.
  3. Install amdgpu-install package. This will add two repositories: "AMDGPU some_number repository" and "AMDGPU some_number Proprietary repository".
  4. After that you will be able to install AMD ORCA OpenCL driver by installing opencl-legacy-amdgpu-pro-icd package.
  5. Some versions of ROCm have empty Proprietary repository. To acquire packages select those versions which contain stuff.

For Leap 15.4 use installation script with "--opencl=legacy" option.
For Leap 15.3 install AMD proprietary drivers – all packages or OpenCL only (‘headless’ option). With installing headless AMD drivers you will use Mesa 3D drivers for 3D and AMD OpenCL drivers for OpenCL.
Another way: extract ORCA drivers from 20.10 drivers.
You need amdgpu kernel driver to use AMD OpenCL drivers. For GCN1 & GCN2 you need kernel parameters to use amdgpu instead of radeon one. Read here for details.
For GCN2 – GCN4 with AMD PAL drivers you will get OpenCL 2.0 instead of OpenCL 1.2.


AMD PAL

For Leap 15.3 use this instruction. With installing headless AMD drivers you will use Mesa 3D drivers for 3D and AMD OpenCL drivers for OpenCL.
For Leap 15.6 install OpenCL packages from 20.40 drivers and other packages from https://repo.radeon.com/amdgpu/ repo.
This instruction mimic installation of 20.40 drivers using installation script. In details for 5.7.1 repo:

  1. Download amdgpu-install rpm package into some folder.
  2. Add this folder as a repo.
  3. Install amdgpu-install package. This will add two repositories: "AMDGPU 5.7.1 repository" and "AMDGPU 5.7.1 Proprietary repository".
  4. Install these packages from AMDGPU 5.7.1 repository:
    1. amdgpu-core
    2. amdgpu-doc
    3. libdrm-amdgpu
    4. libdrm-amdgpu-common
    5. libdrm-amdgpu-devel
    6. llvm-amdgpu
    7. llvm-amdgpu-libs
    8. llvm160-amdgpu
  5. Uninstall libdrm_amdgpu1 package.
  6. Install these packages from AMDGPU 5.7.1 Pro repository:
    1. amdgpu-pro-core
    2. clinfo-amdgpu-pro
  7. Download Radeon Software for Linux 20.40 archive into some folder. Link to archive is here. Extract files from tar into some folder. Look into that folder. Find RPMS subfolder in it. Add this RPMS folder with rpm packages as "AMDGPU Pro 20.40" repo.
  8. Install these packages from this "AMDGPU Pro 20.40" repository:
    1. ocl-icd-amdgpu-pro
    2. ocl-icd-amdgpu-pro-devel
    3. opencl-amdgpu-pro-comgr
    4. opencl-amdgpu-pro-icd

To use clinfo from AMDGPU Pro repo run 

:~> /opt/amdgpu-pro/bin/clinfo


Short form of installing AMD PAL OpenCL driver

You need only two files - OpenCL driver (libamdocl64.so file) and link to it (amdocl64.icd file). You can install opencl-amdgpu-pro-icd package from Linux 20.40 archive. Or you can extract these files from package and copy them to needed locations (/etc/OpenCL/vendors/amdocl64.icd for icd file, some visible folder for libamdocl64.so file. This icd file shows OS where OpenCL driver is located).
For more info read above starting with string "Download Radeon Software for Linux 20.40 archive...". You don't need to install packages from https://repo.radeon.com/amdgpu/. You don't need to replace libdrm_amdgpu1 package with libdrm-amdgpu one.


AMD ROCm

Documentation is here. Instructions are there.
Doesn't work with GCN5.0 - bug filed.


Foreword

ROCm is a fully open source solution from AMD for calculations on AMD GPUs using HIP and OpenCL (and CUDA via translator). AMD creates ROCm mainly for supercomputer market, for CDNA product line and its predecessor GCN5 (a.k.a. Vega graphics). Only PRO series video cards support ROCm by default, and only discrete video cards. You can use ROCm with consumer video cards with adjusted environment variable - see chapter "ROCm - Running on unsupported hardware" for details. GCN3 & GCN4 may work, possibly after recompiling of ROCm with changed parameters, GCN1 - GCN2 generations need tests & additional work. OpenCL on iGPUs with GCN5 graphics works sometimes, possibly with reduced speed (compared to PAL driver). Special edition of ROCm 3.10 (and 3.3.0) for APUs is available from Bruhnspace, but it might bear some limitations.


Requirements

Reference Guide about supported hardware is here, it is usually outdated and has mistakes.
For AMD ROCm drivers you need support for PCIe atomics feature (PCIe 3.0 and newer). Some old CPUs and motherboards do not support PCIe atomics. AMD Ryzen family (including Athlon 200GE/300GE/3000G and newer) supports PCIe atomics. PCIe bridges and re-clockers also must support PCIe atomics to provide delivering this feature between CPU(s) and GPU(s). AMD AM3+ and older sockets lack PCIe atomics.
To check compatibility use command (with GPU being installed) 

sudo dmesg | grep -i kfd

If you find something like 

kfd kfd: added device 1002:15d8

, then your system supports PCIe atomics. Without PCIe atomics you will get something like 

kfd: skipped device 1002:7300, PCI rejects atomics


Installation

Add user to "video" and "render" groups. Install packages using AMD repositories after installing package "amdgpu-install", then add one of ROCm repository and install needed packages from it. Read AMD manual here. Some ROCm (https://repo.radeon.com/rocm/zyp/) repos (use one of them):

https://repo.radeon.com/rocm/zyp/latest/
https://repo.radeon.com/rocm/zyp/zypper/
https://repo.radeon.com/rocm/zyp/5.3.2/

Quite often you need full re-installation of ROCm packages, not just simple upgrade.
ROCm might need additional Perl packages. Adding Perl repo for Leap: 

sudo zypper addrepo https://download.opensuse.org/repositories/devel:/languages:/perl/$releasever/

After installation check udev rules at "/etc/udev/rules.d/"
1. for amdgpu driver (file nn-amdgpu.rules): 

KERNEL=="kfd", GROUP=="video", MODE="0660"

2. for amdkfd driver (file nn-kfd.rules): 

SUBSYSTEM=="kfd", KERNEL=="kfd", TAG+="uaccess", GROUP="video"


ROCm - Running on unsupported hardware

You can force ROCm (HIP) to run on unsupported hardware with HSA_OVERRIDE_GFX_VERSION environment variable (link 1, link 2, link 3). For instance, for RDNA1 & RDNA2 you may use 

export HSA_OVERRIDE_GFX_VERSION=10.3.0

This tricks ROCm into treating your card as a Navi 21 card (link).
For RDNA3 cards use 

export HSA_OVERRIDE_GFX_VERSION=11.0.0

It is better to use GFX number from the same generation. For GCN use GCN substitutes, not RDNA ones. To emulate Vega (GCN5) try this one (GCN5.0): 

export HSA_OVERRIDE_GFX_VERSION=9.0.0

or (GCN5.1 - works?) 

export HSA_OVERRIDE_GFX_VERSION=9.0.6

For GCN4 (Radeon RX 400 & 500 series) try to use 

ROC_ENABLE_PRE_VEGA=1

You don't need the newest ROCm to get OpenCL working with old hardware. For instance, with RX Vega 56/64 (Vega 10, GCN5.0) you can use ROCm 4.5 (4.5.2 is the latest). ROCm 5.0 needs some changes to run on GCN5.0 hardware because it uses new instructions from GCN5.1.


ROCm - Tumbleweed and Docker with Longterm Kernel

AMDs recommendation is to run applications that use ROCm in a container. Docker containers share the kernel with their host, so the amdgpu-dkms driver is required for this usecase. This is easy on openSUSE Tumbleweed if you can use the longterm kernel. You can switch to Slowroll variant of Tumbleweed to get LTS kernel by default.

Installing kernel: 

sudo zypper install kernel-longterm kernel-longterm-devel

Reboot and select the longterm kernel to make sure it works: 

sudo systemctl reboot

Remove the default kernel, as it will cause problems compiling later: 

sudo zypper remove kernel-default

Add the latest branch of the amdgpu repository: 

sudo zypper addrepo https://repo.radeon.com/amdgpu/latest/sle/15.6/main/x86_64/ amdgpu-latest

Install amdgpu-dkms: 

sudo zypper install amdgpu-dkms amdgpu-dkms-firmware

From here, follow the documentation on AMD's site AMD's site. Mainly, you will need to add the device flags --device /dev/kfd --device /dev/dri.

You can read more about Docker on openSUSE as well.


GPGPU on openSUSE Tumbleweed

You can use ROCm and other drivers via virtualisation or Docker or Distrobox. You can switch to Slowroll variant of Tumbleweed to make some things easier.
While AMD supports HIP & OpenCL on SUSE SLE (= openSUSE Leap) only, you can use ROCm with Tumbleweed by manually choosing which packages to install. Or you can compile drivers by yourself.
User's instructions and discussion of using ROCm on Tumbleweed via Distrobox on forum are here.
Of course, you can use Mesa 3D OpenCL (Rusticl) drivers with Tumbleweed.

OpenSUSE provides some packages with GPU:ROCm repository. Add GPU:ROCm repository into your system to see all available packages.

Adding repo: 

zypper addrepo https://download.opensuse.org/repositories/science:GPU:ROCm/openSUSE_Factory/science:GPU:ROCm.repo


Installing AMD HIP runtime (https://software.opensuse.org/package/hip): 

zypper refresh
zypper install hip


Installing OpenCL driver (https://software.opensuse.org/package/amd-opencl): 

zypper refresh
zypper install amd-opencl


This will install these files: 

/usr/lib64/libamdocl64.so
/usr/share/OpenCL/vendors/amdocl64.icd
/usr/share/doc/packages/rocm-ocl-icd/LICENSE

This GPU:ROCm repo is of openSUSE-Factory kind and is incompatible with Leap. Adding this repo to openSUSE Leap will break your OS.


Non-standard paths for OpenCL icd files in openSUSE Tumbleweed

Default path for *.icd files is "/etc/OpenCL/vendors". As unneeded sophistication Tumbleweed uses path "/usr/share/OpenCL/vendors" instead of default one. Use symlinks or copy files to get standard behaviour. Read here for details: link1, link2, link3.


GPGPU on APU (iGPU)

AMD provides no official support for GPGPU on integrated GPUs for consumer products. Some ROCm versions support iGPU, but they may work much slower than expected (compared to PAL driver). Driver installer supports legacy option, but this legacy driver is ORCA driver, supports(?) GCN1 - GCN4, and has no support for GCN5 (Vega) and newer GPUs.
ROCm 6.x requires disabling supported AMD iGPU to use AMD dGPU - see previous info. Some info about settings for iGPU: link. Amdgpu driver get some enhancements with the Linux 6.10 kernel - news, patch.
PAL driver works. But AMD drops PAL driver in favour of ROCm, so no new versions are available since 20.40 driver.
Good option is in using Mesa Rusticl OpenCL driver.


Using amdgpu driver for GCN1 & GCN2 instead of radeon

GCN1 & GCN2 hardware is using drm/radeon kernel driver by default. For GCN1 & GCN2 amdgpu driver is still in experimental phase, so you need kernel parameters to use amdgpu driver instead of radeon one. It is profitable to use amdgpu, because with it you'll get Vulkan and OpenGL 4.6 support (with SPIR-V).
Use kernel parameters to get amdgpu driver instead of radeon:

for GCN1: 

radeon.si_support=0 amdgpu.si_support=1

for GCN2: 

radeon.cik_support=0 amdgpu.cik_support=1

Read here for details.
With GCN1 & GCN2 and amdgpu driver VGA output in video card might not work.


Dedicating GPU for GPGPU

With using GPU for GPGPU you can easily get 100% GPU load, so your desktop windows can stutter, web browser might hang. To avoid this behaviour dedicate one GPU (auxiliary GPU, the simplest one) for desktop usage, and use other(s) for GPGPU. Also this solution will free some video RAM in accelerator for calculations.
Main rule for dedicating GPU for GPGPU: do not attach video cables to card(s) which you select for computations. Attach video cables to auxiliary GPU.
Reverse dedication: you can use graphics card for general usage and built-in GPU for GPGPU.
To get the best performance install GPGPU card into main graphical slot (PCIe x16 slot which is the closest to CPU). Read motherboard's manual for details. Some motherboards (for instance, with AMD X370/470/570/670/etc. chipsets) can split PCIe x16 lanes for GPU to a pair of PCIe 8x lanes, so you get two PCIe x16 graphical slots on motherboard which you can configure in BIOS to use x16 + x0 lanes or x8 + x8 ones. You can use PCIe slots provided by chipset for auxiliary GPU. If your motherboard doesn't provide PCIe x16 slots in needed quantity, you can use riser to convert PCIe x1 to PCIe x16. If PCIe slot is open at the end, then you can install longer PCIe card in it.
AMD GPUs are the best for using them for video output and auxiliary tasks. Use GCN1 and newer GPUs with amdgpu driver. To use amdgpu driver with GCN1 & GCN2 chips read this instruction. But there are some restrictions:

  1. Incompatibility of AMD ROCm with iGPU from supported architectures
  2. AM4 APUs are slower than AM4 CPUs and provide limited PCIe lanes for GPU


Special Cases

Blender

Download archive from blender.org, extract, and use it as a portable install. Blender doesn’t support Terascale chips since 2.91, needs GCN1 and newer just to run. For Terascale 2 & 3 try using Blender 2.90 (albeit not supported, it might still work). Blender 2.x can use OpenCL. Download older Blender at Previous versions page.

You can use openSUSE repos: https://software.opensuse.org/package/blender. For Leap use Experimental repo.

Since Blender 4.3 acceleration with raytracing hardware via HIP RT is available for Linux (news). To get acceleration with HIP RT you need support for RT in hardware, which is available since RDNA2 architecture (PR#135179). RDNA1 and older chips have no RT blocks in hardware. Also read this part of manual.

For GCN5.1 use HIP from ROCm 5.3 - 5.7. Since Blender 4.3 AMD HIP is not available for acceleration with Cycles render engine for Vega GPUs (GCN 5.0, GCN 5.1) (commit).

You can check compatibility and speed of GPUs for Blender workloads with this site (using Cycles_X ("Cycles X") renderer). Beware: NVIDIA OptiX produces acceptable, but not 100% accurate results.

For Cycles X renderer (Blender 3.0+) AMD GPUs are the worst case:

  1. Blender 3.0+ doesn't support OpenCL, requires working AMD HIP from AMD ROCm. But AMD HIP is not available or doesn't work quite often. GPUs older than RDNA1 are not officially supported with Blender + HIP.
  2. If GPU works, then it does it much slower than rivals from Nvidia and Intel.

But if you use another renderer (EEVEE, Indigo), then you have another results.
It is possible to use ZLUDA translator to run Blender with CUDA backend (tests). With ZLUDA + CUDA + ROCm backend Blender works about 20% faster compared to HIP + ROCm, but possibly due to errors in rendering.

DaVinci Resolve

DaVinci Resolve requires unconventional OpenCL extensions cl_khr_gl_sharing and cl_khr_image2d_from_buffer, which may not be available in some drivers. Extension cl_khr_image2d_from_buffer is a part of OpenCL 2.0. ORCA OpenCL driver is OpenCL 1.2 compliant, and can happily hangs with it. Use compatible drivers (PAL, ROCm/ROCr). Read here for details: link1, link2, link3. Also you may need to install Workstation (Proprietary) (previously AMDGPU-PRO) OpenGL drivers. Extension cl_khr_gl_sharing is implemented in Mesa 3D for radeonsi. Extension cl_khr_image2d_from_buffer is implemented in Mesa 3D for mesa, llvmpipe and iris, but not for radeonsi.
Zink driver will get support for cl_khr_gl_sharing and cl_khr_image2d_from_buffer extensions with Mesa 25.1 (link1, link2).

Darktable

Darktable requires OpenCL 1.2, so you can use it with rather old hardware and drivers. Terascale chips lack some commands, so you need tuning via settings in a file to use OpenCL in Darktable - read its manual for details.

POCL

POCL = Portable Computing Language - an OpenCL implementation (package) - might not work due to wrong paths: see Bug 1182054.


Hints

  1. Old AMD drivers contain installer and all packages in it. Script creates subfolder in root containing all packages and then installs packages from it. New AMD drivers installer uses AMD repositories at https://repo.radeon.com.
  2. You need special tuning to use amdgpu and radeon drivers simultaneously when system has GCN and Terascale GPUs (link1, link2).
  3. Installation of ROCm drivers blacklists "radeon" video driver by default.
  4. GPGPU on Terascale 1 and older AMD/ATI chips is not supported (well, it is possible, but meaningless). Image support is not available for Rusticl driver on Terascale 2 & 3. Mesa 3D is dropping OpenCL support for Terascale 2 & 3 (with r600g driver).
  5. AMD officially supports GPGPU only with selected discrete video cards and server APUs. However, GPGPU might work on consumer APUs.
  6. Previously amdkfd was a separate driver. Now it is a part of amdgpu driver - news, patch.
Retrieved from "https://en.opensuse.org/index.php?title=SDB:AMD_GPGPU&oldid=194512"