openSUSE JeOS image testing takes a part of the baseline distribution testing as the images are usually built along with the installable mediums.
Automated monitoring of the specified JeOS build projects allows us to fetch images as soon as they are accessible in The Open Build Service and initiate execution of predefined test suites for each image flavour.
For instance, here you may see predefined assets of our interest to be pulled and tested in openQA from openSUSE:Factory project
As of today, we can make use of QEMU hypervisor workers only. QEMU VMs are being spawned with CPU model qemu64, bootable either in EFI or bios_boot, using features of QEMU's user-network and create memory area size of 1GB or 2GB.
Testing scope can be defined among several areas:
- filesystems and utilities
- server applications
- pre-selected yast2 modules
- user customization
- image properties (size, machine id etc.)
The test status can be seen on the OpenQA UI directly (green/yellow/red) to have a quick overview of what the result is. When a sub-test has failed, the suite is marked as failed and the QA engineers take a look to identify the issue and create a bug if necessary. The logs of each test are uploaded in OpenQA for further troubleshooting.
Here you can see the list of tests that are executed for Leap 15.2:
If you open one of them (i.e. containers) you can see all the details and output of all the tests.
A set of fixed variables are defined for each job and some other variables can be defined for each flavor which will make the tests behave different according to the needs.
For instance, all JeOS jobs will set BOOT_HDD_IMAGE=1 and will boot from an existing image defined in HDD_1 variable. Those variables can be seen in the Settings tab and also in vars.json which can be opened in the Logs&Assets tab.
Main difference between openSUSE and SLE image testing comes from the nature of both environments.
SLE is basically a collection of modules containing sets of enterprise packages. Most of the modules become available after successful product registration. Usually we store product registration code in openQA test variable named `SCC_REGCODE`.
The rest can be registered additionally using `SUSEConnect` utility although some might required module-specific registration code.
The order of module registration matters as the modules come in sort of hierarchy.
For instance, if one would like to register SDK module, BASE and DESKTOP APPs modules have to be registered apriori. Usually this can be controlled by openQA test suite variable `SCC_ADDONS` holding a list of modules to be registered by openQA test job.
Product under development are using SCC features to test registration, however the returned list of modules and their related URLs cannot be pointing to official update servers as they do not store packages to be tested.
In such a case we use so-called fake scc (https://github.com/SUSE/scc-qa-proxy) to redirect repositories to pull test packages from openQA instead of update servers.