openSUSE:Build Service Concept CrossDevelopment/Preinclude
Some thoughts, experiences and proposal for obs which are especially relevant to cross-build and kiwi.
Abstract
Currently, the OBS has use cases were calling code in the worker/local build is interrupted to do special processing. The code itself does not belong to the target OS and architecture, so these packages should not be in the target repo/arch and should therefore also not be installed from the target repo/arch.
Applications
Kiwi-inside-obs
Lets start with the latest changes to the kiwi-inside-obs code. Now the worker uses build and not the hostsystem. This is great, but has some side-effects here: I use here basically Base:build, busybox and some more libs to build a mini-distro. Before the change, the workers used the hosts kiwi, which worked ok. Now i don't have the needed dependencies (kiwi, createrepo, python, sqlite, ...) in my small repository. They have to be inside the "image" repo and "image" needs also the be derived from the topmost repo (if repos were derived/stacked). As kiwi executes the imaging in an own chroot, the kiwi package and its dependencies could be taken e.g. from 11.0. This would make imaging possible also for repos which don't have kiwi itself or make using newer kiwi really easy. The imaging itself isn't affected by this.
Thinking of cross-kiwi, this is mandantory for this case, as the kiwi-chroot has to be in $HOSTARCH.
As conclusion, there's some need to make the kiwi-chroot independent of the repositories used for imaging.
Cross-build using XEN-workers
Now think of cross-build using XEN-workers: To use the qemu-user-mode inside a XEN-worker, a minimal System plus qemu in $HOSTARCH (!) is needed to start the XEN-VM. First step is then to register the qemu-binfmt-magic. All other Packages can then be $TARGETARCH, as the user-mode is active. This means: Preinstall (and Vminstall) plus qemu must be taken from $HOSTARCH and not from the repo/arch of the building package. This is the place for Martin and the worker-api ;), but its basically the same procedure as above.
build/baselibs.conf
Currently, the baselibs support is provided by special treatment inside the OBS worker or target OS packages. This special treatment could be removed and be done in the same way as qemu preprocessing or image/KWI processing is done with the new feature. So it is no longer needed to call target repo/arch code from the worker host os/arch directly.
rpmlint
Similiar to build/baselibs.conf case is the use of the rpmlint checker to check build results. I has like build/baselibs.conf an own, optional configuration file. Similiar is also that first an additional package, rpmlint, is installed. Similiar is also that after the build result, some actions are done.
Testframework
Another application similiar to the ones listed is calling a testframework, which is independent from tests performed inside a package. This could be more "static" analsysis like rpmlint (which in fact also checks compile message) or runtime tests performed with the packages (like ABI tests, consistency tests).
Common solution
How all this comes together: To get this all in one Step, I'd propose to create a mechanism for the prjconf with 2 variables - Preincludeproject and Preincluderepo. What do these do?
- Preincludeproject should set the project where the packages for the Preinstall (+Vminstall/qemu or +kiwi/qemu) is taken from.
- Preincluderepo is for kiwi imaging, as there you can select the exact repo to take the kiwi chroot packages from. For cross-build this can be omitted and the repo is taken with the same name as in the original project to provide consistency.
This has some side-effects:
- The scheduler needs to track the Packages of the Preinstall (+VM/+kiwi as needed) in the Preincludeproject/repo and not the original project.
- The worker needs to get the packages then also from Preincludeproject/repo
Benefits / use-cases
The benefits are:
- Cross-Build could be implemented for the XEN-Workers
- The kiwi-inside-obs imaging would be independent of the used repositories defined the .kiwi file (as it is if invoked manually). The "image" repo could be empty !
- Cross-Kiwi would be possible inside OBS
Conclusion
With these extension, the current cross-compile code can be taken out of the worker. All steps can be configured in the prjconf and through specialized packages of the included repository. Thus the process is transparent to the worker. For kiwi this makes the imaging independent of the used repositories inside the .kiwi-definition.