Home Wiki > openSUSE:Build Service Concept CrossDevelopment
Sign up | Login

openSUSE:Build Service Concept CrossDevelopment

tagline: From openSUSE

Crossbuild in OBS

The cross-build in OBS feature is part of the normal OBS development. It allows you to build, test, run applications for other processor architectures using emulators. Emulators are already a normal part of OBS. Due to the requirements described below, it turned out in practise that this feature is implemented as a combination of normal development for OBS taking place inside the OBS subversion repository and additional packages. Update: Download on Demand and Cross Development are now submitted to svn trunk. No additional versions are needed to be installed, just normal OBS snapshot.

The cross-build in OBS project

The Authors of cross-build are Martin Mohring, Jan-Simon Möller and Marcus Hüwe (providing the complete download on demand feature). The concepts and its implementation were discussed and reviewed together with the OBS Team before and during Hackweek III. The download on demand feature was originally developed independently of cross-build, and can be used on its own.

Types of crossbuild

  • Type 1: Build a complete set of packages using one big Package in OBS. Examples are busybox and the uClibc buildroot system
  • Type 2: Put a complete distro including cross-tools into a Project, by modifying all single Packages to build for the cross-target. No original packages for the target can be used.
  • Type 3: In contrast to Type 2, the build system is modified so original source and binary packages can be build and used in a repository
  • Type 4: To make things even more compatible, an emulator is used in the build system to provide the original target system. Now, the orginial binaries can be even run. Also, testsuites can be run.

Also, a mixture of these types is possible by putting together differntly scoped packages in a project.

Requirements on cross-build in OBS

The aim of the project is to put support for cross-build into OBS as a generic, orthogonal feature. It should behave as much as possible as the original system that is emulated.

For that purpose, the originally used distributions in OBS that enable you to build for different targets should also be used for cross-build. New architecture types to build for should be transparent by just selecting cpu architectures like arm, mips, sh4 like the presently supported ones without much thinking. Technolgically speaking, cross-build should use already used technologies and infrastucture like virtual machines, emulators, chroos environments for the new purpose, as currently used for native building of packages, projects and even complete distributions.

Cross-build should be usable in local build with "osc build" as well as server based build using OBS and the workers. The buildenvironment should be used in an extended way to support cross-build.

As an example of this compatibility, next to the currently working openSUSE and Debian distributions for i586, x86_64 and ppc architecures, cross-build should enable OBS to seamlessly work also with the arm, mips and other versions of the same distro without much intervention. For Debian:Etch, arm, mips, powerpc, sparc and many other architectures are provied out of the box. Especially, the arm architecture should work well due to its wide use in embedded and mobile applications.

The codepath for normal (not cross-build) should behave exactly like without cross-build.

For cross-build, the download on demand feature, local binary repositories and remote OBS repositories should be usable transparently.

About QEMU

The Qemu Emulator is an Opensource project that implements emulations for complete systems as well as so called user emulations that is able to run the binaries for certain operating systems on other cpus than the host cpu.

The emulated cpus are arm, powerpc, x86, sparc and a growing list of newly added cpus, that can be found on the website of the project. QEMU can currently run on linux, bsd and darwin. The user mode of QEMU supports at enables you to run linux and bsd binaries without setting up a complete bootable system.

Signalling crossbuild in OBS

Cross-build is simply signalled as shown in the examples below under the practise section inside the project and package descriptions by using architectures other than the architecture of the build host or worker. It is planned to make this a configurable feature of the scheduler and the worker configuration.

Download on demand feature in OBS

Status of crossbuild in OBS

The cross-build of Types 1, 2 and 4 have now been successfully implemented and tested. It is also planned to implement a combination of Type 3 and Type 4 cross-build for performance reasons, when Type 4 cross-build implementation is finished. A feasibility test of this combination has already been made.

The generic code for cross-build went into the subversion repository subsequently. Versions of OBS with cross build are now part of normal OBS code (as of release 1.5 of OBS - with feature freeze on Dec/2008). Developer snapshots are provided inside build.o.o via openSUSE:Tools:{Devel,Unstable}/obs-all-svn. QEMU copies are provided under openSUSE:Tools:{Devel,Unstable}/qemu-svn.

Also, we are working on a more elegant solution to install the qemu packages in the chroot before the worker / local build starts in the emulator. And the KIWI support inside OBS should also work with cross-build repositories.

And last but not least, the code provided to download the packages bases and meta data for complete distributions on demand from Deb or Rpmmd trees will also be incorporated into cross-build to make it easier to run cross-build for preexisting distribuitons which support multiple architectures like openSUSE or Debian. Always keeping a complete copy of a distribtion can result in massive storage requirements in your binary repository server. For example, openSUSE 11.0 needs ca. 16 GB of package space for the complete distro per architecture, where about 8 GB are shared packages between all architectures and about 8 GB are architecture specific packages needed for every architecture. Installing Debian:Etch for i586, x86_64, ppc and arm could result in 40 GB of needed disk space, while already sharing the noarch packages.

Download on demand feature is merged partially and working. We have used it as a testbase for more broadly testing the quality of the qemu provided with cross-build OBS.

In order to get good figures on how well a linux distro can be "cross-buildized", a study with a 400 packages subset of openSUSE had been done for a cross-build type 2 implementation.

Cross-build has so far been tested to work properly with download on demand, remote OBS and local copied repository types.


For the roadmap of cross development support, see the OBS generic development roadmap. As of yet, cross development support has made it already into the OBS svn repository.

Some practise

This can be skipped, should Adrian provide Debian:Etch packages for Arm in the official OBS. You need to be root to install the packages. I assume you have ca. 55 GB harddisk space free, and you need to download ca. 36 GB via the internet. The example is for a openSUSE 11.0 machine with an x86_64 processor.

Installation of a cross-build OBS

Currently, should you decide to run an own OBS, the openSUSE:Tools:Devel Project, package obs-all-cross provides ready to install packages for an OBS that can run cross-build

Now, first save your old /etc/sysconfig/obs-server

# mv /etc/sysconfig/obs-server /etc/sysconfig/obs-server.old

Now, edit the new /etc/sysconfig/obs-server file. You have some possibilities. Normally, you would also run the OBS scheduler for x86_64 and i586, so you choose the line with armv4l inside:

## Path:        Applications/OBS
## Description: define for which architectures the packages should get build
## Type:        stringlist
## Default:     "i586"
## Config:      OBS
# This needs to be a space seperated list of all supported architectures

Now your OBS is ready to be started again. Start everything except the scheduler.

Updates for "qemu" for cross-build

To run cross-build, you also need to install a specifically fixed version of qemu on all worker nodes or on the local host where you intent to run local build with "osc build". These packages are provides also in the project openSUSE:Tools:Devel, package qemu-svn+gcc3.

Add a crossbuild project to your OBS

You have now 3 options to add a project to your obs for running cross-build

  • Make a project by putting a local copy of the binaries into the repository server
  • Make a project using the remote OBS feature. This is not described here, because currently there are no such projects in the official OBS
  • Make a project using the new download on demand feature. This is usually the fastests way and saves you a lot of disk space

Add a crossbuild project with a local copy to your OBS

Now depending on you local OBS setup, you either modify or add a new project with the name Debian:Etch. Once done, add an updated architecture record to it, with the new architecture armv4l added:

# osc -A api.opensuse.org meta prj Debian:Etch >de.xml

Now edit file "de.xml" with you favorite editor of you choice and add change the architecture block:

  <repository name="standard">

Now save the changed file in your local OBS. Should you use download on demand, save copies of the "Packages" files from the Deb distro under the <arch>/:full inside the binary repository.

Next copy the meta prjconf from the api.opensuse.org to you local OBS:

# osc -A api.opensuse.org meta prjconf Debian:Etch >de.conf
# osc -A <your local ip> meta prjconf -F de.conf Debian:Etch

To complete the installation, you now finally install the x86_64, i586 as a copy from the official OBS (downloads ca. 22 GB and takes a while):

# obs_mirror_project Debian:Etch standard i586
# obs_mirror_project Debian:Etch standard x86_64

After this, the appropriate directories "/srv/obs/build/Debian:Etch/standard/i586/:full" and "/srv/obs/build/Debian:Etch/standard/x86_64/:full" should have been created and filled with lots of .deb files.

Then you also need the Debian:Etch binaries for Arm, found on 3 full DVDs. Download them (another ca. 13 GB):

# wget -c -q ftp://ftp.tu-chemnitz.de/pub/linux/debian/debian-cd/4.0_r4a/arm/iso-dvd/MD5SUMS
# wget -c -q ftp://ftp.tu-chemnitz.de/pub/linux/debian/debian-cd/4.0_r4a/arm/iso-dvd/debian-40r4a-arm-DVD-1.iso
# wget -c -q ftp://ftp.tu-chemnitz.de/pub/linux/debian/debian-cd/4.0_r4a/arm/iso-dvd/debian-40r4a-arm-DVD-2.iso
# wget -c -q ftp://ftp.tu-chemnitz.de/pub/linux/debian/debian-cd/4.0_r4a/arm/iso-dvd/debian-40r4a-arm-DVD-3.iso

Then check that the MD5SUMS so the downloads were all ok. Mount the images with:

# mkdir ./mnt
# mount -o,ro -t iso9660 ./debian-40r4a-arm-DVD-1.iso ./mnt -o loop=/dev/loop3

Then copy all files with name "*.deb" inside all the 3 DVDs to "/srv/obs/build/Debian:Etch/standard/armv4l/:full". Finally, chown all correctly to the obsrun user:

# chown -R obsrun.obsrun /srv/obs/build/Debian:Etch/standard

Add a crossbuild project with download on demand to your OBS

Should you be using the download on demand feature, you do not have to get the complete packages from DVD mirror. In case of Deb meta data format (indicated by the mtype filed in the download directive, you grab a Packges file for each architecture and place it in :full directory:

# cd /srv/obs/build/Debian:Etch/standard/armv4l/:full
# wget -c -q ftp://ftp.de.debian.org/debian/dists/etch/main/binary-arm/Packages.bz2
# bunzip2 Packages.bz2

The project description for a Debian:Etch distro for arm, i586, ppc and x86_64 now contains fields as follows:

  <download baseurl="http://ftp.de.debian.org/debian" metafile="Packages" mtype="debmd" arch="armv4l" />
  <download baseurl="http://ftp.de.debian.org/debian" metafile="Packages" mtype="debmd" arch="i586" />
  <download baseurl="http://ftp.de.debian.org/debian" metafile="Packages" mtype="debmd" arch="ppc" />
  <download baseurl="http://ftp.de.debian.org/debian" metafile="Packages" mtype="debmd" arch="x86_64" />
  <repository name="standard">

Run your first cross-build

Now youre done on the server and can start the scheduler again. It should show up with a line: "armv4l: running for about ..." on the OBS monitor webpage, indicating all is done correctly on your local OBS server. The scheduler now needs a while to start up and rescan the new binaries.

Now we are ready to run the first "osc build" command for cross-build. To do that, you need a package able to build for Debian:Etch targets. I suggest you to copy e.g. openSUSE:Tools/lzma to your local OBS. Then switch off building for armv4l targets for it inside the meta prj data, and switch on a arm target for the package lzma inside:

<disable arch='armv4l'/>
<disable arch='armv4l'/>
  <repository name="Debian_Etch">
    <path repository="Debian_Etch" project="openSUSE:Tools"/>

Now, checkout the package and try your first cross-build:

$ cd <goto some local dir>
$ osc -A <local obs ip> co openSUSE:Tools/lzma
$ cd  openSUSE:Tools/lzma
$ osc -A <local obs ip> build --userootforbuild --clean Debian_Etch armv4l lzma.dsc
Building lzma.dsc for Debian_Etch/armv4l
Getting buildinfo from server
Updating cache of required packages
Writing build configuration
Getting buildconfig from server
Running build
  sudo /usr/bin/build --root=/var/tmp/oscbuild-root/martin/Debian_Etch/armv4l --rpmlist=/tmp/rpmlist.gpIne8 --dist=/tmp/buildconfig.8xD3vp lzma.dsc --clean
--changelog --arch=armv4l
logging output to /var/tmp/oscbuild-root/martin/Debian_Etch/armv4l/.build.log...
Memory limit set to 5418004KB
Using BUILD_ROOT=/var/tmp/oscbuild-root/martin/Debian_Etch/armv4l
Using BUILD_ARCH=armv4l

jupiter started "build lzma.dsc" at Mon Sep  1 01:42:01 CEST 2008.

processing specfile /tmp/obs-update/tmp/openSUSE:Tools/lzma/lzma.dsc...
running changelog2spec --target debian --file /tmp/obs-update/tmp/openSUSE:Tools/lzma/lzma.dsc

...............cut off in the middle.................

dpkg-gencontrol: warning: unknown substitution variable ${misc:Depends}
dpkg-deb: building package `lzma' in `../lzma_4.32-6_arm.deb'.
dpkg-genchanges: including full source code in upload
dpkg-buildpackage: full upload; Debian-native package (full source is included)


And cheers, you got it.

You can experiment with the resulting chroot environment inside the buildroot:

$ sudo chroot /var/tmp/oscbuild-root/martin/Debian_Etch/armv4l/
root's password:
# uname -a
Linux jupiter #1 SMP 2008-07-13 20:48:28 +0200 armv5tel GNU/Linux
# file /bin/bash
/bin/bash: ELF 32-bit LSB executable, ARM, version 1 (ARM), for GNU/Linux 2.4.1, dynamically linked (uses shared libs), for GNU/Linux 2.4.1, stripped
# ldd /bin/bash
        libncurses.so.5 => /lib/libncurses.so.5 (0x42084000)
        libdl.so.2 => /lib/libdl.so.2 (0x420cf000)
        libc.so.6 => /lib/libc.so.6 (0x420db000)
        /lib/ld-linux.so.2 (0x40081000)

We have a nice new toy now to play with! Keep happy hacking.