openSUSE:Build Service Tips and Tricks

Jump to: navigation, search
Open Build Service
End user - Packager - Development - Installations - Collaboration - Comparison

This howto lists some small unsorted tips and tricks to work with the build service. Most of these tips should be included in other sides and removed from here. The intention of this site is to get a first place for gathering information that should be more documented or be fixed as soon as possible.

Most of the information are gathered from the buildservice-mailinglist (archive).

Disable build of packages

There are several ways to disable single packages to be built. One way to disable building of packages is to use the 'osc' command-line client. Just use 

osc api -X POST "/source/PROJECT/PACKAGE?cmd=set_flag&flag=build&status=disable" # and later
osc api -X POST "/source/PROJECT/PACKAGE?cmd=set_flag&flag=build&status=enable" # or better
osc api -X POST "/source/PROJECT/PACKAGE?cmd=remove_flag&flag=build"

or edit the XML with 

osc meta pkg -e <project> <package>

and add the <disable>-Tag inside <build> to the metadata of the package. You can either use: 

<!-- [...] -->
<build>
  <disable arch="<build-arch>"/>
</build>
<!-- [...] -->

or 

<!-- [...] -->
<build>
  <disable repository="<name-of-build-repository>"/>
</build>
<!-- [...] -->

or both, like: 

<!-- [...] -->
<build>
  <disable repository="SLES_9" arch="x86_64"/>
</build>
<!-- [...] -->

Another way is to use the webclient and click the appropriate button while showing the package in question.

Note: Jobs that already have been started won't get stopped. In order to remove them permanently, you need to remove them in the directory /srv/obs/jobs/ on your local obs.

Aborting currently building package

osc abortbuild <project> <package> <repo> <arch>

Expansion errors

If you get something like 

expansion error have choice for inet-daemon needed by imap: inetd xinetd

for a package, then you have two options:

  • the quick fix is to add one of the above packages (inetd or xinetd) to your BuildRequires
  • another way to fix this is to set a preference in you project config with : osc meta -e prjconf <Project> and add the line: Prefer: xinetd
  • the long term fix is to write a mail to opensuse-buildservice mailinglist and ask to add one of the files to the .dsc file of the repository, so other packages with the same problem can be fixed automatically.

Find Packages in a Project

Sometimes you would like to find out how a package is named, specifically in a base project to use it in the BuildRequires line. There is a tricky osc call to find that out.

This command: 

 osc ls -b -r standard -a i586 Fedora:20

lists all binary packages in Fedora 20. Pick the package name from the output.

See 

 osc ls --help

for more details.

Building 32bit rpms with local build only

Then you can build a 32bit package locally on a x84_64 machine, try: 

osc build <TARGET> i586

You can also create i586 binaries which can be installed from a x86_64 with the -32bit suffix. To get this work, you need to add a baselibs.conf (explaination below) to your project. Then you add the parameter --baselibs to your osc build command. 

osc build --baselibs <TARGET> i586

This should also produce -32bit packages in your RPMS/ directory locally without the help from the OBS.

Permission denied errors

If your package builds locally without any problem but you get error messages like: 

 /usr/bin/install: cannot create regular file `/usr/lib/libfoo.so.0.0`: Permission denied

when you try to build the same package in the buildservice, please remember that the buildservice currently only supports building in the chroot environment (and your package doesn't seem to use DESTDIR resp. RPM_BUILD_ROOT at the moment).

Just add the line: 

# norootforbuild

to your specfile and you'll see the same (btw: ugly) failures in your local build environment: fix these errors first before submitting to the buildservice...

_link and _aggregate

The difference between _link and _aggregate

To avoid rebuilds of packages that are just needed as build requirement for other packages or just needed because the Project wants to distribute a complete set of packages for end-users, there is the _aggregate feature in the build service.

A package that is "linked" in a new project via aggregate is not rebuilt in the new project and the binaries from the original project will be updated in the new project whenever they change.

Here are some arguments pro and contra _link or _aggregate:

Reason _link _aggregate Comment
No source change is needed.   X Aggregate is possible, but slows down your build and requires double space. Better build directly against the other repository.
I need some changes in the source for my project. X    
The original package does not build on all needed distributions or architectures. X X The best way could be to have two different pseudo packages: one that uses the _aggregate function to always get just the binary packages from the original project and one that uses the _link function to build the package for the rest. (But the easiest way is certainly to ask the original maintainer to enable the missing distributions/architectures in his project.)

Make sure to not confuse package linking with project linking which is an additional feature of the build service. It is described on a project Linking page.

Examples for links

Example of a simple _link

Create a new package in your project (the name can be the same as the original package - but this is not essential) and add a file named _link with content like this: 

<link project='openSUSE:Factory' package='tse3'/>

Linking against another OBS instance

If you're running a local OBS instance and just want to link some packages from the official Build Service or any other OBS instance with an available api server, add a _link-file like this: 

<link project='openSUSE.org:openSUSE:Factory' package='tse3'/>

This will link the package tse3 from the openSUSE:Factory project from the Build Service instance openSUSE.org into your project.

For this example, your Build Service Admin has created an empty project named openSUSE.org in your local instance containing the line: 
<remoteurl>https://api.opensuse.org/public</remoteurl>
in the project definition. No further administration work is needed.

Patches against linked packages

> Today I tried to link a package from another project to my project which 
> wasn't a problem. Then I checked out the new package with osc but I only 
> find a "_link"-file in the package dir. So how can I add a patch for 
> example or how can I modify the specfile?

Files may be added to the linked project alone. Files in the linked project replace those with the same name in the original project. This also includes the specfile. A patch-file can be used to patch files in the original project without overwriting them. The patch must then be specified in the _link file. 

<link project='openSUSE:Factory' package='tse3'>
  <patches>
    <apply name="patch" />
  </patches>
</link>

Adding patch files in linked packages

To have a new patch file applied to the source (in addition to any patch files already listed in the spec file), you could create a patch for the spec file that adds the patch to the spec file and apply using <apply>, but there is an easier way using the <add> command: 

<link project='openSUSE:Factory' package='tse3'>
  <patches>
    <add name="mybugfix.patch" />
  </patches>
</link>

This will add 'mybugfix.patch' to the spec file's list of patches to apply (this works by the build service inserting the appropriate lines into the spec file for you). The new patch will be applied after any existing patches already defined in the spec file. Of course, 'mybugfix.patch' must exist in your linked project.

The <add> command also supports the attributes 'popt="NUMBER"' to add a -pNUMBER argument to %patch, 'dir="some/dir"' to change do some/dir before applying the patch and 'after="NUMBER"' to add %patch just after line with number NUMBER.

Adding a line at the top of the spec file

To add a line at the top of the spec file (for example a %define), you don't need to patch the spec file. Instead you can use <topadd>: 

<link project="myproject" package="theotherpackage">
  <patches>
    <topadd>%define small_build 1</topadd>
  </patches>
</link>

To obtain this file checkout the package with the --unexpand-link argument, like this:

osc checkout --unexpand-link someProject somePackage

After this look for the file called _link, edit the file and add the above shown snippet to the file.

To verify that all worked successfully checkout the package in another location, and inspect top of the spec file, it should have the line between the <topadd> tags.

Source and details: [1]

Examples for aggregates

Example of an _aggregate

Create a new package in your project (the name can be the same as the original package - but this is not essential) and add a file named _aggregate with content like this. Please note that _aggregates are considered as bad style usually, because the project requires double space, the build service may postpone your package builds and the risk of incompatible breakages is rather high. Usually it is better to build against further repositories like explained below to get access to packages from other repositories..: 

<aggregatelist>
  <aggregate project="KDE:Backports">
    <package>ksimus</package>
  </aggregate>
</aggregatelist>

If the repository names don't match you can specify a mapping with the <repository> element (source is the repository name from the original package): 

<aggregatelist>
  <aggregate project="KDE:Backports">
    <package>ksimus</package>
    <repository target="openSUSE_10.2" source="SUSE_10.2" />
    <repository target="openSUSE_10.1" source="SUSE_10.1" />
  </aggregate>
</aggregatelist>

Full syntax example: 

<aggregatelist>
  <aggregate project="projectname" >
    <package>glibc</package>
    <package>libz</package>
    <binary>gcc</binary>
    <binary>gcc-c++</binary>
    <nosources/>
    <repository target="my_sle_11_repo" source="SLE_11" />
  </aggregate>
  <aggregate project="anotherproject" >
    <binary>gcc</binary>
    <repository target="my_sle_11_repo" source="SLE_11" />
  </aggregate>
  <aggregate project="lastproject" >
    <package>glibc</package>
    <repository target="my_sle_11_repo" source="SLE_11" />
  </aggregate>
</aggregatelist>

Choosing i686 package on i586

If you want a package from another architecture instead of your default one (glibc i686 on i586 for example), you need:

osc meta prjconf <yourproject> -e

and add: 

ExportFilter: \.i686\.rpm$ .

to your project config.

Afterwards, use something like 

<aggregatelist>
  <aggregate project="openSUSE:Factory">
    <package>glibc.i686</package>
  </aggregate>
</aggregatelist>

to get the i686 package on i586.

Adding multiple repositories to a project

Adding multiple repositories to a project is very handy if a package depends (Requires or BuildRequires) on more than one Build Service repositories. 

 osc meta -e prj <project name>

This will open project meta editor, add more repositories under any target using <path "projectname"/>. Please note that it is not needed to add dependent repositories. The following example adds only one repository from openSUSE:Tools project. But this one builds against the openSUSE:10.3/standard repo, so all these packages gets used as well. 

 <repository name="openSUSE_10.3">
   <path project="openSUSE:Tools" repository="openSUSE_10.3"/>
   <arch>i586</arch>
   <arch>x86_64</arch>
 </repository>

Another example: Add the NonFree and Update repositories to an openSUSE:10.2 project. The base openSUSE:10.2/standard repo is not needed, because both of them builds against it as well. Please note that adding the :Update project usually can lead to the situation that users will need to have all updates or they run into trouble, while building for the base distribution should work for users with and without updates. The only exception, where the :Update repo is needed are kernel modules (because the Linux kernel quite often becomes incompatible, while all other packages are not allowed to become incompatible by policy): 

<repository name="openSUSE_10.2">
   <path project="openSUSE:10.2:Update" repository="standard"/>
   <path project="openSUSE:10.2:NonFree" repository="standard"/>
   <arch>i586</arch>
   <arch>x86_64</arch>
</repository>

Please note that:

  • The order of the included repositories is important: The BuildService will try to use a package from the first repository that includes the package, even if the version doesn't match.
  • Dependencies will be included recursively
  • Users will have to have all the repositories configured in their systems, so that zypper can resolve the Requires: at install time. If you use uncommon projects, it is hard to guess them correctly.
  • The last path element (but only the last one!) in the list is "expanded" by the build service. This means that all repositories referenced by the project= target of this element are implicitly included in the project, and will be searched for packages. This is meant to simplify branching.

Example:
Project A has 

 <repository name="openSUSE_Factory">
   <path repository="snapshot" project="openSUSE:Factory"/>
   <arch>i586</arch>
   <arch>x86_64</arch>
</repository>

You want to build against packages from Project A in Project B:
Either use: 

 <repository name="openSUSE_Factory">
   <path repository="openSUSE_Factory" project="A"/>
   <path repository="snapshot" project="openSUSE:Factory"/>
   <arch>i586</arch>
   <arch>x86_64</arch>
</repository>

or (as openSUSE:Factory/snapshot is already included from Project A): 

 <repository name="openSUSE_Factory">
   <path repository="openSUSE_Factory" project="A"/>
   <arch>i586</arch>
   <arch>x86_64</arch>
</repository>

A real world example, adding the devel:languages:python source to your (existing) repositories: 

 <repository name="openSUSE_12.1">
   <path project="devel:languages:python" repository="openSUSE_12.1"/>
   <path project="openSUSE:12.1" repository="standard"/>
   <arch>i586</arch>
   <arch>x86_64</arch>
 </repository>
 <repository name="openSUSE_12.2">
   <path project="devel:languages:python" repository="openSUSE_12.2"/>
   <path project="openSUSE:12.2" repository="standard"/>
   <arch>i586</arch>
   <arch>x86_64</arch>
 </repository>

How is a buildenvironment defined?

It's defined /usr/lib/build/configs/$distro.conf in the build package. Here comes a description of the fields in this file:

That's about all there is to know about the build environment setup. Oh, we also do dependency expansion here, so packages needed because of package dependencies automatically get added to the "Required" list.

Building a package that needs an X-Server during build

This is the solution for "RuntimeError: Gtk couldn't be initialized" error. It's because of a bug version of python-gobject on openSUSE 12.3
> the package XXXXXX requires an X-server (or
> at least a DISPLAY) to build (without changing the configuration
> files).
>
> Can I access an X-Server in the buildservice?
> What are the required changes in my spec file?

You can run an Xvfb (start it in your specfile) to workaround this problem.

As example, you can use something like the lines below in your specfile: 

BuildRequires:  xorg-x11-server
%define         X_display         ":98"
...
%install
#############################################
### Launch a virtual framebuffer X server ###
#############################################
export DISPLAY=%{X_display}
Xvfb %{X_display} >& Xvfb.log &
trap "kill $! || true" EXIT
sleep 10
...
# start your application/testsuite here

Using different spec files for different platforms

In an ideal world, you would have a single spec file that would work on all RPM-based platforms past, present, and future. In reality, though, often a single spec file either isn't possible, or would require so many %if branches in your spec file that it would be unreadable.

Fortunately, the build service has a way in which you can use multiple spec files for individual platforms.

Imagine that you have a package named "foo" and a spec file in it called "foo.spec". Let's assume that this spec file works great for SUSE-based distributions like openSUSE, SLES, and SLED, but differences in the package layout mean that you really need a separate spec file to build for a completely different distribution like Fedora. If the repository in the build service is named "Fedora_Extras_6", you can create a spec file called "foo-Fedora_Extras_6.spec" that will be built only for that platform instead of using "foo.spec".

The only real downside to this is that it only works on an individual repository basis. There is no way to use a single spec file for multiple repositories. Revisiting the example above, if you wanted to build "foo" for Fedora Extras 4, 5 and 6, you would have to create three spec files: "foo-Fedora_Extras_4.spec", "foo-Fedora_Extras_5.spec", and "foo-Fedora_Extras_6.spec" even if each of these spec files is exactly the same.

A good example package that actually uses this is the "net-snmp-main-snapshot" package in the net-snmp project.

Removing disabled but built packages from a repository

Currently the Webfrontend doesn't support this. In osc simply type: 

osc wipebinaries <projectname>

You can also "talk" directly with the buildservice-API to solve this. If you want to remove all build rpms which are marked as disabled for a whole repository (in this example: home:foo), try the following way:

  • Create a ~/.netrc file containing your login information for build.opensuse.org or use the "-u username:password" option
  • curl -n -X POST -d '' 'https://api.opensuse.org/build/home:foo?cmd=wipe&code=disabled'
    or 
    curl -u username:password -X POST -d '' 'https://api.opensuse.org/build/home:foo?cmd=wipe&code=disabled'

For more "commands" have a look at the API documentation.

List available packages in a distro

When you don't know the exact name of the package you need for your build, you can either use the web search or osc ls -b.

Example: which package contains mysql development files on Ubuntu? 

#           <repository> <arch> <project>
$ osc ls -b -r standard -a i586 Ubuntu:7.10 | grep 'mysql.*dev'
libmysqlclient15-dev_5.0.45-1ubuntu3_i386.deb
# -> the package is called "libmysqlclient15-dev"

Enabling rpmlint checks in buildservice repositories

Please have a look at the RpmLint page and this chapter.

Building xxbit packages for other architectures (baselibs)

See openSUSE:Build Service_baselibs.conf

How to control a Release number of resulted packages

Normally BuildService handle a Release tags automatically, so the content Release: field is replaced by tuple CI_CNT.B_CNT, where CI_CNT is a number of commits and B_CNT is a number of rebuilds (when the rebuild is triggered).

This behavior is an equivalent of 

Release: <CI_CNT>.<B_CNT>

in prjconf (accessible via osc meta prjconf -e [PROJECT]).

You are able to improve it. For example for import of jpackage.org RPMs you can define in prjconf 

Release: %%{?release_prefix}.<CI_CNT>.<B_CNT>

where rpm macro %{release_prefix} is defined in a spec file. The resulting rpm will have a release number compatible with both - projects jpackage.org and openSUSE. That means you will be able to upgrade your SUSE packages to jpackage.org ones.

Or in your specfile, if your package was generate by jenkins job 123, you can directly say 

Release: <CI_CNT>.<B_CNT>.j123

The resulting rpm will have '.j123' appended to its normal build release number, so that you later can review the jenkins job that initiated the build.

See also openSUSE:Build_Service_prjconf#Release

How to fix "does not exist" when trying to delete a project

When the backend has a timeout, you can end up with an inconsistent state between the api database and the backend. To delete such a project just edit it's meta file with osc: 

osc meta prj -e <projectname>

save it, and remove it afterwards: 

osc rdelete <projectname>


Build of a SLES 11 SP1 LiveCD

Tested with kiwi-3.74-5.101.1 Install clicfs-1.1.3-3.1.x86_64.rpm and liblzma0-4.999.9beta-11.1.x86_64.rpm on the build system and copy these files to the repository directory that you configured in your /usr/share/kiwi/image/isoboot/suse-SLES11/config.xml

Modify the file /usr/share/kiwi/image/isoboot/suse-SLES11/config.xml and added the following line to <packages type="bootstrap"> section <package name="clicfs"/>

It might be a bug, it didn't work with this line (CD doesn't boot)

<packages type="bootstrap" profiles="std">

but with type=image the CD works fine:

<packages type="image" profiles="std">


  1. kiwi --createhash /usr/share/kiwi/image/isoboot/suse-SLES11

Modify the config.xml for your Live CD (not the config.xml you just edited) to use clicfs: <type primary="true" boot="isoboot/suse-SLES11" flags="clic">iso</type>

Setup a SLES 10 SP2/SP3 KIWI image with squashfs and aufs

My build host was a SLES 10 SP2 x86_64, too.

This is just a quick guide, take a look at the other KIWI documentations.

Step 1 Installing KIWI

Note: You may will find some packages on the SLES 10 SDK repository. I installed the current kiwi-3.01-93.1 from

http://download.opensuse.org/repositories/openSUSE:/Tools/SLE_10

Note: For kiwi-desc-oemboot and kiwi-desc-vmxboot you need qemu, I did not install them.

Install the smart packagemanager 

rpm -ivh /usr/share/kiwi/repo/suse-repo/suse-sle10-repo/smart-0.41-23.4.$(arch).rpm

Download squashfs for SLES 10 SP2/SP3 from http://download.opensuse.org/repositories/home:/mopp:/squashfs/

Download aufs for SLES 10 SP2/SP3 from http://download.opensuse.org/repositories/home:/mopp:/aufs

Install the squashfs package on the KIWI server 

rpm -ivh /tmp/kiwi/squashfs-3.4-35.1.x86_64.rpm

Step 2 Creating the KIWI boot image environment

Of course you need the SLES 10 SP2/3 sources, I copied them to a local directory. Change paths to the directories that fit to your system. 

cp -a aufs-kmp* squashfs-kmp* /usr/share/kiwi/repo/suse-repo/suse-sle10-repo/

Note: the aufs and squashfs module gets installed to /lib/modules/<kernel>/updates/. Make sure the kernel modules fit to the kernel you are installing in the KIWI image.

Now it is time to modify the config.xml (boot/initrd image) to find the aufs and squashfs module. Maybe you copy the directory before editing, because after updating kiwi the config.xml probably will be overwritten. 

cd /usr/share/kiwi/image/isoboot/suse-SLES10
vi config.xml

It is very important to add aufs and squashfs to the <drivers type="drivers"> section add the lines, without these modules the LiveCD will not be found. 

<file name="../updates/aufs.ko"/>
<file name="../updates/fs/squashfs/*"/>

To the <packages type="bootstrap"> section add the lines: 

<package name="squashfs-kmp-default"/>
<package name="squashfs-kmp-smp"/>
<package name="aufs-kmp-default"/>
<package name="aufs-kmp-smp"/>

Create new hashes 

kiwi --createhash /usr/share/kiwi/image/isoboot/suse-SLES10

Step 3 Prepare the System Image

Make sure you do not mix up the kernels (-default and -smp) between the boot image kernel and the system image kernel.

Prepare the system image (I used a suse-11.0 configuration, as it is a minimal configuration) 

cp -pr /usr/share/doc/packages/kiwi/examples/suse-11.0/suse-live-iso /usr/local/kiwi/suse-sle10sp2-live-iso

Edit the config.xml in /usr/local/kiwi/suse-sle10sp2-live-iso, at least you have to change the "boot" tag and the "repository"

Change 

<type primary="true" boot="isoboot/suse-11.0" flags="unified">iso</type>

to 

<type primary="true" boot="isoboot/suse-SLES10" flags="unified">iso</type>

Remove the package "bootsplash-branding-openSUSE" as it is not available on SLES 10.

You need to add the repository to find additional packages, like smart. 

       <repository type="rpm-dir">

 <source path="/usr/share/kiwi/repo/suse-repo/suse-sle10-repo/"/>

       </repository>

Add or remove your needed packages in the config.xml and customize your system.

You may add the dhcpcd package, to enable dhcp.

The opensusePattern seems not to work with SLES.

Step 4 Create the ISO Image

Make sure you have enough space available. 

kiwi --prepare /usr/local/kiwi/suse-sle10sp2-live-iso --root /tmp/myiso

Check the log file. To check the created target system you can do a 

chroot /tmp/myiso

If everything looks fine, it is time to create the iso image 

kiwi --create /tmp/myiso -d /tmp/myiso-result

Check the log file.

Try to boot the iso image

Perform a nightly build from a VCS checkout

OBS now has support for downloading directly from a VCS via a _service file. See this concept page and the OBS reference guide.

If you would like to use the _service file technique and avoid parsing/editing the .spec file and even allow for sharing of version information with debian packaging try https://github.com/boombatower/obs-git-update. Otherwise keep reading.

You need a machine with cron and osc for that.

Prepare a working directory with a VCS checkout of your code and a checkout of your package from the build service. Prepare a spec file template with a placeholder in the version field. That may look like this: 

Name:         monitord
Version:      ##TIMESTAMP##
Requires:     alsa
...

Prepare a shell script doing a local update, the preparation of the tarball, modification of the spec-file and the upload to the build service.

Here is an example script for a SVN repository: 

#!/bin/bash

# the relative path to the OBS checkout
PACK_PATH=home:cwh:monitord-nightly/monitord

# the relative path to the SVN checkout
REPO_PATH=trunk

TIMESTAMP=`date +"%Y%m%d"`

# enter SVN directory
pushd $REPO_PATH

# update to current revision
svn up

# prepare tarball
make dist

# move resulting tarball to the package directory 
cp monitord-2.0svn.tar.gz ../$PACK_PATH

popd

# copy a modifyed spec file to the package directory
perl -p -e "s/##TIMESTAMP##/$TIMESTAMP/" monitord.spec > $PACK_PATH/monitord.spec

# enter package directory
pushd $PACK_PATH

# upload changes (new tarball and spec file) to the OBS
osc commit -m "updated to current SVN snapshot $TIMESTAMP"

popd

echo "Done"

Then call this script via cron.

Example: 

0 5 * * *       cd /space/monitord-nightly && ./push_to-obs.sh
Retrieved from "https://en.opensuse.org/index.php?title=openSUSE:Build_Service_Tips_and_Tricks&oldid=142243"