openSUSE:Build Service Debian builds

Jump to: navigation, search

Debian Builds

This page provides info on packaging for the Debian-based Linux distributions supported by the openSUSE Build Service, currently Debian itself and Ubuntu.

Debian packages are divided in two categories, binary and source, just like RPM. The binary packages have the .deb extension and they contain the compiled version of the application you want to distribute. The source packages are not a single file (like .src.rpm mechanism); instead, they are composed of (at least) three files:

  • The pristine tarball
  • A diff or a .debian.tar.gz file
  • A .dsc file
The Build Service will not create a source repository for .deb-based distributions; only the .deb file will be created and published.

Hence, it is important to keep in mind that a "standard" Debian repository is composed of at least two directories:

(your repository root)
|
+-binary
+-source

while the Build Service will create only the binary directory with the name of distribution, e.g. http://download.opensuse.org/repositories/home:/yourself/Debian_Etch/ corresponding to the binary directory instead of the repository root (which would be http://download.opensuse.org/repositories/home:/yourself/ ).

Also, keep in mind that the three parts for creating a source package are not needed in order to create a .deb package. Instead, they are necessary when you create a source repository and allow for automatically compiling and packaging on the user's machine via apt-get.

The Build Service will not create a source repository, so you simply do not need to know how they are constructed and used.

Building standard Debian packages

If you’re familiar with the standard way of creating packages used in Debian or Ubuntu, you can create (or reuse) a standard Debian source package using native Debian or Ubuntu tools. To recap, a standard Debian source package consists of three files:

  • The pristine tarball
  • Either a diff file or a debian tarball
  • A .dsc file

Such a source package can be created on Debian systems using scripts such as `dpkg-buildpackage -S` or `dpkg-source -b` as described in the Debian documentation.

Such a triplet of files can be uploaded to the Build Service as source files and the Build Service can use these files to build a package. The information in the various debian.xxx files is still there, but it is concealed in the diff file or the debian tarball.

Often, Debian source packages need to be created for reasons having nothing to do with the BuildService. If such a source package exists, there is no reason not to use it.

Alternatively, the "service" feature of "add file" can be used to specify the files to be used by URI. That way, the build service will download the three files from an existing Debian or Ubuntu source repository!

If one looks at source packages displayed by http://packages.ubuntu.com/ (Ubuntu Packages Search) one will find at the bottom of a page displaying a source package, the URLs of the three files that are the source package. These URLs can be copied to the clipboard using your browser's "copy link" function. The URLs can then be used by the buildservice's "add file" pages. (In the "Upload from remote URL" field of this page.) In this way you can cause the buildservice to rebuild one of the packages from say, the Ubuntu "Universe" repository, without ever having any form of the source on your system.

Providing different configurations for different distro versions

In order to use different configurations for multiple Debian or Ubuntu versions, you have to use the alternative method of creating the Debian package. The .dsc files should be named in a similar manner to the scheme for alternative .spec files for RPM based distros, i.e. projectName-repository.dsc , . Different .dsc files can refer to different source tarballs if necessary.

Example:

  • wonderproj-xUbuntu_10.10.dsc
  • wonderproj-new_0.0.1-1.debian.tar.gz
  • wonderproj-new_0.0.1.orig.tar.bz2

In this case, wonderproj-xUbuntu_10.10.dsc would contain references like this:

Format: 3.0 (quilt)
Source: wonderproj-new
Binary: wonderproj-new
Architecture: any
Version: 0.0.1-1
Maintainer: I.M.Coder <im@wonder-code.org>
Standards-Version: 0.0.1
Build-Depends:  cdbs, cmake, debhelper, pkg-kde-tools, kdelibs5-dev
Checksums-Sha1:
 0ba9af478421ce1b0ca652b0d035a49ed4f5513f 2893892 wonderproj-new_0.0.1.orig.tar.bz2
 f0d12c000ac09c48439408e2978eea56f152ae1c 20710 wonderproj-new_0.0.1-1.debian.tar.gz
Checksums-Sha256:
 0afd23eecf11a8d387019a007064c6554dad3adaf640c28a39a12143ac8ccc19 2893892 wonderproj-new_0.0.1.orig.tar.bz2
 f10e4a9ff5b629b2b73e8b6ca97f4d47b7ac4e53c33930db1e5fefc317f2c123 20710 wonderproj-new_0.0.1-1.debian.tar.gz
Files:
 437a8d1df895203af8b08e64717a6e30 2893892 wonderproj-new_0.0.1.orig.tar.bz2
 0323ebdd4a1983de61a5e4f326194426 20710 wonderproj-new_0.0.1-1.debian.tar.gz

Handling build dependency differences

If you need to supply extra build dependencies for some distro versions, they can be supplied in the Build-Depends line in the .dsc files.

If any build dependencies contained in debian/control (in the .debian.tar.gz file) are not applicable to some distro versions, remove them from debian/control and, for distro versions which still require them, add them to the appropriate .dsc files instead.

You may need to remove version number specifications from build dependencies in the debian/control file, if these are wrong for some distro versions.

If the debian/control file has been modified, make it available as follows:

  • Recreate the .debian.tar.gz file to include the modified debian/control file
  • Modify the .dsc files to contain the new checksums and file size of the .debian.tar.gz file. The new checksums can be obtained using the commands md5sum, sha1sum and sha256sum.

An Alternative method of creating a Debian package

Instead of the above, there is an alternative way to create a Debian package. You must create at least 5 files:

  • packageName.dsc
  • debian.changelog
  • debian.control
  • debian.rules
  • source tarball

The proper source package will be automatically generated from those files by OBS internally.

DEBTRANSFORM tags

The Build Service also uses some magic. Some extra information can be given in the .dsc file in the form of "debtransform" tags. Note that these tags are used only if there is any debian.* file in the OBS package, but not for the "Alternative way" of packaging described below.

Add DEBTRANSFORM-RELEASE: 1 to have OBS append the autoincremented build number to the version of the debian package.

  • DEBTRANSFORM-TAR
  • DEBTRANSFORM-FILES-TAR
  • DEBTRANSFORM-SERIES

These can be used to specify a tar file (.gz/.bz2/.xz) file as source that can be shared between RPM and Debian based builds, and can also be used to reuse the RPM patches in the Debian builds. You can use multiple tar files for the DEBTRANSFORM-FILES-TAR like this:

Debtransform-Files-Tar: debian.tar.gz debian-control-xUbuntu_6.06.tar.gz

The source code can be found at https://github.com/openSUSE/obs-build/blob/master/debtransform, and some more hints are in this mail: http://lists.opensuse.org/opensuse-buildservice/2007-03/msg00067.html.

Required RPM packages on the host system to build .deb packages

The following packages are needed on openSUSE based distributions to build .deb packages locally with `osc build`:

  • dpkg (for dpkg)
  • binutils (for ar)
  • devscripts (for debchange, see below, optional)

Install them with:

# zypper install dpkg binutils devscripts

Minimum set of files required to create .deb

In order to create a .deb package successfully, you have to create at least these files:

  • packageName.dsc
  • debian.changelog
  • debian.control
  • debian.rules

And of course a tarball that contains the package source code to compile.

packageName.dsc

A minimal template for this file is:

Format: 1.0
Source: packageName
Version: 5.6-3
Binary: packageName
Maintainer: FirstName LastName <email@hostname.org>
Architecture: any
Build-Depends: debhelper (>= 4.1.16), nameOfPackageNeededToBuildIt
Files: 
 d57283ebb8157ae919762c58419353c8 133282 packageName_5.6.orig.tar.gz
 2fecf324a32123b08cefc0f047bca5ee 63176 packageName_5.6-1.diff.tar.gz

In the above template, only the mandatory fields are listed, but there are many optional fields. You can find all information about these fields at http://www.debian.org/doc/debian-policy/ch-controlfields.html .

Let’s explain here a summary for let you to understand what they means:

  • Format: is the format version of the .deb package. debtransform only knows 1.0.
  • Source: is the package name in the distribution
  • Version: is composed by the version of the source and the revision of the package; in the template, 5.6 is the version of the source code, while 3 (after the dash) is the revision of the Debian package. Every time you change one of the files necessary to create a .deb, you should increment it.
  1. Binary: is the name of binary package as seen by the apt manager (e.g. to install it, you specify `apt-get install theNameYouWriteInBinaryField`)
  2. Maintainer : is the name of source maintainer (not your name, or packager name)
  3. Architecture: the list of architecture you want to compile it for
  4. Build-Depends: the library necessary to compile it. You must specify debhelper (>= 4.1.16) because it contains all helper scripts used by the system for compilation.
  5. Files: in theory, you must specify the MD5 sum and file size of the files used, but in practice, just put some values, it is not important if they are incorrect. What is important is that there are two lines with three field as shown in the template. (You can copy and paste the two lines in the template and substitute packageName with the name of the .deb you want to create and it will be ok. debtransform will do the rest.)

debian.changelog

The Debian mechanism for creating a .deb is to put a directory called "debian" with a lot of files necessary to automate the compiling and packaging process inside the source tree.

You do not need to do this by yourself, as debtransform will create the directory for you and it will place all files called debian.fileName into it.

This is valid for all .deb-based distributions, including Ubuntu, for which debian.changelog (and not ubuntu.changelog) applies likewise.

This is the minimal template:

packageName (5.6-3) stable; urgency=low

  * Initial Release

 -- YourName <youremail@hostname.de>  Mon, 25 Dec 2007 10:50:38 +0100

You could think that it is just a changelog, but the syntax is so constrained that a little error (such as missing whitespace) causes the entire process to fail. :-( Pay attention to the syntax specified at http://www.debian.org/doc/debian-policy/ch-source.html#s-dpkgchangelog To avoid the hassle with the syntax, you may use the debchange command (from the package called devscripts) to edit this file.

If you add a debian.changelog, OBS checks if the version number of the changelog matches the one found in the .dsc file. If not, it adds a fake entry containing the version number from the .dsc file. This ensures that the version of source and binary package stays in sync.

debian.control

This file is used to describe the package and its dependencies. Here is a minimal template:

Source: packageName
Section: sectionName
Priority: optional
Maintainer: yourName <yourEmail@hostname.de>
Build-Depends: debhelper (>= 9), nameOfPackageNeededToBuildIt

Package: nameOfPackage
Architecture: any
Depends: ${shlibs:Depends}
Description: single-line brief description
 Then, here there is the long description of the package.
 .
 Prepend every line with a single space. Use a single full stop
 prepended by a space to separate paragraphs.
 .
 For more information on the syntax, see the Debian Policy:
 http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Description

There is some redundant information that you can copy from .dsc file: Build-Depends; Package is the same as Binary; Architecture and Maintainer are typically the same.

debian.rules

This is a Makefile that contains all the rules for extracting, compiling, installing and packaging the source. You can change what you like inside this file, but it is simpler to change just a few lines to get it to compile. The rest of the file can be left untouched.

#!/usr/bin/make -f
# See debhelper(7) (uncomment to enable)
# output every command that modifies files on the build system.
#DH_VERBOSE = 1

# see FEATURE AREAS in dpkg-buildflags(1)
#export DEB_BUILD_MAINT_OPTIONS = hardening=+all

# see ENVIRONMENT in dpkg-buildflags(1)
# package maintainers to append CFLAGS
#export DEB_CFLAGS_MAINT_APPEND  = -Wall -pedantic
# package maintainers to append LDFLAGS
#export DEB_LDFLAGS_MAINT_APPEND = -Wl,--as-needed

%:
	dh $@

# Extracted from Debian Maintainer Guide:
#
# You may need to run dh_* commands invoked via the new dh with added arguments,
# or to run additional commands with them, or to skip them. For such cases, you
# create an override_dh_foo target with its rule in the rules file defining an
# override_dh_foo target for the dh_foo command you want to change. It basically
# says run me instead.
# 
# Please note that the dh_auto_* commands tend to do more than what has been
# discussed in this (over)simplified explanation to take care of all the corner
# cases. It is a bad idea to use override_dh_* targets to substitute simplified
# equivalent commands (except for the override_dh_auto_clean target) since it
# may bypass such smart debhelper features.
# 
# So, for instance, if you want to store system configuration data in the
# /etc/gentoo directory instead of the usual /etc directory for the recent
# gentoo package using Autotools, you can override the default --sysconfig=/etc
# argument given by the dh_auto_configure command to the ./configure command by
# the following:
#
# override_dh_auto_configure:
#	dh_auto_configure -- --sysconfig=/etc/gentoo
# 
# The arguments given after -- are appended to the default arguments of the
# auto-executed program to override them. Using the dh_auto_configure command is
# better than directly invoking the ./configure command here since it will only
# override the --sysconfig argument and retain any other, benign arguments to
# the ./configure command.
# 
# If the Makefile in the source for gentoo requires you to specify build as its
# target to build it [52], you create an override_dh_auto_build target to enable
# this.
# 
# override_dh_auto_build:
#	dh_auto_build -- build
# 
# This ensures $(MAKE) is run with all the default arguments given by the
# dh_auto_build command plus the build argument.
# 
# If the Makefile in the source for gentoo requires you to specify the
# packageclean target to clean it for the Debian package instead of using
# distclean or clean targets, you can create an override_dh_auto_clean target to
# enable it.
# 
# override_dh_auto_clean:
#	$(MAKE) packageclean
# 
# If the Makefile in the source for gentoo contains a test target which you do
# not want to run for the Debian package building process, you can use an empty
# override_dh_auto_test target to skip it.
# 
# override_dh_auto_test:
# 
# If gentoo has an unusual upstream changelog file called FIXES,
# dh_installchangelogs will not install that file by default. The
# dh_installchangelogs command requires FIXES as its argument to install it.
# 
# override_dh_installchangelogs:
#	dh_installchangelogs FIXES

Adding a patch

In order to apply a patch to your sources, you need to add a debian.series file containing the name of the patch(es) and parameters to the patch utility, e.g.

avoid_sysarg.patch -p1

Configuring sources.list

After the creation of .deb packages, you want to add the repository to the sources.list file (probably in /etc/apt/) in order to use apt-get to install your package(s).

Here is an example entry.

deb http://download.opensuse.org/repositories/home:/robermann79:/codesounding/Ubuntu_8.10  ./

The quick way is to get a mirror list from the repository itself:

  • Browse to the mirror list:
http://download.opensuse.org/repositories/<project>://<package>/<repository>/Packages?mirrorlist

So, for example:

http://download.opensuse.org/repositories/home://robermann79://codesounding/Ubuntu_8.10/Packages?mirrorlist
  • pick one of the mirrors listed, for example:
http://widehat.opensuse.org/repositories/home:/robermann79:/codesounding/Ubuntu_8.10/Packages

then remove the last "/Packages" path and add the URL to your /etc/apt/sources.list, as follows:

deb http://widehat.opensuse.org/repositories/home:/robermann79:/codesounding/Ubuntu_8.10 ./

Adding the apt-key to the system

For modern Debian GNU/Linux based distributions, you will also need to add the project's repository key to the list of keys accepted by apt.

Then you can add the downloaded key:

sudo apt-key add <downloaded-file>

Examples

You can see examples in my home project directory: https://build.opensuse.org/project/show?project=home%3AEmmeG

Common pitfalls

Note on packageName

RPM packages names do not have a strict syntax like the debian ones. So watch out before uploading your archives !

  • Package names (both source and binary, see Package, Section 5.6.7) must consist only of lower case letters (a-z), digits (0-9), plus (+) and minus (-) signs, and periods (.). They must be at least two characters long and must start with an alphanumeric character.

More informations at http://www.debian.org/doc/debian-policy/ch-controlfields.html

Note on compression formats

When the upstream tar file is in not in the format you would like to use, the recompress service will be useful to recompress it into the standard tar.gz format.

A useful example:

<services>
  <service name="download_url">
    <param name="host">master.dl.sourceforge.net</param>
    <param name="protocol">http</param>
    <param name="path">/project/someproject/somefolder/someproject.data.tar</param>
  </service>
  <service name="recompress">
    <param name="file">_service:download_url:someproject.data.tar</param>
    <param name="compression">gz</param>
  </service>
</services>

Note the _service:download_url: prefix, it's required, and it won't work without it if you're recompressing a file downloaded by a service.

In the Web UI, this means you have to edit the service's parameters accordingly after creating the entry (it won't be initially correct)

Note on "postinst" (post installation) scripts

If you need to run some script on post installation, you could:

  • upload a file named debian.postinst (see the prefix "debian"), as for debian.rules
  • put your package.postinst file (where "package" is your package name) inside the debian.tar.gz

See as example the package https://build.opensuse.org/package/show?package=openvas-scanner&project=security%3AOpenVAS%3ASTABLE%3Av4

Debian documentation

Some useful links to Debian documentation: