Building derived containers

Jump to: navigation, search
This article provides a guide and a specification for building container images for openSUSE distributions.

Specification

This is the specification for all container images inside openSUSE distributions. If you build images in your home project on OBS or in a devel project not intended for submission to a openSUSE distribution, they are not required to follow this specification.

Checking most rules during build is being worked on. Once that is active, any violation of the rules will be reported during build, and inside of openSUSE: projects result in a build failure.

Certain terms referring to parts of a container image reference are used throughout this document: 

                     namespace     tag
                     |-------|    |----|
registry.opensuse.org/foo/bar/baz:latest
|--------------------|-----------|
       registry       repository
|--------------------------------------|
           reference

Choice of Build Tool

The container image must be built either using a KIWI recipe or a Dockerfile (built using podman).

Each method has their own benefits and drawbacks. Kiwi supports using the package manager from the host/build system, so it can build base images and derive images which don't contain a package manager, like opensuse/busybox. With Dockerfile, it's practically required to use a full base image like opensuse/tumbleweed.

General Rules

  • An image submitted as foo-image needs to contain a matching foo.changes file
  • Mentioning repositories directly is not allowed (using obsrepositories:/ is ok)
  • When using kiwi, the package name used for submission and <image name="foo"/> attribute inside the .kiwi file have to be identical
  • The build recipe and accompanying files have to be under an OSI approved license. The full license text has to be included as a separate file.

Image Name

All images published from openSUSE:Containers subprojects on OBS are visible on registry.opensuse.org with the repositories and tags defined by those images (See Tagging). Unlike for home: and devel projects, there is no forced prefix based on the project and (build) repository names.

As every image can define its location(s) on the registry freely, it is important that there is coordination and collaboration between them. This is mainly to avoid conflicts and confusion, but also to ensure consistency, because those references are publicly visible.

Because registry.opensuse.org is shared between the officially released images and arbitrary images from devel and home projects, special care has to be taken that it's easy to tell whether a specific image is from an official source. Currently this is achieved by reserving special prefixes which don't overlap with non-official projects: opensuse/, kubic/ and kubevirt/.

While the repository and tag names can be any valid OCI identifiers, there are certain restrictions imposed by external tools:

  • docker defaults to using Docker Hub as registry (hardcoded!), where anyone can upload images. It is beneficial that the chosen repository name is not allocated on Docker Hub, otherwise that might be used by accident. The opensuse/ prefix (e.g. used by base containers) on Docker Hub is empty, thus there is no ambiguity.
  • Some registries do not support repositories at the root (registry.opensuse.org/foo:latest) or nested namespaces (registry.opensuse.org/asdf/bar/baz:latest). The latter is becoming less of an issue lately though.
  • Repository names can't also be used as namespaces, e.g. (registry.opensuse.org/some/container:latest and registry.opensuse.org/some/container/withstuff:latest), which means that for instance opensuse/tumbleweed/httpd is not possible.

Package Name

The package name (e.g. opensuse-tumbleweed-image) has to end in either -image or -container and have a relation to the main image name. For opensuse/hello-world, matching package names would be opensuse-hello-world-image or just hello-world-container.

Tags

Tags can be defined freely. On every release of a new build (Tumbleweed Snapshot, Leap Images build), the new builds are released into a subproject of openSUSE:Container and the old images are kept. If a tag is provided by multiple images, the newest image is used.

There has to be a tag which is provided by a single specific build only. This can only be achieved by including the full version (with build counter).

When using kiwi_metainfo_helper service, this could be expressed as e.g. 

 #!BuildTag: opensuse/example:1.0 opensuse/example:1.0.%RELEASE%

This tag is the one which has to be used in the org.opensuse.reference label.

Labels

One drawback of using Docker labels (which we need for compatibility) is that there is only one level of them in an image and not one per layer. This means that labels defined in the base container can be overwritten by labels in derived images. Those which aren't overwritten aren't shown as coming from the base container. To workaround that, specific information has to be cloned in container-specific labels, which are prefixed with org.opensuse.<containername>.

Here's the full list of labels/annotations defined in a openSUSE base container. Most of them need to be provided by derived images as well.

Label(s) OCI Annotation Must be provided by derived images Description Example
org.opencontainers.image.title org.opensuse.base.title org.opencontainers.image.title Yes Title of the image openSUSE Tumbleweed Base Container
org.opencontainers.image.description org.opensuse.base.description org.opencontainers.image.description Yes Short description of the image Image containing a minimal environment for containers based on openSUSE Tumbleweed.
org.opencontainers.image.version org.opensuse.base.version org.opencontainers.image.version Yes Version of the image. <version>.<cicnt>.<bldcnt> 20181224.4.2
org.opencontainers.image.created org.opensuse.base.created org.opencontainers.image.created Yes Date/time of image build 2019-01-05T14:12:30Z
org.opencontainers.image.vendor org.opensuse.base.vendor org.opencontainers.image.vendor No Vendor of the image openSUSE Project
org.opencontainers.image.url org.opensuse.base.url org.opencontainers.image.url No URL with more information https://www.opensuse.org/
org.openbuildservice.disturl org.opensuse.base.disturl org.openbuildservice.disturl Yes Dist URL as provided by OBS obs://build.opensuse.org/openSUSE:Factory/images/4985c42063a1f194292ecb777d41923b-opensuse-tumbleweed-image:docker
org.opensuse.reference org.opensuse.base.reference org.opensuse.reference Yes Reference pointing to this specific image (registry.opensuse.org/imagename[:tag]) registry.opensuse.org/opensuse/tumbleweed:20181224.4.2

History

Each image should define a history entry with a human-readable description of what it is. This way it's easy to get an overview on what each layer in an image is. 

  > docker history opensuse/tumbleweed
  IMAGE               CREATED             CREATED BY          SIZE                COMMENT
  aaa05880bad2        2 days ago          KIWI 9.17.15        123MB               openSUSE Tumbleweed 20190312 Base Container

Getting started

The easiest way to get started with a new container image build is by using the predefined templates and submitting them.

Branching a Template

On the OBS main page, there's a link to an overview of available templates. The "Application Container using Dockerfile/KIWI" and "Derived Container using KIWI" templates are usable as a base for submission to openSUSE distributions. If you click on the title of the template, you can look around inside and find out which one fits best

Depending on your choice of tool, select the appropriate template for the distribution you want to build against. Make sure to branch into a project which does not exist yet or the image will not build correctly. If the target project is already set up to build images against the distro you selected, for instance by branching into the project previously, that's fine too.

Adjusting the Build Recipe Files

Each template has a README file with instructions specific to the template inside.

Adding a .changes file

Either use the Web UI to create the foo.changes file (matching the package name) and create an entry or run osc vc foo.changes in the OSC checkout. The first changelog entry should contain a short description of what the image is.

Adding a README

This step isn't mandatory, but recommended for images with a custom entrypoint. Documentation on how to use the image and which variables it accepts belongs in here.

Testing

After enabling publishing for the package (or entire project) on OBS, the image is accessible through registry.opensuse.org. You can use docker and podman to download it, see the registry overview page.

The location is the project name on OBS with ':' substituted with '/' and the repository (KIWI: images, Dockerfile: containers) appended. e.g. https://build.opensuse.org/package/show/openSUSE:Templates:Images:Tumbleweed/dockerfile-application-container builds opensuse/example:latest and is available as 

 registry.opensuse.org/opensuse/templates/images/tumbleweed/containers/opensuse/example:latest

Submission

Container images can be submitted to openSUSE:Factory and openSUSE:Leap:15.1 and later. For Leap it's possible to introduce new images by submitting it as maintenance update.

The usual rules for submission to distros apply, so a devel project needs to be picked and used as submission base.

Useful OBS services

Multiple OBS services aid in implementing the specification by simplifying some processes like label definition and version handling.

They are added and configured in the _service files (see the templates for examples) and run various scripts before the build begins.

Dockerfile + KIWI: obs-service-kiwi_metainfo_helper

To make it possible to include buildtime information like the current build number in labels and tags, this service substitutes certain placeholders in the recipe file. It works for both .kiwi and Dockerfile recipes.

An updated list of all placeholders and their content is in the package itself, you can read it here.

KIWI only: obs-service-kiwi_label_helper

To make it easier to define labels for the "org.opensuse.<containername>" namespace, there's an OBS service that converts the OCI annotations to the final labels.

Copy of the README:

This service can be enabled to run during buildtime, when it will edit the kiwi image descriptions to expand the <suse_label_helper:add_prefix/> element.

Example: 

<image [...] xmlns:suse_label_helper="com.suse.label_helper">
[...]
       <containerconfig [...]>
         <labels>
          <suse_label_helper:add_prefix prefix="org.opensuse.base">
            <label name="org.opencontainers.image.title" value="openSUSE Tumbleweed Base Container"/>
          </suse_label_helper:add_prefix>

expands to 

<image [...] xmlns:suse_label_helper="com.suse.label_helper">
[...]
       <containerconfig [...]>
         <labels>
           <label name="org.opencontainers.image.title" value="openSUSE Tumbleweed Base Container"/>
           <label name="org.opensuse.base.title" value="openSUSE Tumbleweed Base Container"/>

Dockerfile only: obs-service-docker_label_helper

To make it easier to define labels for the "org.opensuse.<containername>" namespace, there's an OBS service which provides a PREFIXEDLABEL pseudoinstruction which expands to both the original label as well as the container-specific one with the defined labelprefix.

See the README for details.

Tips for using KIWI

Tagging

KIWI supports an additionaltags attribute and obs services, such as obs-service-kiwi_metainfo_helper, can add build-time information to tags. 

  <preferences>
    <type image="docker">
       <containerconfig
            name="opensuse/leap"
            tag="15.0.%RELEASE%"
            additionaltags="15.0"
            maintainer="Fabian Vogt &lt;fvogt@suse.com&gt;">
         <labels>

Tips for using Dockerfile

Tagging

Dockerfile recipes don't have any information about how the built image should be called, this is passed to docker/podman build manually when building an image. OBS gets that information from special comments inside the Dockerfile, such as: 

  #!BuildTag: opensuse/example:1.0 opensuse/example:latest

You can mention multiple names in the same BuildTag comment or add multiple of those comments.

Release mechanism

Containers are built in openSUSE:Factory and openSUSE:Leap:15.x:Images next to other images in the same repo. A list in the OSRT:ToTestManagerConfig attribute for each project tells the totest-manager bot which of the built images are container images and should be released to the registry instead of the download server.

Those images are released into the containers repo of the :ToTest subproject, where they are pushed to registry.opensuse.org and openQA can download and test them from there. This is done by the "image" test module, which is instantiated for multiple container runtimes (docker, podman, ...).

After openQA marks a snapshot/build as good, ttm releases the containers from :ToTest/containers into its releasetarget. For openSUSE:Factory:ToTest/containers that is openSUSE:Containers:Tumbleweed. Those target projects have kind="maintenance_release" set in their meta, so on every release, binaries are added instead of overwritten. This means that every container released (and not cleaned up) is available on OBS and the registry.

For every repo on the registry (e.g. opensuse/tumbleweed and kubic/pause), there's exactly one project/repo combination on OBS which provides the images to feed those. This is not an issue for Tumbleweed and Kubic currently, but as all versions of Leap share one repo but the images are distributed over multiple projects (openSUSE:Containers:Leap:15.x), this has to be worked around. It is achieved by creating an intermediate project at openSUSE:Containers:Leap which aggregates all binaries ending up in the opensuse/leap repo on the registry. The origin projects themselves are published disabled to make it work.

For providing the dynamic opensuse/leap:latest and opensuse/leap:15 tags, those are added to the aggregates as specified in the project config.

Retrieved from "https://en.opensuse.org/index.php?title=Building_derived_containers&oldid=170380"