Home Wiki > openSUSE:Build Service Deployment
Sign up | Login

openSUSE:Build Service Deployment

tagline: From openSUSE

OBS Service Deployment

Any and all OBS services can co-exist on a single machine. As load increases a reasonable grouping of services for a scaled out solution is:

  • 'frontend' running api and webui: one or more
  • 'storage' running srcserver and mysql : only ever a single instance per OBS
  • 'backend' running rep server, scheduler, dispatcher : one or more - maybe split per architecture if they do not need each other. This is the case when ExportFilters need to be able to push binaries from one architecture to another one.
  • 'worker' running the workers : many
  • 'signer' is the signing server (isolated for security reasons) : one
  • 'downloads' runs http and acts as a frontend/rsync-mirror for rep server : one or more

frontend

The frontend service provides the interface and access control logic for the http(s) osc api service and the webui.

openSUSE packages to install:

  • obs-api
  • memcached (optionally, for performance)

Services running:

  • rclighttpd (runs obs api and webui)
  • rcobsapidelayed
  • rcobswebuidelayed

This host has usually no user data for backup.

storage

This is the service which should answer fast even under high load srcserver and mysql are critical in answer times to have fast responses to osc and webui and software search service (cf software.opensuse.org). This server contains all critical data and should receive a good backup.

openSUSE packages to install:

  • mysql
  • obs-server
  • obs-signd (optionally)
  • obs-source_service (Can be moved also to a seperate system for security reasons)

Services running:

  • rcobssrcserver
  • rcobsservice (source_service)

backend

This is where the batch processing is planned, scheduled, triggered and resolved. Built packages may be served from the build area on this host directly, via a reverse-proxy or exported by rsync.

Most data on this host should be rebuildable. Exceptions are binary imported projects. A backup of this host is recommended, but not critical.

openSUSE packages to install:

  • obs-server
  • obs-signd (optionally)
  • createrepo (optionally, for building rpm packages)
  • dpkg (optionally, for building debian packages)
  • lighttpd (optionally, for hosting repo)

Services running:

  • rcobsscheduler
  • rcobsdispatcher
  • rcobsrepserver
  • rcobspublisher
  • rcobswarden (optionally)
  • rcobssigner (optionally)

worker

This is where initial scaling will almost certainly take place. These systems need minimal local storage and data although they have a transient cache.

Virtualisation can/should be used to secure the builds as build jobs can get root and could exploit security holes.

openSUSE packages to install:

  • obs-worker
  • Xen/KVM (optionally, for worker containment)
  • qemu (optionally, for cross-building)
  • mount-static (optionally, for cross-building)
  • bash-static (optionally, for cross-building)

Services running:

  • rcobsworker

signer

If using this service you should consider the degree of security it needs.

openSUSE packages to install:

  • obs-signer

Services running:

  • rcobssigner

download

The download service is not strictly part of the OBS and is managed as any http server. It can be a reverse-proxy to the backend server or can be kept up-to-date using shared storage or rsync.

Also rsync pushing to a mirror host is possible.

openSUSE packages to install:

  • lighttpd

Data Distribution

User data lives on 'storage' and 'backend'; 'storage' is absolute critical, most information on 'backend' is rebuildable.

In terms of mount-points and storage distribtution:

  • /srv/obs/{projects,sources,requests,db,remotecache,diffcache} live on 'storage' with the srcserver
  • /srv/obs/{events,log,run} will be created on both 'storage' and 'backend'
  • /srv/obs/{info,build,workers,jobs,repos,repos_sync} will be created on 'backend'

The backend /srv/obs/build/ area is selectively mirrored to the download server by the publisher (how?) or may be accessed directly or via a reverse proxy.

Still to describe: /srv/obs/

trees/
upload/