Archive:Build Service Deployment
|
This article about the Open Build Service is obsolete! You can find up to date information on https://openbuildservice.org/help/manuals/obs-admin-guide/ |
OBS Service Deployment
Any and all OBS services can co-exist on a single machine. As load increases a reasonable grouping of services for a scaled out solution is:
- 'frontend' running api and webui: one or more
- 'storage' running srcserver and mysql : only ever a single instance per OBS
- 'backend' running rep server, scheduler, dispatcher : one or more - maybe split per architecture if they do not need each other. This is the case when ExportFilters need to be able to push binaries from one architecture to another one.
- 'worker' running the workers : many
- 'signer' is the signing server (isolated for security reasons) : one
- 'downloads' runs http and acts as a frontend/rsync-mirror for rep server : one or more
frontend
The frontend service provides the interface and access control logic for the http(s) osc api service and the webui.
openSUSE packages to install:
- obs-api
- memcached (optionally, for performance)
Services running:
- rclighttpd (runs obs api and webui)
- rcobsapidelayed
- rcobswebuidelayed
This host has usually no user data for backup.
storage
This is the service which should answer fast even under high load srcserver and mysql are critical in answer times to have fast responses to osc and webui and software search service (cf software.opensuse.org). This server contains all critical data and should receive a good backup.
openSUSE packages to install:
- mysql
- obs-server
- obs-signd (optionally)
- obs-source_service (Can be moved also to a seperate system for security reasons)
Services running:
- rcobssrcserver
- rcobsservice (source_service)
backend
This is where the batch processing is planned, scheduled, triggered and resolved. Built packages may be served from the build area on this host directly, via a reverse-proxy or exported by rsync.
Most data on this host should be rebuildable. Exceptions are binary imported projects. A backup of this host is recommended, but not critical.
openSUSE packages to install:
- obs-server
- obs-signd (optionally)
- createrepo (optionally, for building rpm packages)
- dpkg (optionally, for building debian packages)
- lighttpd (optionally, for hosting repo)
Services running:
- rcobsscheduler
- rcobsdispatcher
- rcobsrepserver
- rcobspublisher
- rcobswarden (optionally)
- rcobssigner (optionally)
worker
This is where initial scaling will almost certainly take place. These systems need minimal local storage and data although they have a transient cache.
Virtualisation can/should be used to secure the builds as build jobs can get root and could exploit security holes.
openSUSE packages to install:
- obs-worker
- Xen/KVM (optionally, for worker containment)
- qemu (optionally, for cross-building)
- mount-static (optionally, for cross-building)
- bash-static (optionally, for cross-building)
Services running:
- rcobsworker
signer
If using this service you should consider the degree of security it needs.
openSUSE packages to install:
- obs-signer
Services running:
- rcobssigner
download
The download service is not strictly part of the OBS and is managed as any http server. It can be a reverse-proxy to the backend server or can be kept up-to-date using shared storage or rsync.
Also rsync pushing to a mirror host is possible.
openSUSE packages to install:
- lighttpd
Data Distribution
User data lives on 'storage' and 'backend'; 'storage' is absolute critical, most information on 'backend' is rebuildable.
In terms of mount-points and storage distribtution:
- /srv/obs/{projects,sources,requests,db,remotecache,diffcache} live on 'storage' with the srcserver
- /srv/obs/{events,log,run} will be created on both 'storage' and 'backend'
- /srv/obs/{info,build,workers,jobs,repos,repos_sync} will be created on 'backend'
The backend /srv/obs/build/ area is selectively mirrored to the download server by the publisher (how?) or may be accessed directly or via a reverse proxy.
Still to describe: /srv/obs/
trees/ upload/