openSUSE:Build Service Concept Smart Dispatcher Worker
A proposal for a smart dispatcher <-> worker handling
Smart dispatcher / smart worker
Goal: Enhance dispatching procedure and crossbuild worker capabilities.
Use cases / User story
1) Adding package for cross-build depending on worker hostarch The cross-build setup needs to add/remove packages depending on the combination of worker-host/target architecture.
Example:
- worker hostarch = i586 , target arch = armv7
- install additional cross-compilers from i586 -> armv7
- worker hostarch = armv7 , target arch = armv7
- install no additional cross-compilers as they're not needed in native builds.
2) Adding, removing or (not) installing packages for speedup The cross-build setup can speed up operation through adding hostarch binaries into the chroot and exchanging the native target binaries.
Example:
- worker hostarch = i586 , target arch = armv7
- preinstall rpm and rpm-x86, but do not (re-)install rpm again (not switching archs during build again)
- preinstall bash and bash-x86, but do not (re-)install bash again (not switching archs during build again)
- worker hostarch = armv7, target arch = armv7
- do not preinstall/install any *-x86 package as they're not needed.
3) Assigning -jN depending on worker load If the load is low, workers can give more ressources to single tasks.
4) Assigning expensive jobs to strong workers Big jobs should preferably be executed on high power machines instead of the smaller workers if available
[Anas]: AFAIK, this is already possible using powerhosts and powerjobs variables in BSConfig.pm.
5) Expansion or shrinking of workers Resize the disk images for large jobs and also reserve cpu power for these. Huge jobs (e.g. OpenOffice) need large disk space. Reserve more space by shrinking other workers on this host and accept only smaller or less jobs.
This can be:
- resizing disk for big job, accept only smaller jobs for the others
- assigning more cpu to one job and accept less other jobs
- reserving more RAM for one job and accept smaller or less jobs
6) Advertise additional features of workers for dispatcher use
- XEN/KVM/CHROOT, RAM, DISK, CPUs, "Bogomips" (speed of the host)
- Additional scripting - e.g. acceptance tests or test scripts.
7) Graceful worker shutdown
- Finish running jobs before shutting down the workers.
- Stop accepting jobs on a certain worker to allow maintenance without shutting down dispatcher.
8)
9)
10)
Brainstorming
We need:
- workers telling the dispatcher their capabilities (KVM/XEN, CPUs, RAM, DISK, target arches, cross-capabilities, scripts ? )
- dispatcher needs to assign jobs according to the worker capabilities and reserve/block slots for repartitioning
- workers need to be able to resize their resources and also change the numbers of active worker jobs
- cross-build package lists need reflect the hostarch/targetarch combination, dispatcher needs to eval combinations
- A script to run on the worker to help determine best configuration regarding instances, jobs, cache depending on the host metrics such as memory, disk space and CPU power. The script can be interactive to allow the user to answer some basic questions regarding the nature of the jobs, architecture (cross-compile) and if a VM setup is desired. By default, the best configuration to address most common scenarios is given.
1) and 2) are related to package lists, involves scheduler and possibly dispatcher.
3), 4), 5), 6), 7) are related to worker and dispatcher.