Kubic:MicroOS/Design

Jump to: navigation, search

openSUSE MicroOS Design

Architecture

Installed Packages

The openSUSE Kubic installation media contains all packages, which are

  • necessary to boot the system.
  • necessary to run containers.
  • necessary to configure and run the "Container as a Service" stack.

The package list is similar to the SUSE Linux Enterprise Server minimal system.

There is no guarantee for a stable ABI: packages will be introduced if needed and removed if no longer needed. This is not considered to be a disadvantage, as the customer workload runs in a container. On the contrary the advantage is that only the minimal set of software necessary to do the requested job.

Additional RPM packages for hardware enablement, logging, monitoring and similar tasks are available on the installation media. The online repository is identical to the openSUSE Tumbleweed repository.

Job scheduler

openSUSE MicroOS is using systemd timers (see man systemd.timer) for job execution. Compared to cron systemd timers provide better control and debug options and avoid problems with cronjobs and systemd session management. cron is not installed by default, so regular cronjobs will not be executed.

Init system

openSUSE MicroOS is using systemd as it's init system. Support for legacy SysV init scripts (which includes LSB compatible init scripts) is not included by default, SysV init scripts should be converted to systemd services instead.

Filesystem

The only available and supported filesystem for the root filesystem is btrfs. Other filesystems like ext4 and xfs are available and supported for data partitions. The root filesystem is read-only, some subvolumes are available to store data, like /var, /home and /root. To store modified configuration files, overlayfs is used for /etc. The work directory for /etc/ is /var/lib/overlay/etc.

Filesystem Layout

Subvolumes

  /@/<subvolumes>            - Default subvolumes (see the list of default subvolumes on the BTRFS support page)
  -> /root                   - root user home directory
  -> /cloud-init-config      - Configuration files for cloud-init stored in the image
  -> /.snapshots/1/snapshot  - Initial installation of Base OS
  -> /.snapshots/2/snapshot  - Base OS after first update
  -> /.snapshots/3/snapshot  - Base OS after second update
  -> /.snapshots/X/snasphot  - Base OS after (X-1) updates

With the exception of the .snapshots subvolumes, the non-default subvolumes listed above are added by default in openSUSE Kubic in order to ensure it's possible to write to those locations when the rest of the root filesystem is read-only.

Important Folders

   /var/lib/docker

to store containers should be an own btrfs partition. This is so snaphots and rollback have qgroups enabled on the root filesystem. This will be a massive performance bottleneck for containers.

   /var/lib/overlayfs

is used to provide the overlay store mounted for /etc. It doesn't have to be on a separate partition.

Configuration

The system is pre-configured as far as possible during installation - usually no additional configuration is needed by the system administrator. For configuration openSUSE MicroOS is using cloud-init to adjust the system during the boot phase. Primary configuration items are network and ssh keys to allow the admin to login to the machine.
Note that by default a root password will only be set if the installation is done with YaST2. Otherwise a local login is not possible, so either an account needs to be configured or ssh keys have to be installed.

System configuration (cloud-init)

Cloud-init is a flexible and popular framework for customizing cloud instances; it is used to customize the openSUSE MicroOS installations. The cloud-init configuration was modified to support caasp roles setup (master and cluster nodes).

Some enhancements were necessary to configure repositories (e.g. the update repositories) and to be able to read the configuration from a local directory. Else an USB disk would always be necessary.

The default search order for configuration files is:

  • Local directory
  • USB flash drive or ISO image
  • Configuration server (No advanced Network configuration possible):
    • NoCloud
    • OpenStack

There is some documentation how to setup cloud-init.

Health Check

Several checks for errors are done during boot phase. If an error was detected, the following rules will be used:

  • Error with new snapshot:
    • Rollback to last known working snapshot if one exist
  • Error with already successfully booted snapshot
    • Try first reboot
    • Shut down services, inform the system administrator

This process needs access to the hard disk. If the boot process fails in or before initrd, the system administrator has to fix this manually.

Security and Immutability

  • Apparmor
    • Fully supported
  • SELinux
    • Under evaluation, the framework is there, but a policy is missing
  • IMA & EVM
    • "Secureboot down to the filesystem"
    • All files are signed cryptically or with hashes
    • Implementation is work-in-progress for SUSE Linux Enterprise Server, afterwards we will evaluate whether this will work with openSUSE MicroOS, too
  • tallow
    • Tallow is a service that uses systemd's native journal API to scan for attempted ssh logins, and issues temporary IP bans for clients that violate certain login patterns.