openSUSE:Usr merge preparation

UsrMerge in openSUSE is done, this article is kept for background. See openSUSE:UsrMerge for main article

How Fedora did it

Fedora introduced a filesystem package that contained /bin etc as symlinks to /usr. Since rpm can't replace a directory with a symlink itself, they introduced a magic rpmlib(X-CheckUnifiedSystemdir) provides tag in rpm itself. Rpm would only offer that that if /bin was a symlink. So the new filesystem package could not be installed on legacy installations.

Since replacing directories and especially four of them couldn't be done atomically the conversion itself was delegated to dracut. When manually adding a "rd.convertfs" parameter on the boot command line dracut would replace the top level directories with symlinks after mounting the root file system but before starting init.

The actual code still exists and does the following:

copy eg /usr/bin to /usr/bin.usrmove-new by using hardlinks.
copy /bin into /usr/bin.usrmove-new overwriting but backing up duplicates
delete all backed up symlinks. Probably assuming that's links pointing to the top level dirs eg /usr/bin/foo -> /bin/foo.
restore backed up binaries if the new file is a symlink. Ie assuming /bin/foo -> /usr/bin/foo where the latter is the binary.

initial situation in openSUSE

A previous attempt of the UsrMerge from 2012 was never finished. That approach moved all binaries into /usr and packaged legacy symlinks in /bin (see previous revision of this page). Therefore openSUSE has almost one hundred packages with symlinks in /bin pointing to /usr.

The previous approach did not explain how to transition from that situation to the actual directories as symlinks though.

Taking the existing dracut code to run it on openSUSE leads to the following findings:

Users have to manually trigger the conversion. That means some probably never will and then keep running into issues with the new filesystem package.

The Fedora idea was to Require: rpmlib(X-CheckUnifiedSystemdir) in the filesystem package that ships the top level dirs as symlink. To actually get that into the repodata, createrepo_c has to be patched as rpmlib() strings are normally ignored.

On a system that does not have the rpmlib(X-CheckUnifiedSystemdir) property (ie with legacy /bin), zypper behaves the following way:

       # zypper dup
       Loading repository data...
       Reading installed packages...
       Computing distribution upgrade...
 
       Problem: nothing provides rpmlib(X-CheckUnifiedSystemdir) needed by filesystem-15.5-41.1.x86_64
        Solution 1: deinstallation of filesystem-15.5-33.1.x86_64
        Solution 2: keep obsolete filesystem-15.5-33.1.x86_64
        Solution 3: break filesystem-15.5-41.1.x86_64 by ignoring some of its dependencies
 
       Choose from above solutions by number or cancel [1/2/3/c/d/?] (c):

The result can be tweaked a bit by preventing deinstallation as option:

       # echo requires:filesystem >> /etc/zypp/systemCheck
       # zypper dup
       Loading repository data...
       Reading installed packages...
       Computing distribution upgrade...
 
       Problem: This request will break your system!
         nothing provides rpmlib(X-CheckUnifiedSystemdir) needed by filesystem-15.5-41.1.x86_64
 
        Solution 1: Following actions will be done:
         ignore the warning of a broken system (requires:filesystem)
         deinstallation of filesystem-15.5-33.1.x86_64
        Solution 2: keep obsolete filesystem-15.5-33.1.x86_64
        Solution 3: break filesystem-15.5-41.1.x86_64 by ignoring some of its dependencies
 
       Choose from above solutions by number or cancel [1/2/3/c/d/?] (c):

So choosing that approach means existing systems would not be able to upgrade to a new TW snapshot that has the UsrMerge implemented.

After the conversion installed packages that had files in both top level and usr directories are broken according to rpm --verify as only one of the files can exist.

After the conversion, packages that contain files with the same name in top level directories as well as /usr would no longer be installable due to file conflicts with themselves.

   # rm -rf /tmp/r
   # mkdir -p /tmp/r/usr/bin
   # ln -s usr/bin /tmp/r/bin
   # rpm --root /tmp/r --initdb
   # rpm -Uvh --noscripts --nodeps --root /tmp/r bash-4.4-lp152.11.80.x86_64.rpm
      Preparing...                          ################################# [100%]
           file /usr/bin/sh conflicts between attempted installs of bash-4.4-lp152.11.80.x86_64 and bash-4.4-lp152.11.80.x86_64
   # echo $?
   1

The conversion algorithm has gaps. The kbd package in openSUSE for example contains this:

 /bin/psfgettable -> /usr/bin/psfgettable
 /usr/bin/psfgettable -> psfxtable

The algorithm would leave a broken /usr/bin/psfgettable symlink pointing to itself.

Preparation

Packages

The first step going forward would be to create an environment that allows to test ways to do the actual conversion. For that the compat symlinks have to get out of the way. So packages with compat symlinks in /(s)bin have to add an %if condition around the compat sections checking for %usrmerged. That allows to rebuild all affected packages in a consistent via prjconf swtich without actually changing package content in Factory right away.

Example:

       %install
       ...
       %if !0%{?usrmerged}
       mkdir -p %{buildroot}/sbin
       ln -s %{_sbindir}/%{name} %{buildroot}/sbin
       %endif
       ...
       %files
       ...
       %if !0%{?usrmerged}
       /sbin/%{name}
       %endif

Build System

openSUSE Factory is fully self hosting. Means built packages are used automatically to build other packages. Means that for example changing the filesystem package to contain the `/bin -> /usr/bin` symlink would immediately leads to breaking builds of all other packages. That's because eg bash has both /bin/bash and /usr/bin/bash, ie as outlined earlier would lead to a file conflict with itself, so bash couldn't be installed anymore. Doing it the other way around ie building bash without including /bin/bash, the build system would also break as /bin/bash would be missing then while required by other packages and the build script itself.

There are two ways to overcome this chicken and egg problem:

use file triggers to automatically create /bin symlinks
turn off the `useforbuild` setting of OBS until all packages are migrated.

Both methods have pros and cons. The symlink method takes quite some time (packages building on after another) and attention to do the right thing in the right moment. Also, the file trigger code is non-trivial (contained in compat-usrmerge package). The method works on standalone, self hosting projects.

The useforbuild method on the other hand requires less steps and is rather quick as all affected packages can be built in parallel. The method does not work for self hosted projects though, it relies on a parent project to contain all required packages for building.

The steps for methods are the following:

Symlink Method

All packages need to be prepared with %usrmerged macro
disable building of project and set rebuild=local
adjust prjconf: %usrmerged 1
enable compat-usrmerge
add compat-usermerge and compat-usrmerge-build as Support in prjconf
enable bash
add bash-legacybin to prjconf as Preinstall
explicitly disable building of filesystem, glibc and gcc10 (gcc for speed reasons, not critical)
enable building of project again, remove explicit enable from bash
add Support: !post-build-checks for filesystem as installing the package over the old one won't work here
remove disabling of filesystem
remove post-build-check disabling from prjconf
remove bash-legacybin from prjconf
enable glibc
enable gcc10 and switch to rebuild=transitive again

Useforbuild method

All packages need to be prepared with %usrmerged macro
disable building of project and set rebuild=local, disable useforbuild
adjust prjconf:

 %define usrmerged 1
 Macros:
 %usrmerged 1
 %_pamdir /usr/%{_lib}/security
 %_firmwaredir /usr/lib/firmware
 :Macros

enable compat-usrmerge [only as long as it's not in Factory already]
enable useforbuild for compat-usrmerge
add compat-usrmerge-build as Support in prjconf
enable building
trigger all affected packages
remove disabling of useforbuild
remove _pamdir and _firmwaredir defines from prjconf
rebuild failing packages
prjconf for gawk

 Support: !post-build-checks

enable useforbuild and switch to transitive for full rebuild

Overall the useforbuild method seems easier. For Factory that means

Ring0 should be build disabled to make sure there's a working bootstrap set
Factory needs to build against it's last snapshot during the conversion
Stagings build against aggregated binaries from Ring0 so should work.
Finally Ring0 could build against an aggregated version of itself, ie copy aggregates from a staging.