The wikis are now using the new authentication system.
If you did not migrate your account yet, visit https://idp-portal-info.suse.com/

openSUSE:usr merge

Jump to: navigation, search

Modern Linux operating systems no longer stick to the historic

and in may ways arbitrary separation of eg /bin vs /usr/bin. Instead all vendor supplied operating system components are nowadays unified below the /usr tree. Arguments and refences to background can be found at freedesktop.org

In order to stay compatible and make all files available both via eg /bin and /usr/bin, the goal is to replace the top level directories /bin, /sbin, /lib and depending on architecure /lib64 with symlinks to the directories of the same name in /usr.



How Fedora did it

Fedora introduced a filesystem package that contained /bin etc as symlinks to /usr. Since rpm can't replace a directory with a symlink itself, they introduced a magic rpmlib(X-CheckUnifiedSystemdir) provides tag in rpm itself. Rpm would only offer that that if /bin was a symlink. So the new filesystem package could not be installed on legacy installations.

Since replacing directories and especially four of them couldn't be done atomically the conversion itself was delegated to dracut. When manually adding a "rd.convertfs" parameter on the boot command line dracut would replace the top level directories with symlinks after mounting the root file system but before starting init.

The actual code still exists and does the following:

  • copy eg /usr/bin to /usr/bin.usrmove-new by using hardlinks.
  • copy /bin into /usr/bin.usrmove-new overwriting but backing up duplicates
  • delete all backed up symlinks. Probably assuming that's links pointing to the top level dirs eg /usr/bin/foo -> /bin/foo.
  • restore backed up binaries if the new file is a symlink. Ie assuming /bin/foo -> /usr/bin/foo where the latter is the binary.

Current state in openSUSE

A previous attempt of the UsrMerge from 2012 was never finished. That approach moved all binaries into /usr and packaged legacy symlinks in /bin (see previous revision of this page). Therefore openSUSE has almost one hundred packages with symlinks in /bin pointing to /usr.

The previous approach did not explain how to transition from that situation to the actual directories as symlinks though.

Taking the existing dracut code to run it on openSUSE leads to the following findings:

  • Users have to manually trigger the conversion. That means some probably never will and then keep running into issues with the new filesystem package.

    The Fedora idea was to Require: rpmlib(X-CheckUnifiedSystemdir) in the filesystem package that ships the top level dirs as symlink. To actually get that into the repodata, createrepo_c has to be patched as rpmlib() strings are normally ignored.

    On a system that does not have the rpmlib(X-CheckUnifiedSystemdir) property (ie with legacy /bin), zypper behaves the following way:
       # zypper dup
       Loading repository data...
       Reading installed packages...
       Computing distribution upgrade...
 
       Problem: nothing provides rpmlib(X-CheckUnifiedSystemdir) needed by filesystem-15.5-41.1.x86_64
        Solution 1: deinstallation of filesystem-15.5-33.1.x86_64
        Solution 2: keep obsolete filesystem-15.5-33.1.x86_64
        Solution 3: break filesystem-15.5-41.1.x86_64 by ignoring some of its dependencies
 
       Choose from above solutions by number or cancel [1/2/3/c/d/?] (c): 

The result can be tweaked a bit by preventing deinstallation as option:

       # echo requires:filesystem >> /etc/zypp/systemCheck
       # zypper dup
       Loading repository data...
       Reading installed packages...
       Computing distribution upgrade...
 
       Problem: This request will break your system!
         nothing provides rpmlib(X-CheckUnifiedSystemdir) needed by filesystem-15.5-41.1.x86_64
 
        Solution 1: Following actions will be done:
         ignore the warning of a broken system (requires:filesystem)
         deinstallation of filesystem-15.5-33.1.x86_64
        Solution 2: keep obsolete filesystem-15.5-33.1.x86_64
        Solution 3: break filesystem-15.5-41.1.x86_64 by ignoring some of its dependencies
 
       Choose from above solutions by number or cancel [1/2/3/c/d/?] (c):

So choosing that approach means existing systems would not be able to upgrade to a new TW snapshot that has the UsrMerge implemented.

  • After the conversion installed packages that had files in both top level and usr directories are broken according to rpm --verify as only one of the files can exist.
  • After the conversion, packages that contain files with the same name in top level directories as well as /usr would no longer be installable due to file conflicts with themselves.
   # rm -rf /tmp/r
   # mkdir -p /tmp/r/usr/bin
   # ln -s usr/bin /tmp/r/bin
   # rpm --root /tmp/r --initdb
   # rpm -Uvh --noscripts --nodeps --root /tmp/r bash-4.4-lp152.11.80.x86_64.rpm
      Preparing...                          ################################# [100%]
           file /usr/bin/sh conflicts between attempted installs of bash-4.4-lp152.11.80.x86_64 and bash-4.4-lp152.11.80.x86_64
   # echo $?
   1
  • The conversion algorithm has gaps. The kbd package in openSUSE for example contains this:
 /bin/psfgettable -> /usr/bin/psfgettable
 /usr/bin/psfgettable -> psfxtable

The algorithm would leave a broken /usr/bin/psfgettable symlink pointing to itself.

Preparation

The first step going forward would be to create an environment that allows to test ways to do the actual conversion. For that the compat symlinks have to get out of the way. So packages with compat symlinks in /(s)bin have to add an %if condition around the compat sections checking for %usrmerged. That allows to rebuild all affected packages in a consistent via prjconf swtich without actually changing package content in Factory right away.

Example:

       %install
       ...
       %if !0%{?usrmerged}
       mkdir -p %{buildroot}/sbin
       ln -s %{_sbindir}/%{name} %{buildroot}/sbin
       %endif
       ...
       %files
       ...
       %if !0%{?usrmerged}
       /sbin/%{name}
       %endif

Issues to solve

update-alternatives

update-alternatives cannot handle switching to the /usr version as master

   # /usr/sbin/update-alternatives --install /bin/foo foo /usr/bin/true 15 --slave /usr/bin/foo usr-bin-foo /usr/bin/true
   update-alternatives: using /usr/bin/true to provide /bin/foo (foo) in auto mode

Declaring the usr version as master does not work:

   # /usr/sbin/update-alternatives --install /usr/bin/foo usr-bin-foo /usr/bin/true 15
   update-alternatives: error: alternative usr-bin-foo can't be master: it is a slave of foo

Moving the /bin variant to /usr leads to missing symlinks:

   # /usr/sbin/update-alternatives --install /usr/bin/foo foo /usr/bin/true 15
   update-alternatives: renaming foo link from /bin/foo to /usr/bin/foo
   update-alternatives: warning: forcing reinstallation of alternative /usr/bin/true because link group foo is broken
   # l {/usr,}/bin/foo
   /usr/bin/ls: cannot access '/usr/bin/foo': No such file or directory
   /usr/bin/ls: cannot access '/bin/foo': No such file or directory
   # l /etc/alternatives/foo
   lrwxrwxrwx 1 root root 13 Nov 30 16:29 /etc/alternatives/foo -> /usr/bin/true*


Can be fixed by calling --auto once:

   # update-alternatives --auto foo
   update-alternatives: warning: forcing reinstallation of alternative /usr/bin/true because link group foo is broken
   # l {/usr,}/bin/foo
   /usr/bin/ls: cannot access '/bin/foo': No such file or directory
   lrwxrwxrwx 1 root root 21 Nov 30 16:30 /usr/bin/foo -> /etc/alternatives/foo*
   # update-alternatives --auto foo
   #

Conversion

How to perform the actual conversion is not fully clear yet. Unlike Fedora the goal would be to not require the user to trigger the conversion manually. That means either a mechanism is needed that determines whether a conversion is possible yet and then do it (either live or via dracut on next reboot), or eg installation of a new "filesystem" package simply does it. As of 2020 rpm cannot replace a directory with a symlink though.

FAQ

Do we really need to do this?

Adjusting a hundred packages is a lot of effort. Getting the conversion right is tricky with potential breakage all over the place. So maybe just keep the status quo? The current state is an inconsistent mess though. Some stuff in /bin and some in /usr/bin, links from /bin to /usr/bin and also vice versa. Libraries randomly in either /usr/lib or /lib etc. So that deserves to be cleaned up to reach some consistency again. Undoing the previous efforts however wouldn't work as existing script may already use eg /usr/bin/cp. So the way out is to finally do it.

Do we need to fix scripts to use eg /usr/bin/sh?

Fortunately not. The point of the merge is that both locations are valid. So scripts may continue to use /bin/sh. Also, ELF binaries will continue to use eg /lib/ld-linux.so.2

Can't we create the compat symlinks via brp scripts?

That would be nice as packages would automatically adopt to usrmerged or not situations at build time without the %usrmerged macro. Unfortunately brp scripts can't modify the file list so that won't help.

Communication

openSUSE Factory mailinglist


See also

External links