SDB:BTRFS
Default Subvolumes
By default, openSUSE is set up using Btrfs and snapshots for the root partition. Snapshots allow you to easily roll back your system if needed after applying updates, or to back up files. Snapshots can easily be managed with Snapper.
When using a snapshot to roll back the system, it must be ensured that data such as user's home directories, Web and FTP server contents or log files do not get lost or overwritten during a roll back. This is achieved by using Btrfs subvolumes on the root file system. Subvolumes are excluded from snapshots by default. The default root file system setup on openSUSE as proposed by YaST during the installation contains the following subvolumes. They are excluded from snapshots for the reasons given below.
/boot/grub2/i386-pc, /boot/grub2/x86_64-efi, /boot/grub2/powerpc-ieee1275, /boot/grub2/s390x-emu
A rollback of the boot loader configuration is not supported. The directories listed above are architecture-specific. The first two directories are present on AMD64/Intel 64 machines, the latter two on IBM POWER and on IBM z Systems, respectively.
/home
If /home does not reside on a separate partition, it is excluded to avoid data loss on rollbacks.
/opt
Third-party products usually get installed to /opt. It is excluded to avoid uninstalling these applications on rollbacks.
/root
The root users home directory should also be preserved during a rollback
/srv
Contains data for Web and FTP servers. It is excluded to avoid data loss on rollbacks.
/tmp
All directories containing temporary files and caches are excluded from snapshots.
/usr/local
This directory is used when manually installing software. It is excluded to avoid uninstalling these installations on rollbacks.
/var
This directory contains many variable files, including logs, temporary caches, third party products in /var/opt, and is the default location for many virtual machine images and databases. Therefore this subvolume is created to exclude all of this variable data from snapshots and is created with Copy-On-Write disabled.
Old /var/* subvolume layout (pre Jan 2018)
In older *SUSE distributions (SLE 12/Leap 42.x/and Tumbleweed installed before Jan 2018) the default btrfs subvolume layout considered /var as part of the root filesystem and instead included the following subvolumes under /var
/var/opt
Third-party products usually get installed to /opt. It is excluded to avoid uninstalling these applications on
/var/tmp, /var/cache, /var/crash
All directories containing temporary files and caches are excluded from snapshots.
/var/lib/libvirt/images
The default location for virtual machine images managed with libvirt. Excluded to ensure virtual machine images are not replaced with older versions during a rollback. By default, this subvolume is created with the option no copy on write.
/var/lib/mailman, /var/spool
Directories containing mails or mail queues are excluded to avoid a loss of mails after a rollback.
/var/lib/named
Contains zone data for the DNS server. Excluded from snapshots to ensure a name server can operate after a rollback.
/var/lib/mariadb, /var/lib/mysql, /var/lib/pgqsl
These directories contain database data. By default, these subvolumes are created with the option no copy on write.
/var/log
Log file location. Excluded from snapshots to allow log file analysis after the rollback of a broken system.
Compressed btrfs filesystems
In Leap & Tumbleweed, compression for Btrfs file systems is supported. Use the compress or compress-force option and select the compression algorithm, lzo or zlib (the default). The zlib compression has a higher compression ratio while lzo is faster and takes less CPU load. For example:
mount -o compress /dev/sdx /mnt
In case you create a file, write to it, and the compressed result is greater or equal to the uncompressed size, Btrfs will skip compression for future write operations forever for this file. If you do not like this behaviour, use the compress-force option. This can be useful for files that have some initial uncompressible data.
Note, compression takes effect for new files only. Files that were written without compression are not compressed when the file system is mounted with the compress or compress-force option. Furthermore, files with the nodatacow attribute never get their extents compressed:
chattr +C FILE mount -o nodatacow /dev/sdx /mnt
In regard to encryption, this is independent from any compression. After you have written some data to this partition, print the details:
btrfs filesystem show /mnt
btrfs filesystem show /mnt
Label: 'Test-Btrfs' uuid: 62f0c378-e93e-4aa1-9532-93c6b780749d
Total devices 1 FS bytes used 3.22MiB
devid 1 size 2.00GiB used 240.62MiB path /dev/sdb1
If you want this to be permanent, add the compress or compress-force option into the /etc/fstab configuration file. For example:
UUID=1a2b3c4d /home btrfs subvol=@/home,compress 0 0
Checking Free Space
File system usage is usually checked by running the df command. On a Btrfs file system, the output of df can be misleading, because in addition to the space the raw data allocates, a Btrfs file system also allocates and uses space for metadata.
Consequently a Btrfs file system may report being out of space even though it seems that plenty of space is still available. In that case, all space allocated for the metadata is used up. Use the following commands to check for used and available space on a Btrfs file system:
btrfs filesystem show
sudo btrfs filesystem show /
Label: 'ROOT' uuid: 52011c5e-5711-42d8-8c50-718a005ec4b3
Total devices 1 FS bytes used 10.02GiB
devid 1 size 20.02GiB used 13.78GiB path /dev/sda3
Shows the total size of the file system and its usage. If these two values in the last line match, all space on the file system has been allocated.
btrfs filesystem df
sudo btrfs filesystem df / Data, single: total=13.00GiB, used=9.61GiB System, single: total=32.00MiB, used=16.00KiB Metadata, single: total=768.00MiB, used=421.36MiB GlobalReserve, single: total=144.00MiB, used=0.00B
Shows values for allocated (total) and used space of the file system. If the values for total and used for the metadata are almost equal, all space for metadata has been allocated.
btrfs filesystem usage
sudo btrfs filesystem usage /Overall:
Device size: 20.02GiB
Device allocated: 13.78GiB
Device unallocated: 6.24GiB
Device missing: 0.00B
Used: 10.02GiB
Free (estimated): 9.63GiB (min: 9.63GiB)
Data ratio: 1.00
Metadata ratio: 1.00
Global reserve: 144.00MiB (used: 0.00B)
Data Metadata System
Id Path single single single Unallocated
-- --------- -------- --------- -------- -----------
1 /dev/sda3 13.00GiB 768.00MiB 32.00MiB 6.24GiB
-- --------- -------- --------- -------- -----------
Total 13.00GiB 768.00MiB 32.00MiB 6.24GiB
Used 9.61GiB 421.36MiB 16.00KiBShows data similar to that of the two previous commands combined.
For more information refer to man 8 btrfs-filesystem and https://btrfs.wiki.kernel.org/index.php/FAQ.
Disk Space Full Because of Snapper
If Snapper is running for the Btrfs file system, the “No space left on device” problem is typically caused by having too much data stored as snapshots on your system.
You can remove some snapshots from Snapper, however, the snapshots are not deleted immediately and might not free up as much space as you need.
To delete files from Snapper:
- Open a terminal console.
- At the command prompt, enter btrfs filesystem show, for example: user $
sudo btrfs filesystem show
Label: none uuid: 40123456-cb2c-4678-8b3d-d014d1c78c78 Total devices 1 FS bytes used 20.00GB devid 1 size 20.00GB used 20.00GB path /dev/sda3
- Enter user $This command attempts to relocate data in empty or near-empty data chunks, allowing the space to be reclaimed and reassigned to metadata. This can take a while (many hours for 1 TB) although the system is otherwise usable during this time.
sudo btrfs fi balance start </mountpoint> -dusage=5 - List the snapshots in Snapper. Enter user $
sudo snapper -c root list - Delete one or more snapshots from Snapper. Enter user $
sudo snapper -c root delete snapshot_number(s)
Ensure that you delete the oldest snapshots first. The older a snapshot is, the more disk space it occupies.
How to repair a broken/unmountable btrfs filesystem
The below are the recommended steps for any major btrfs filesystem issue, especially if its unmountable. Reading dmesg or syslog might help you identify which step you could skip to in order to fix a particular problem, but the initial steps are normally useful regardless as btrfs scrub is a very safe repair tool.
- Boot to a suitable alternative system, such as a rescue shell, different installation of openSUSE, a liveCD, or an openSUSE installation DVD. The installation DVD for the version of openSUSE you are running is usually a good choice as it will certainly use the same kernel/btrfs version. A recent Tumbleweed disk might be better as it will include newer kernel/btrfs
- Go to a suitable console and make sure you do the below as root
- Try to mount your partition to /mnt, just to confirm it's really broken
mount /dev/sda1 /mnt
- If it mounts - are you sure it's broken? if Yes - run
btrfs scrub start /mnt
to scrub the system, and
btrfs scrub status /mnt
to monitor it
- If it doesn't mount, try to scrub the device just in case it works
btrfs scrub start /dev/sda1
and
btrfs scrub status /dev/sda1
to monitor. Once complete, try mounting, if yes, you're fixed.
- If scrubbing is not an option or does not resolve the issue
then instead try mount -o usebackuproot
mount -o usebackuproot /dev/sda1 /mnt
- Run "btrfs check <device>"
btrfs check /dev/sda1
This isn't going to help, but save the log somewhere, it will be useful for the bug report.
- Seriously consider running "btrfs restore <device> <somewhereto copy data>"
btrfs restore /dev/sda1 /mnt/usbdrive
This won't fix anything but it will scan the filesystem and recover everything it can to the mounted device. This especially useful if your btrfs issues are actually caused by failing hardware and not btrfs fault.
- Run "btrfs rescue super-recover <device>"
btrfs rescue super-recover /dev/sda1
Then try to mount the device normally. If it works, stop going.
- Run "btrfs rescue zero-log <device>"
btrfs rescue zero-log /dev/sda1
Then try to mount the device normally. If it works, stop going.
- Run "btrfs rescue chunk-recover <device>"
btrfs rescue chunk-recover /dev/sda1"
This will take a LONG while. Then try to mount the device normally. If it works, stop going.
- If you didn't run it earlier, be sure to now run "btrfs restore <device> <somewhere to copy data>"
btrfs restore /dev/sda1 /mnt/usbdrive
- Failure to use btrfs restore at this point but continuing to attempt repairs means you are at a very high risk of data loss. It is advisable to use btrfs restore to recover as much data as possible before continuing.
- Now, ONLY NOW, try btrfsck aka "btrfs check --repair <device>"
btrfs check --repair /dev/sda1
How to manually re-install grub on opensuse/btrfs
The general approach to re-install grub is a 3 step procedure:
- mount the root filesystem and bind mount virtual system folders (/proc, /sys, /dev)
- chroot to the root FS mounted above
- issue the appropriate grub command to re-install it.
On openSUSE/Tumbleweed the first step is tricky since the root FS and some subfolders are subvolumes.
The purpose of this section is present you a step-by-step procedure to re-install grub, focused on the first step above, assuming a standard openSUSE disk layout.
booting
Start by booting the computer using the openSUSE Leap or Tumbleweed installation media on a usb stick.
Use the 'Rescue System' menu option to get a full root shell. On openSUSE installation media, it is under the menu More...
Identify the boot disk and partition
Once the usb boot is complete you get an interactive root shell to issue the commands needed to put the fdisk back in use.
The first thing is to identify which disk and partition you have openSUSE installed. For that you can use the lsblk -f to list the current disk drives, partitions and the filesystem type. Look for FSTYPE of btrfs.
lsblk -f NAME FSTYPE FSVER ... sda ... |-sda1 ntfs ... |-sda2 ntfs ... |-sda3 ntfs ... `-sda4 btrfs ...
Based on above, looks like the openSUSE disk and partition it is /dev/sda4.
The example above it is from a disk where another OS is installed in dual boot with openSUSE/Tumbleweed.
Your openSUSE/Tumbleweed installation may be on a different partition number or even it is installed on a different disk.Mount the disk
Next step it is to mount that partition to have access to it. Before mounting we need a folder as mount point and we will create that folder at /mnt/volume.
mkdir /mnt/volume mount /dev/sdXn /mnt/volume # use the correct disk and partition you got above.
At this point take a look at /mnt/volume
ls /mnt/volume .snapshots bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
If the output looks like the above you found the right disk/partition to mount.
If not, un-mount the FS right now with umount /mnt/volume to avoid messing with it and take another look at output of lsblk -f
- check the current subvolumes available:
The way the FS was mounted at /mnt/volume it is not appropriated to install grub. As many system folders are in fact btrfs subvolumes we need to mount those subvolumes in a appropriate way and for that we need to know what are the subvolumes names.
To list all subvolume names at /mnt/volume use the command below:
btrfs subvolume list /mnt/volume ID 256 gen 31 top level 5 path @ ID 257 gen 6421 top level 256 path @/var ID 258 gen 6065 top level 256 path @/usr/local ID 259 gen 6065 top level 256 path @/srv ID 260 gen 6417 top level 256 path @/root ID 261 gen 6135 top level 256 path @/opt ID 262 gen 6421 top level 256 path @/home ID 263 gen 6016 top level 256 path @/boot/grub2/x86_64-efi ID 264 gen 6016 top level 256 path @/boot/grub2/i386-pc ID 265 gen 6327 top level 256 path @/.snapshots ID 266 gen 6420 top level 265 path @/.snapshots/1/snapshot ... # more lines follows...
Take a note of pairs of <ID_number>/path for each relevant path in the following list:
- @
- @/var
- @/boot/grub2/x86_64-efi
- @/boot/grub2/i386-pc
- create another mount point to the root FS and mount the subvolumes
Now we will mount each of subvolumes above in a proper folder
mkdir /mnt/my-root mount /dev/sdXn -o subvolid=<ID for path @> /mnt/my-root mount /dev/sdXn -o subvolid=<ID for path @/var> /mnt/my-root/var mount /dev/sdXn -o subvolid=<ID for path @/boot/grub2/x86_64-efi> /mnt/my-root/boot/grub2/x86_64-efi mount /dev/sdXn -o subvolid=<ID for path @/boot/grub2/i386-pc> /mnt/my-root/boot/grub2/i386-pc
- bind mounts
We need the virtual folders /proc, /sys and /dev from the current running system.
mount /proc -o bind /mnt/my-root/proc mount /sys -o bind /mnt/my-root/sys mount /dev -o bind /mnt/my-root/dev
Change root and install grub
This is the last and final step. No news here.
It is no different from other FS types, like ext4 you may have read on internet.
Run chroot to the new mounted FS at /mnt/my-root and run either update-bootloader --reinit or/and grub2-install
chroot /mnt/my-root update-bootloader --reinit # or grub2-install