Home Wiki > SDB:KIWI Cookbook Live USB-Stick
Sign up | Login

SDB:KIWI Cookbook Live USB-Stick

tagline: From openSUSE

Build a Live system for a USB stick. Use some "firstboot magic" to configure the system on initial boot.
Icon-checked.png This procedure was tested on at least Kiwi version 3.25 ( kiwi --version )



Pimp your USB stick - our fourth recipe


This example shows how you can use Kiwi to create a system image that can bee installed on a USB stick. The resulting live USB stick allows you to boot the OS and use the system as if the OS were installed on the system hard drive. Using the system on the USB device will not effect any other OS installed on any of the systems other storage devices.

In order to use the USB based system a computer capable of booting from the USB stick is required. The booting capability of a computer is controlled by the BIOS. Almost all modern BIOS implementations support a feature that lets you boot of a USB stick. Should your system not boot from the stick there is a possibility that the BIOS boot order is set to look at your hard drive for an MBR (Master Boot Record) prior to looking at a USB device.

Once the OS is installed on the USB stick, the stick serves as the OS system disk and data is written to and read from the stick.

USB


Livestick recipe

Prep Time:

  • 30 min

Cook Time:

  • 10 - 12 min depending on bandwidth (see previous discussion) and hardware of host

Ingredients:

  • a running openSUSE 11.1 system
  • a openSUSE 11.1 repository
  • installed recent release of the KIWI toolset (at least version 3.25)
  • installed kiwi-doc package
  • about 1 GB free disk space


Within this example we also take a look at a few more features of KIWI

  • Use of the YaST2-Firstboot feature
  • Delete Packages you do not need or want in order to save space
  • Adding Users and Groups
  • More preferences elements
  • Additional values for the packages element
  • The config.sh file
  • The overlay concept

Basic setup for this recipe

  • Making the directories used

As a general best practice it is recommended to work in different directories for different image types.

The files we will be working with should also be copied over.

# mkdir /tmp/myusb
# cd /tmp/myusb
# cp -a /usr/share/doc/packages/kiwi/examples/suse-11.1/suse-live-stick

Use of the YaST2-Firstboot feature

The YaST firstboot feature enables you to configure the system being booted as part of the initial boot. The user is guided through a series of steps in a way reflecting a regular install. The firstboot feature is useful for image deployments where the system installation is completed, spare some final configuration steps such as the root password, user logins, or network setup.

The default work flow for the interface is as follows:

  1.  The Welcome screen
  2.  The License Agreement(s)
  3.  Date & Time
  4.  Network
  5.  Root Password
  6.  User Account
  7.  Hardware
  8.  Finish


Using the firstboot feature requires that the yast2-firstboot package is installed. The yast2-firstboot package is not part of any software pattern or any dependency chain. Thus, the package must be added to the image using the <package> element in the config.xml file as shown below:

  <package name="yast2-firstboot"/>

The firstboot work flow is controlled by the file firstboot.xml. This file is a subset of the YaST-control.xml file used to control a complete SUSE installation. The firstboot.xml control file is installed with the yast2-firstboot package and can be found in the /etc/YaST2 directory on an installed system. Within the control file the work flow and proposal configurations are used to add or remove configuration steps during the initial boot of the system. By modifying the file one can setup the firstboot procedure to match the booting system's installation requirements. In addition to the SUSE provided firstboot components custom screens can be added; providing flexibility during the post installation configuration.

For additional details about firstboot please refer to the YaST2 Firstboot documentation online or if you have the package installed on your local machine you can access the documentation using the /usr/share/doc/packages/yast2-firstboot/index.html path.

Enabling the firstboot procedure for a KIWI image is a straight forward process. Add the yast2-firstboot package to config.xml as shown above and place a file named config-yast-firstboot.xml at the top level of the configuration directory, such that config.xml and config-yast-firstboot.xml are located at the same level of the configuration directory. You may customize the config-yast-firstboot.xml file to meet your needs and KIWI will handle the details.

A good starting point for the first boot configuration file is to use the provided firstboot.xml configuration file. Install the yast2-firstboot package on your system then copy the file to your configuration directory.

cp /etc/YaST2/firstboot.xml <MY_CONFIG_DIR>/config-yast-firstboot.xml

Note: Installing the yast2-firstboot package on your system will not trigger the firstboot procedure the next time you reboot your system.

When the config-yast-firstboot.xml file exists in your configuration directory KIWI will process the file and setup your image by

1. enabling the firstboot feature
2. starting YaST in firstboot mode when booting the image

The firstboot procedure will then follow the steps configured in the config-yast-firstboot.xml file. When the firstboot process completes successfully the environment is changed such that the firstboot procedure will not be executed during a subsequent boot of the system.

The firstboot process offers an additional feature for customization. It is possible to execute scripts at the end of the firstboot configuration. Place your scripts in the /usr/share/firstboot/scripts directory and they will be executed after the init-scripts.

For example, if you have a shell script called myscript.sh that should be executed at the end of the firstboot procedure copy the script into the directory <MY_CONFIG_DIR>/root/usr/share/firstboot/scripts.

chmod 755 myscript.sh
cp myscript.sh <MY_CONFIG_DIR>/root/usr/share/firstboot/scripts

The script must be executable. The feature works for any executable file, thus you can also place compiled binaries into <MY_CONFIG_DIR>/root/usr/share/firstboot/scripts to have them executed when the system is initially booted.

Basic setup for this recipe

  • Making the directories used

As a general best practice it is recommended to work in different directories for different image types.

The files we will be working with should also be copied over.

# mkdir /tmp/myusb
# cd /tmp/myusb
# cp -a /usr/share/doc/packages/kiwi/examples/suse-11.1/suse-live-stick

Adding Users and Groups

The optional <users> element specifies a list of users to be added to the image. The mandatory group attribute of the <users> element specifies the group to which the listed users belong. If the specified group does not exist it will be created. The users list may not be empty and must contain at least one entry of type <user>. The <user> element has the attributes name, pwd and home to specify the username, encrypted password, and path to the user's home directory, respectively.

The encrypted password can be created by the kiwi −−createpassword tool and and you can then cut and paste the result into the config.xml file.

  <users group="users">
      <user pwd="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0" home="/home/tux" name="tux"/>
  </users>
  <users group="root">
      <user pwd="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0" home="/root" name="root"/>
  </users>

In this example we created a user named tux within the group users and a user root within the group root. Both have the same password.

It is also possible to add a user-id or a group-id when using the optional id attribute of the <users> or <user> elements as shown below.

 <users group="users" id="[the-id-as-number]">
   <user pwd=... home=... name=... id="[the-id-as-number]">
 ...
 </users>

A second look at preference settings

The mandatory <preferences> element is a list element containing information about the desired image type(s), the package manager, the version of this image and other optional preference elements.

Previously we noted the <type> element, used to control the image type that will be created by Kiwi. In addition to the type element the preferences list may contain a number of other elements. Some of these are described in more detail below. Below is the preferences list for our current example.

  <preferences>
     <type filesystem="squashfs" boot="usbboot/suse-11.1">usb</type>
     <type filesystem="squashfs" boot="oemboot/suse-11.1">oem</type>
     <type filesystem="squashfs" boot="vmxboot/suse-11.1">vmx</type>
     <version>1.1.2</version>
     <packagemanager>zypper</packagemanager>
     <rpm-check-signatures>false</rpm-check-signatures>
     <rpm-force>true</rpm-force>
     <rpm-excludedocs>true</rpm-excludedocs>
     <oem-swap>false</oem-swap>
     <oem-kiwi-initrd>true</oem-kiwi-initrd>
     <oem-boot-title>USB</oem-boot-title>
     <locale>en_US.UTF-8</locale>
     <keytable>us.map.gz</keytable>
     <timezone>US/Eastern</timezone>
  </preferences>

Starting with the type element which we have encountered in previous examples, it is notable that we can build 3 different types of images, usb, oem, and vmx, with this description. None of the specified types is marked as primary and therefore when using this description the desired image type must be provided on the command line.

type usb
Use this type to create a USB stick system along with the attributes filesystem and boot=”usbboot/suse-*”
type oem
Use this type to create a preload disk system along with the attributes filesystem, boot=”oemboot/suse-*”
With the optional format attribute, set to ”iso” or ”usb” KIWI will additionally create an installation image suitable for a CD/DVD or a USB stick.
The oem image will deploy the pre-load system onto the storage devices of the system being booted from the installation media. The storage devices are detected at boot time.
type vmx
Use this type to create a virtual disk system along with the attributes filesystem, boot=”vmxboot/suse-*” and the optional format attribute.
The format attribute specifies one of the qemu supported virtualization formats, for example vmdk or qcow2.
The optional attribute vga can be specified to configure the kernel framebuffer mode. Detailed information about the possible values can be found in /usr/src/linux/Documentation/fb/vesafb.txt.
The vga attribute also works for the image types usb and oem.
Additional types
In addition to the types listed above kiwi supports types of iso, xen, pxe and ec2.

For details about the types please refer to the KIWI Image System Cookbook

In this example the filessytem attribute is set to squashfs. You can choose any file system supported by Linux as a boot partition. Some special characteristics of the compressed squashfs file system are listed below.

filesystem="squashfs"
This will compress the image using the squashfs file system.
The boot process will automatically use the aufs file system as an overlay file system to mount the complete tree in read-write mode.
Kiwi creates an additional ext2 partition on the USB stick to support write mode. In order to support compression with squashfs it is required that the squashfs package is installed on the system used to create the image. This requirement also applies to aufs.


Next we will take a look at other entries within the preferences list element.

The OEM Type and OEM preference settings

By default the oemboot process will create or modify a swap, /home and / (root) partition. Kiwi supports OEM specific elements to customize the behavior of the oemboot process.

<oem-swap>true|false</oem-swap>
Specify if a swap partition should be create.
<oem-kiwi-initrd>true|false</oem-kiwi-initrd>
If this element is set to true the initial oemboot boot image (initrd) will not be replaced by the system (mkinitrd) created initrd.
This option makes sense if the target storage device for the image is not a fixed disk but for example a USB stick. In that case it might be required to re-detect the storage location on first boot which is done as part of the oemboot boot image
<oem-boot-title>text</oem-boot-title>
By default the string OEM will be appended to the boot manager menu when KIWI creates the grub configuration during first deployment. The oem-boot-title value allows to set a custom name which is used instead of OEM.

Kiwi supports additional oem specific option element, please refer to the KIWI Image System Cookbook for more details.

Some package management (rpm) specific elements

rpm-check-signatures
Specifies whether RPM should check the package signature or not
rpm-force
Specifies whether RPM should be called with –force
rpm-excludedocs
Specifies whether RPM should skip installing package documentation

Localization elements

keytable
Specifies the name of the console key-map to use. The value corresponds to a map file in /usr/share/kbd/keymaps such as us.map.gz, de.map.gz, or dvorak.map.gz. The KEYTABLE variable in /etc/sysconfig/keyboard file is set according to the keyboard mapping.
timezone
Specifies the time zone. Available time zones are located in the /usr/share/zoneinfo directory. Specify the attribute value relative to /usr/share/zoneinfo.
For example, specify Europe/Berlin for /usr/share/zoneinfo/Europe/Berlin. KIWI uses this value to configure the timezone in /etc/localtime for the image
locale
Specifies the name of the locale to use, which defines the contents of the RC_LANG system environment variable in /etc/sysconfig/language. The value corresponds to entries in /usr/share/X11/locale such as en_US.UTF-8 or ja_JP.UTF-8.

Attributes of the packages element

The packages list element contains the list of packages and patterns used to build the image. Patterns are SUSE specific and available with openSUSE 10.1 or later. A pattern is identified by a name, such as gnome or kde and identifies a number of packages associated with the named pattern. Using patterns reduces the number of packages that need to be listed explicitly.

By default, only required patterns and packages are automatically pulled into the image.

  <packages type="image" patternType="plusRecommended">
  <packages type="image" patternPackageType="plusRecommended">

Possible values for the patternType and patternPackageType are

plusRecommended
Incorporates patterns and packages that are required and recommended by the given pattern.
plusSuggested
Incorporates patterns and packages that are required and suggested by the given pattern
onlyRequired
Incorporates only patterns and packages that are required by the given pattern - this is the default

The specified list of packages (all packages listed by name and packages included through a pattern name) is used to generate a list of packages to be installed. The generated list is the solution of the dependency resolution for all specified packages. This can imply that the inclusion of a suggested package results in the inclusion of additional required and recommended packages.

Additional details may be found in the KIWI Image System Cookbook

The config.sh file

At the end of the preparation step (kiwi -p) the config.sh script is executed if it exists at the top level of the configuration directory. The script is intended to be used for configuration tasks of the operating system within the appliance.

Configuration tasks are steps such as

activating services
creating configuration files
preparing an environment for a firstboot work flow
etc.

If config.sh exits with an exit code != 0 kiwi will exit the prepare step with an error.

It is strongly discouraged to use config.sh as a means to "manually" remove packages or parts of packages. This may have a negative impact on the integrity of the image and may cause instabilities.

For additional software management it is recommended to use the images.sh script. The images.sh script is called, if it exists at the top level of the configuration directory at the beginning of the create (kiwi -c) step.

Unlike the config.sh script the purpose of the install.sh script is less defined and it is provided as a hook to work with the image being created outside the bounds of what would be considered image configuration. The images.sh script is commonly used remove packages included in the image via dependencies but that are not needed for the later use of the operating system.

Within the config.sh and images.sh scripts a set of functions are available to complete common tasks. For a detailed description of these functions please refer to the kiwi::config.sh or kiwi::images.sh manual page

# man kiwi::config.sh

or

# man kiwi::images.sh

The overlay concept

In our previous examples you may have noticed a directory named root at the top level of the configuration directory and may have wondered about its purpose especially considering the special name. The root directory in the image description represents an entry point to the root tree (/) of the image being created. Any files and directories that exist inside the root directory will be copied to the unpacked image root tree after all packages have been installed.

With the root directory overlay concept you have a very powerful customization method at your finger tips. It is very easy to apply specific configuration settings to the image by simply placing the configuration file in the correct location inside the root directory.

In this example the overlay mechanism is used for the following configuration:

./root
 `-- etc
   |-- X11
   |   `-- xorg.conf             <== the X configuration
   |-- init.d
   |   |-- earlyxdm              <== necessary start script for X
   |   |-- sax                   <== X configuration program
   |   `-- xdm                   <== start script for a displaymanager
   |-- inittab
   `-- sysconfig
       |-- displaymanager        <= settings for the displaymanager started with help of xdm
       |-- network
       |   `-- ifcfg-eth0        <= a default network configuration file
       `-- windowmanager         <= settings for the windowmanager



The overlay concept

In our previous examples you may have noticed a directory named root at the top level of the configuration directory and may have wondered about its purpose especially considering the special name. The root directory in the image description represents an entry point to the root tree (/) of the image being created. Any files and directories that exist inside the root directory will be copied to the unpacked image root tree after all packages have been installed.

With the root directory overlay concept you have a very powerful customization method at your finger tips. It is very easy to apply specific configuration settings to the image by simply placing the configuration file in the correct location inside the root directory.

In this example the overlay mechanism is used for the following configuration:

./root
 `-- etc
   |-- X11
   |   `-- xorg.conf             <== the X configuration
   |-- init.d
   |   |-- earlyxdm              <== necessary start script for X
   |   |-- sax                   <== X configuration program
   |   `-- xdm                   <== start script for a displaymanager
   |-- inittab
   `-- sysconfig
       |-- displaymanager        <= settings for the displaymanager started with help of xdm
       |-- network
       |   `-- ifcfg-eth0        <= a default network configuration file
       `-- windowmanager         <= settings for the windowmanager



The commands to build our example

# kiwi -p /tmp/myusb/suse-live-stick --root /tmp/myusb/image_root

With this we have created the unpacked image in /tmp/myusb/image_root. From our earlier examination of the configuration file we know that we can create two different images from our tree in image_root.

  • The "usb" image type
creates all required files for booting the OS but requires the use of Kiwi to deploy the image on the USB stick.
  • The "oem" image type
allows you to create a virtual disk which represents a virtual disk geometry including all partitions and boot information in one file. You can deploy this file on the USB stick using the "dd" command. Upon initial boot the virtual disk geometry will be adapted to the real disk geometry of the USB stick.

Type USB

  • Create the image
# kiwi --create /tmp/myusb/image_root -d /tmp/myusb/image --type usb
  • Plug in the stick
  • Deploy the image
# kiwi --bootstick /tmp/myusb/image/initrd-usbboot-suse-11.1.i686-2.1.1.splash.gz \ --bootstick-system /tmp/myusb/image/suse-11.1-live-stick.i686-1.1.2

Type OEM

  • Create the image
# kiwi --create /tmp/myusb/image_root -d /tmp/myusb/image --type oem
  • Plug in the stick
  • Determine the device path
# df
  • Deploy the image
dd if=/tmp/myusb/image/suse-11.1-live-stick.i686-1.1.2.raw of=</dev/yourdev> bs=32 

Caveats with respect to image name and building in 64 bit mode apply as noted previously.

You can test the USB stick using a machine capable of booting from a USB device or using a virtual machine of your choice. If you test the stick using a VM please be aware that the USB stick looks like a normal disk to the VM. This is important as the kiwi boot process searches for the USB stick in order to mount the correct storage device. However, in a virtual environment the USB stick appears as a disk and not as a USB stick. Therefore, if your virtualization solution does not provide a virtual BIOS which supports booting from a USB stick you should test the stick on real hardware.

Login Details

* User root pwd: linux
* User tux  pwd: linux