SDB:KIWI Cookbook Live USB-Stick
tagline: From openSUSE
All of KIWI edit
|This procedure was tested on at least Kiwi version 3.25 ( kiwi --version )|
Pimp your USB stick - our fourth recipe
This example shows how you can use Kiwi to create a system image that can bee installed on a USB stick. The resulting live USB stick allows you to boot the OS and use the system as if the OS were installed on the system hard drive. Using the system on the USB device will not effect any other OS installed on any of the systems other storage devices.
In order to use the USB based system a computer capable of booting from the USB stick is required. The booting capability of a computer is controlled by the BIOS. Almost all modern BIOS implementations support a feature that lets you boot of a USB stick. Should your system not boot from the stick there is a possibility that the BIOS boot order is set to look at your hard drive for an MBR (Master Boot Record) prior to looking at a USB device.
Once the OS is installed on the USB stick, the stick serves as the OS system disk and data is written to and read from the stick.
Within this example we also take a look at a few more features of KIWI
- Use of the YaST2-Firstboot feature
- Delete Packages you do not need or want in order to save space
- Adding Users and Groups
- More preferences elements
- Additional values for the packages element
- The config.sh file
- The overlay concept
Basic setup for this recipe
- Making the directories used
As a general best practice it is recommended to work in different directories for different image types.
The files we will be working with should also be copied over.
Use of the YaST2-Firstboot feature
The YaST firstboot feature enables you to configure the system being booted as part of the initial boot. The user is guided through a series of steps in a way reflecting a regular install. The firstboot feature is useful for image deployments where the system installation is completed, spare some final configuration steps such as the root password, user logins, or network setup.
The default work flow for the interface is as follows:
1. The Welcome screen 2. The License Agreement(s) 3. Date & Time 4. Network 5. Root Password 6. User Account 7. Hardware 8. Finish
Using the firstboot feature requires that the yast2-firstboot package is installed. The yast2-firstboot package is not part of any software pattern or any dependency chain. Thus, the package must be added to the image using the <package> element in the config.xml file as shown below:
The firstboot work flow is controlled by the file firstboot.xml. This file is a subset of the YaST-control.xml file used to control a complete SUSE installation. The firstboot.xml control file is installed with the yast2-firstboot package and can be found in the /etc/YaST2 directory on an installed system. Within the control file the work flow and proposal configurations are used to add or remove configuration steps during the initial boot of the system. By modifying the file one can setup the firstboot procedure to match the booting system's installation requirements. In addition to the SUSE provided firstboot components custom screens can be added; providing flexibility during the post installation configuration.
For additional details about firstboot please refer to the YaST2 Firstboot documentation online or if you have the package installed on your local machine you can access the documentation using the /usr/share/doc/packages/yast2-firstboot/index.html path.
Enabling the firstboot procedure for a KIWI image is a straight forward process. Add the yast2-firstboot package to config.xml as shown above and place a file named conﬁg-yast-ﬁrstboot.xml at the top level of the configuration directory, such that config.xml and conﬁg-yast-ﬁrstboot.xml are located at the same level of the configuration directory. You may customize the conﬁg-yast-ﬁrstboot.xml file to meet your needs and KIWI will handle the details.
A good starting point for the first boot configuration file is to use the provided firstboot.xml configuration file. Install the yast2-firstboot package on your system then copy the file to your configuration directory.
cp /etc/YaST2/firstboot.xml <MY_CONFIG_DIR>/conﬁg-yast-ﬁrstboot.xml
Note: Installing the yast2-firstboot package on your system will not trigger the firstboot procedure the next time you reboot your system.
When the conﬁg-yast-ﬁrstboot.xml file exists in your configuration directory KIWI will process the ﬁle and setup your image by
- 1. enabling the ﬁrstboot feature
- 2. starting YaST in firstboot mode when booting the image
The firstboot procedure will then follow the steps configured in the conﬁg-yast-ﬁrstboot.xml file. When the firstboot process completes successfully the environment is changed such that the ﬁrstboot procedure will not be executed during a subsequent boot of the system.
The firstboot process offers an additional feature for customization. It is possible to execute scripts at the end of the firstboot configuration. Place your scripts in the /usr/share/firstboot/scripts directory and they will be executed after the init-scripts.
For example, if you have a shell script called myscript.sh that should be executed at the end of the firstboot procedure copy the script into the directory <MY_CONFIG_DIR>/root/usr/share/firstboot/scripts.
chmod 755 myscript.sh cp myscript.sh <MY_CONFIG_DIR>/root/usr/share/firstboot/scripts
The script must be executable. The feature works for any executable file, thus you can also place compiled binaries into <MY_CONFIG_DIR>/root/usr/share/firstboot/scripts to have them executed when the system is initially booted.
Delete Packages you do not need/want
The mandatory <packages> element specifies a list of package and/or pattern names. The type attribute of the element determines how the named packages or patterns are used.
In previous examples we focused on the packages list element of type image. Specifying the type image indicates that any packages or patterns listed within the packages list of this type will be part of the system being created.
<packages type="image"> ... </packages>
We also previously encountered the type bootstrap. Packages and patterns named in the packages list of type bootstrap are installed externally to the image and are used to bootstrap the new root system being created by KIWI. This external system just needs enough software to chroot into the new system. Therefore, glibc and filesystem are sufficient as entries in this list.
<packages type="bootstrap"> ... </packages>
The problem with always adding packages and patterns is of course that our image keeps getting bigger. Also it is inevitable that some packages get pulled in that are unwanted or unneeded. Therefore, removing packages maybe just as important as adding packages. Using the value delete for the type attribute of the <packages> element allows you to delete packages from your image.
<packages type="delete"> ... </packages>
When working with patterns there is a good chance that you end up with packages you do not need/want in you image. For example the kde4 pattern includes a large number of packages. Some of these packages may not be suitable for the image you are configuring.
One possibility is that you add all the packages you need want to the <packages type="image"> list instead of pattern. This approach may result in a lot of typing and research into package names and their correlation to patterns. The second possibility is to use the pattern and then delete the packages that are of no interest to you.
In this example we use the patterns base and kde4. However, we would like to exclude the OpenOffice, Firefox and some game packages from the image to reduce the image size. The following specification within the config.xml file allows us to delete these packages.
<packages type="delete"> <package name="MozillaFirefox"/> <package name="OpenOffice_org-branding-openSUSE"/> <package name="OpenOffice_org-templates-labels-a4"/> <package name="OpenOffice_org-calc"/> ... </packages>
The <packages> element supports 2 additional values for the type attribute; These are xen and vmware. With these attribute values one can specify packages to be included only if a Xen or VMware image is being created, respectively. The packages in these sections are included in addition to the packages listed for type image as appropriate.
Adding Users and Groups
The optional <users> element specifies a list of users to be added to the image. The mandatory group attribute of the <users> element specifies the group to which the listed users belong. If the specified group does not exist it will be created. The users list may not be empty and must contain at least one entry of type <user>. The <user> element has the attributes name, pwd and home to specify the username, encrypted password, and path to the user's home directory, respectively.
The encrypted password can be created by the kiwi −−createpassword tool and and you can then cut and paste the result into the config.xml file.
<users group="users"> <user pwd="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0" home="/home/tux" name="tux"/> </users> <users group="root"> <user pwd="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0" home="/root" name="root"/> </users>
In this example we created a user named tux within the group users and a user root within the group root. Both have the same password.
It is also possible to add a user-id or a group-id when using the optional id attribute of the <users> or <user> elements as shown below.
<users group="users" id="[the-id-as-number]"> <user pwd=... home=... name=... id="[the-id-as-number]"> ... </users>
A second look at preference settings
The mandatory <preferences> element is a list element containing information about the desired image type(s), the package manager, the version of this image and other optional preference elements.
Previously we noted the <type> element, used to control the image type that will be created by Kiwi. In addition to the type element the preferences list may contain a number of other elements. Some of these are described in more detail below. Below is the preferences list for our current example.
<preferences> <type filesystem="squashfs" boot="usbboot/suse-11.1">usb</type> <type filesystem="squashfs" boot="oemboot/suse-11.1">oem</type> <type filesystem="squashfs" boot="vmxboot/suse-11.1">vmx</type> <version>1.1.2</version> <packagemanager>zypper</packagemanager> <rpm-check-signatures>false</rpm-check-signatures> <rpm-force>true</rpm-force> <rpm-excludedocs>true</rpm-excludedocs> <oem-swap>false</oem-swap> <oem-kiwi-initrd>true</oem-kiwi-initrd> <oem-boot-title>USB</oem-boot-title> <locale>en_US.UTF-8</locale> <keytable>us.map.gz</keytable> <timezone>US/Eastern</timezone> </preferences>
Starting with the type element which we have encountered in previous examples, it is notable that we can build 3 different types of images, usb, oem, and vmx, with this description. None of the specified types is marked as primary and therefore when using this description the desired image type must be provided on the command line.
- type usb
- Use this type to create a USB stick system along with the attributes filesystem and boot=”usbboot/suse-*”
- type usb
- type oem
- Use this type to create a preload disk system along with the attributes filesystem, boot=”oemboot/suse-*”
- With the optional format attribute, set to ”iso” or ”usb” KIWI will additionally create an installation image suitable for a CD/DVD or a USB stick.
- The oem image will deploy the pre-load system onto the storage devices of the system being booted from the installation media. The storage devices are detected at boot time.
- type oem
- type vmx
- Use this type to create a virtual disk system along with the attributes filesystem, boot=”vmxboot/suse-*” and the optional format attribute.
- The format attribute specifies one of the qemu supported virtualization formats, for example vmdk or qcow2.
- The optional attribute vga can be specified to configure the kernel framebuffer mode. Detailed information about the possible values can be found in /usr/src/linux/Documentation/fb/vesafb.txt.
- The vga attribute also works for the image types usb and oem.
- type vmx
- Additional types
- In addition to the types listed above kiwi supports types of iso, xen, pxe and ec2.
- Additional types
For details about the types please refer to the KIWI Image System Cookbook
In this example the filessytem attribute is set to squashfs. You can choose any file system supported by Linux as a boot partition. Some special characteristics of the compressed squashfs file system are listed below.
- This will compress the image using the squashfs file system.
- The boot process will automatically use the aufs file system as an overlay file system to mount the complete tree in read-write mode.
- Kiwi creates an additional ext2 partition on the USB stick to support write mode. In order to support compression with squashfs it is required that the squashfs package is installed on the system used to create the image. This requirement also applies to aufs.
Next we will take a look at other entries within the preferences list element.
The OEM Type and OEM preference settings
By default the oemboot process will create or modify a swap, /home and / (root) partition. Kiwi supports OEM specific elements to customize the behavior of the oemboot process.
- Specify if a swap partition should be create.
- If this element is set to true the initial oemboot boot image (initrd) will not be replaced by the system (mkinitrd) created initrd.
- This option makes sense if the target storage device for the image is not a fixed disk but for example a USB stick. In that case it might be required to re-detect the storage location on first boot which is done as part of the oemboot boot image
- By default the string OEM will be appended to the boot manager menu when KIWI creates the grub configuration during first deployment. The oem-boot-title value allows to set a custom name which is used instead of OEM.
Kiwi supports additional oem specific option element, please refer to the KIWI Image System Cookbook for more details.
Some package management (rpm) specific elements
- Specifies whether RPM should check the package signature or not
- Specifies whether RPM should be called with –force
- Specifies whether RPM should skip installing package documentation
- Specifies the name of the console key-map to use. The value corresponds to a map file in /usr/share/kbd/keymaps such as us.map.gz, de.map.gz, or dvorak.map.gz. The KEYTABLE variable in /etc/sysconfig/keyboard file is set according to the keyboard mapping.
- Specifies the time zone. Available time zones are located in the /usr/share/zoneinfo directory. Specify the attribute value relative to /usr/share/zoneinfo.
- For example, specify Europe/Berlin for /usr/share/zoneinfo/Europe/Berlin. KIWI uses this value to configure the timezone in /etc/localtime for the image
- Specifies the name of the locale to use, which defines the contents of the RC_LANG system environment variable in /etc/sysconfig/language. The value corresponds to entries in /usr/share/X11/locale such as en_US.UTF-8 or ja_JP.UTF-8.
Attributes of the packages element
The packages list element contains the list of packages and patterns used to build the image. Patterns are SUSE specific and available with openSUSE 10.1 or later. A pattern is identified by a name, such as gnome or kde and identifies a number of packages associated with the named pattern. Using patterns reduces the number of packages that need to be listed explicitly.
By default, only required patterns and packages are automatically pulled into the image.
<packages type="image" patternType="plusRecommended">
<packages type="image" patternPackageType="plusRecommended">
Possible values for the patternType and patternPackageType are
- Incorporates patterns and packages that are required and recommended by the given pattern.
- Incorporates patterns and packages that are required and suggested by the given pattern
- Incorporates only patterns and packages that are required by the given pattern - this is the default
The specified list of packages (all packages listed by name and packages included through a pattern name) is used to generate a list of packages to be installed. The generated list is the solution of the dependency resolution for all specified packages. This can imply that the inclusion of a suggested package results in the inclusion of additional required and recommended packages.
Additional details may be found in the KIWI Image System Cookbook
The config.sh file
At the end of the preparation step (kiwi -p) the config.sh script is executed if it exists at the top level of the configuration directory. The script is intended to be used for configuration tasks of the operating system within the appliance.
Configuration tasks are steps such as
- activating services
- creating configuration files
- preparing an environment for a firstboot work flow
If config.sh exits with an exit code != 0 kiwi will exit the prepare step with an error.
It is strongly discouraged to use config.sh as a means to "manually" remove packages or parts of packages. This may have a negative impact on the integrity of the image and may cause instabilities.
For additional software management it is recommended to use the images.sh script. The images.sh script is called, if it exists at the top level of the configuration directory at the beginning of the create (kiwi -c) step.
Unlike the config.sh script the purpose of the install.sh script is less defined and it is provided as a hook to work with the image being created outside the bounds of what would be considered image configuration. The images.sh script is commonly used remove packages included in the image via dependencies but that are not needed for the later use of the operating system.
Within the config.sh and images.sh scripts a set of functions are available to complete common tasks. For a detailed description of these functions please refer to the kiwi::config.sh or kiwi::images.sh manual page
The overlay concept
In our previous examples you may have noticed a directory named root at the top level of the configuration directory and may have wondered about its purpose especially considering the special name. The root directory in the image description represents an entry point to the root tree (/) of the image being created. Any files and directories that exist inside the root directory will be copied to the unpacked image root tree after all packages have been installed.
With the root directory overlay concept you have a very powerful customization method at your finger tips. It is very easy to apply specific configuration settings to the image by simply placing the configuration file in the correct location inside the root directory.
In this example the overlay mechanism is used for the following configuration:
./root `-- etc |-- X11 | `-- xorg.conf <== the X configuration |-- init.d | |-- earlyxdm <== necessary start script for X | |-- sax <== X configuration program | `-- xdm <== start script for a displaymanager |-- inittab `-- sysconfig |-- displaymanager <= settings for the displaymanager started with help of xdm |-- network | `-- ifcfg-eth0 <= a default network configuration file `-- windowmanager <= settings for the windowmanager
The commands to build our example
With this we have created the unpacked image in /tmp/myusb/image_root. From our earlier examination of the configuration file we know that we can create two different images from our tree in image_root.
- The "usb" image type
- creates all required files for booting the OS but requires the use of Kiwi to deploy the image on the USB stick.
- The "oem" image type
- allows you to create a virtual disk which represents a virtual disk geometry including all partitions and boot information in one file. You can deploy this file on the USB stick using the "dd" command. Upon initial boot the virtual disk geometry will be adapted to the real disk geometry of the USB stick.
- Create the image
- Plug in the stick
- Deploy the image
- Create the image
- Plug in the stick
- Determine the device path
- Deploy the image
dd if=/tmp/myusb/image/suse-11.1-live-stick.i686-1.1.2.raw of=</dev/yourdev> bs=32
Caveats with respect to image name and building in 64 bit mode apply as noted previously.
You can test the USB stick using a machine capable of booting from a USB device or using a virtual machine of your choice. If you test the stick using a VM please be aware that the USB stick looks like a normal disk to the VM. This is important as the kiwi boot process searches for the USB stick in order to mount the correct storage device. However, in a virtual environment the USB stick appears as a disk and not as a USB stick. Therefore, if your virtualization solution does not provide a virtual BIOS which supports booting from a USB stick you should test the stick on real hardware.
* User root pwd: linux * User tux pwd: linux