SDB:KIWI Cookbook Own Cloud

Jump to: navigation, search


Using 3rd party software within our images

All of KIWI edit

Portal:KIWI/Navigation

Icon-checked.png
This procedure was tested on at least Kiwi version 3.25 ( kiwi --version ) - older version did not support all shown features Version


Create your own Cloud

This example will guide you through the steps to create your own cloud infrastructure based on Eucalyptus. The example leads you through the build o 3 separate images, a head node image, a cloud node image and a guest image. With the images created it will take about 10 minutes to set up your own cloud and the deployment on new nodes can be accomplished in as little as 5 minutes.

As of Kiwi version 4.25 the kiwi-doc package contains the suse-euca-cloud example in the suse-11.1 and suse-11.2 directories in /usr/share/doc/packages/kiwi/examples.

CLOUDS

Create the cloud images

Prep Time:

  • 60 min

Cook Time:

  • 40 min

Ingredients:

  • a running openSUSE 11.1 or 11.2 system
  • an openSUSE 11.1 or 11.2 repository
  • installed recent release of the KIWI toolset (at least version 4.25)
  • installed kiwi-doc package
  • downloaded Eucalyptus tarball and Euca2ools tarball
  • about 1 GB free disk space


If you are building your infrastructure based on openSUSE 11.1 you need to use Eucalyptus version 1.6.1. For openSUSE 11.2 or later based infrastructure you can use Eucalyptus 1.6.1 or Eucalyptus 1.6.2.

Creating the head node

Create the head node configuration

Lets start out by copying the example provided by the kiw-doc package to a working directory. For this example the working directory will be /tmp/cloud-setup

cp -r /usr/share/doc/packages/kiwi/examples/suse-11.2/suse-euca-cloud/ /tmp/cloud-setup

If you have not already downloaded the Eucalyptus and the Euca2ools tarballs, download them now and place them in /tmp/cloud-setup.

Next change to our working directory and extract the Eucalyptus packages.

cd /tmp/cloud-setup
tar -xzvf euca2ools-1.2-opensuse-x86_64.tar.gz
tar -xzvf eucalyptus-1.6.2-opensuse-x86_64.tar.gz
If you are building a 32 bit based infrastructure or using a different version of Eucalyptus, substitute the appropriate tarball names in the tar commands above.

With the following step we will add the package directories created by the extraction of the tarballs to the Kiwi configuration as repositories. Edit the config.xml file in the cloud-head-node directory. Replace the entry PATH_TO_EUCALYPTUS_RPMS with the path to the Eucalyptus rpms, in this example the path is /tmp/cloud-setup/eucalyptus-1.6.2-opensuse-x86_64. Then copy the repository declaration and add eucalyptus-1.6.2-rpm-deps-x86_64 to the end of the path. Copy the repository declaration again and point the path to the euca2ools-1.2-opensuse-x86_64 directory. When completed you should have the following repository configuration in config.xml. 

    <repository type="rpm-dir">
        <source path="tmp/cloud-setup/eucalyptus-1.6.2-opensuse-x86_64"/>
	</repository>
    <repository type="rpm-dir">
        <source path="tmp/cloud-setup/eucalyptus-1.6.2-opensuse-x86_64/eucalyptus-1.6.2-rpm-deps-x86_64"/>
    </repository>
    <repository type="rpm-dir">
        <source path="tmp/cloud-setup/euca2ools-1.2-opensuse-x86_64"/>
    </repository>
If you do not want to pull the openSUSE packages over the Internet do not forget to change the repository to some local repository, possibly the mount point of the openSUSE ISO image.

If you want to include the latest and greatest openSUSE packages add the Updates repository to the config.xml file. Adding the Updates repository has the following syntax: 

    <repository type="rpm-md">
	<source path="http://download.opensuse.org/update/11.2/"/>
    </repository>

This completes the changes to config.xml.

If you are building a 64 bit image some changes to the config.sh file are necessary. The 64 bit euca2ools package places required Python files in the 32 bit tree rather than the 64 bit tree. Thus, a link is required. This has been reported and is expected to be fixed in some later version of the euca2ools package. After the suseConfig command in config.sh add the following entries. 

#====================================== 
# Workaround for broken euca2ools package 
#-------------------------------------- 
ln -s /usr/lib/python2.6/site-packages/boto /usr/lib64/python.6/site-packages
ln -s /usr/lib/python2.6/site-packages/euca2ools /usr/lib64/python.6/site-packages

This completes the required changes to the example configuration.

The configuration is set up to create an OEM image in the ISO format, i.e. a self installing ISO image that you can burn to CD. Once booted from the CD the head node image will install itself on the machine you are booting. It is also sensible to run the head node in a VM and you can change the image type in the configuration if you so desire. Next lets build the head node image.

Build the head node image

kiwi --prepare ./cloud-head-node --root /tmp/cloud-setup/cloud-head-unpacked

After the prepare step create the image.

kiwi --create ./cloud-head-unpacked -d /tmp/cloud-setup/cloud-head-image

You can test the image in a virtual environment. With qemu on a 64 bit system this works as follows:

Create a virtual disk

qemu-img create -f qcow2 /tmp/cloud_head_test.img 10G

Boot the created ISO

qemu-kvm /tmp/cloud_head_test.img -cdrom /tmp/cloud-setup/cloud-head-image/Eucalyptus-Head-Node.x86_64-0.9.2.iso -boot d

That completes the head node creation and you can install the image on suitable hardware.

Creating the cloud node

Create the cloud node configuration

Edit the config.xml file in the cloud-cloud-node directory with your favorite editor. As with the head node configuration previously, set up your repositories (you may copy the configured repository elements from your head node configuration file).

Notable about the cloud node configuration is that the image is set up as a Xen hypervisor, i.e. dom0, image. This is accomplished with the <xenconfig> element and the bootprofile and bootkernel attributes of the <type> element. 

    <type image="oem" filesystem="ext4" boot="oemboot/suse-11.2" format="iso" bootprofile="xen" bootkernel="xenk">
        .......
        <xenconfig domain="dom0"/>
    </type>
If you are adapting the config files to work with a version of Kiwi prior to 4.20 you must change the value of the bootkernel attribute from xenk to nex.

The configuration for the cloud node also produces an OEM image in the ISO format. If you would rather create an image that self installs from a USB stick you can simply change the value of the format attribute of the <type> element from iso to usb. The result will be a .raw.install.raw file. Dump this image to a USB stick with the dd command (don't forget to use the bs=32 argument).

Build the cloud node image

Now you can build your cloud node image.

kiwi --prepare ./cloud-cloud-node --root /tmp/cloud-setup/cloud-cloud-unpacked
kiwi --create ./cloud-cloud-unpacked -d /tmp/cloud-setup/cloud-cloud-image


Cloud infrastructure setup

With the head and cloud node images complete you can install the images on suitable hardware. Follow the instructions in the Administrator's Guide to configure the cloud infrastructure.

On the cloud nodes you need to create a network bridge to allow the guest images to connect to the network. You can use the command line or for a nice GUI driven configuration use YaST.

As the cloud infrastructure without guest images is pretty useless the next steps explain how to create the guest images and how to register the guests with Eucalyptus.


Create a cloud image

The configuration of the cloud nodes is set up to work with Xen. Therefore, our cloud virtual machines need to be build as Xen guest (domU) images. For demonstration purposes the xen-guest example provided by the kiwi-doc package is used here.

You may use the xen-guest example without modification. In this case the example is copied to the working directory /tmp/clous-setup to modify the repository declaration to use a local ISO image. The unmodified xen-guest example pulls the packages over the Internet.
cp -r /usr/share/doc/packages/kiwi/examples/suse-11.2/suse-xen-guest .
kiwi --prepare ./suse-xen-guest --root /tmp/cloud-setup/cloud-guest-unpacked
kiwi --create ./cloud-guest-unpacked -d /tmp/cloud-setup/cloud-guest-image

The xen-guest example produces an "unpackaged" Xen image that makes it straight forward to incorporate into Eucalyptus as explained below.

The cloud guest images may also be created using SUSE Studio. With studio the result is a packed Xen guest image and it takes a bit more effort to integrate it into the cloud infrastructure. These steps are outlined below as well.

Integrate the guest image

Prepare the guest image for integration

The Kiwi image
  • Create a tar archive of your results directory, i.e. /tmp/cloud-setup/cloud-guest-image in our example
  • Copy/move the tarball to your Eucalyptus head node to a directory of your choice
  • Unpack the tarball.
A SUSE Studio image
  • Download the tarball from SUSE Studio
  • Copy/move the tarball to your Eucalyptus head node to a directory of your choice
  • Unpack the tarball
    • The tarball contains:
      • The configuration file ending in .xenconfig
      • The disk image file ending in.raw
  • Extract the the kernel and the initrd from the disk image as follows:
losetup /dev/loop1 FILENAME.raw
It is assumed that loop1 is not already in use, check with the mount command and grep loop1 to ascertain this to be true before issuing the losetup command.
kpartx -a /dev/loop1
mount -t ext3 /dev/mapper/loop1p1 /mnt
Make sure your file system type matches the file system used to build the guest image.
 mkdir SOMEWHERE/my_xen_guest
Replace SOMEWHERE with a path of your choice. The name my_xen_guest is arbitrary and only used as an example.
 cp /mnt/boot/initrd.vmx SOMEWHERE/my_xen_guest
 cp /mnt/boot/vmlinuz-KERNEL_VERSION-xen SOMEWHERE/my_xen_guest
The final two steps copy the initrd and the kernel out of the disk image. If you are uncertain about the names, look in /mnt/boot/grub/menu.lst to determine the names for the initrd and the kernel and copy the appropriate files.
  • Now you can undo the setup of the .raw file
umount /mnt
kpartx -d /dev/loop1
losetup -d /dev/loop1

Bundling and Uploading

Each Eucalyptus machine image consists of three files:

  1. filesystem
  2. initial ramdisk (initrd)
  3. kernel

Each of these three image files needs to be bundled, uploaded, and registered with the Eucalyptus infrastructure. Bundling, uploading, and registration is completed with the following three commands:

  1. euca-bundle-image
  2. euca-upload-bundle
  3. euca-register

Detailed information about this process and the commands can be found in the Image Management section of the Eucalyptus Administrator's Guide.

Before you begin with the bundling download your credentials from the web interface and unpack the zip file. The source the eucarc file. This will set the necessary environment variables if you are running bash or sh. For other shells you will need to make appropriate adjustments.

Bundle and Upload Initial Ramdisk
Using the Kiwi created guest

Identify the file name of the initial ramdisk (initrd) by examining the .xenconfig file in the results directory, i.e. cloud-guest-image/suse-11.2-xen-guest.x86_64-1.1.2.xenconfig in our example.

Using a SUSE Studio created guest

Use the initrd.vmx file you previously copied to a directory of your choice.


  • Create a manifest file by bundling the initrd file
 euca-bundle-image --ramdisk true -i <initrd file>
  • Upload the bundle specifying a bucket name and the manifest created in the previous step. The bucket name is a string of your choosing.
 euca-upload-bundle -b <bucket name> -m /tmp/<filename of manifest>
  • Register this image with the Eucalyptus cloud management framework
 euca-register <bucket name>/<filename of manifest>
  • Note the Eucalyptus ramdisk identifier (eri-xxxxxx) that is produced.
Bundle and Upload the Linux Kernel
Using the Kiwi created guest

Identify the file name of the kernel by examining the .xenconfig file in the results directory, i.e. cloud-guest-image/suse-11.2-xen-guest.x86_64-1.1.2.xenconfig in our example.

Using a SUSE Studio created guest

Use the kernel file you previously copied to a directory of your choice.


  • Create a manifest file by bundling the kernel
 euca-bundle-image --kernel true -i <kernel file>
  • Upload the bundle specifying the same bucket name used for the initrd previously and specifying the manifest file
 euca-upload-bundle -b <bucket name> -m /tmp/<filename of manifest>
  • Register this image with the Eucalyptus cloud management framework
 euca-register <bucket name>/<filename of manifest>
  • Note the Eucalyptus kernel identifier (eki-xxxxxx) that is produced.
Bundle and Upload the Root Filesystem
Using the Kiwi created guest

Identify the file name of the filesystem image by examining the .xenconfig file in the results directory, i.e. cloud-guest-image/suse-11.2-xen-guest.x86_64-1.1.2.xenconfig in our example.

Using a SUSE Studio created guest

Use the .raw file you previously copied to a directory of your choice. 

* Create a manifest file by bundling the filesystem, this requires the previously noted identifiers eri-xxxxxx and eki-xxxxxx
 euca-bundle-image --ramdisk <eri-xxxxxx> --kernel <eki-xxxxxx> -i <path to root filesystem>
  • Upload the bundle specifying the same bucket name used for the initrd and kernel previously and specifying the manifest file
 euca-upload-bundle -b <bucket name> -m /tmp/<filename of manifest>
  • Register this image with the Eucalyptus cloud management framework
 euca-register <bucket name>/<filename of manifest>
  • Note the Eucalyptus Machine Image (EMI) identifier (emi-xxxxxx)

This completes the bundling, uploading, and registration process and the newly created image can now be launched using the EMI identifier.

Using the cloud

  • Launch a guest image
 euca-run-instances -k <ssh key id> emi-xxxxxx
  • Show running instances:
euca-describe-instances
  • Show available images, ramdisks, and kernels:
euca-describe-images
  • Terminate Instance
    • Find its instance ID (i-xxxxxx) either from the initial run-instances command, or euca-describe-instances:
 euca-terminate-instances <i-xxxxxx>
Retrieved from "https://en.opensuse.org/index.php?title=SDB:KIWI_Cookbook_Own_Cloud&oldid=49159"