SDB:Hercules s390 emulation
Hercules is not officially supported or sanctioned by IBM. In fact, there are web pages that abundantly describe the position that IBM has regarding Hercules. It's important to note that it is your responsibility to ensure that you do not violate any of IBM's -- or any one else's -- software licensing terms. In this HOWTO I only discuss installing and running SLES for s390 as a guest in the Hercules emulator.
Hercules can run on Windows and Macintosh hosts, too, but this documentation addresses only Hercules running as an application on Linux, and specifically on SUSE Linux Enterprise. The focus provides a succinct set of examples for this particular configuration so that it can be easily duplicated at SUSE.
System Requirements
Hercules can and will run on old hardware and software, but for this application it's important to get the most performance and reliability that can be found. If the host hardware is quite slow, remember that the guest will run significantly slower; it's emulation software, after all. Also note that Hercules is not devoid of bugs. It's possible that you may encounter runtime difficulties under Hercules that you won't encounter on a proper zSeries mainframe.
It's entirely possible to run Hercules on a 32bit host system, but I do not recommend it. I would encourage the use of an x86_64 OS, with either AMD or Intel EM64T. (I've even been able to build and run Hercules on IA64, but I haven't had enough experience running anything within the emulator on IA64 to assert any quality of experience.)
Recently I've attempted running Hercules on a XEN guest (double virtualization). I wouldn't recommend it. But it's apparently possible. (The floating point operations in the s390 guest didn't seem to work properly, and would even core. Any attempt that gcc made to compile using floating point variables crashed the compiler. Really strange.)
As with all applications, the more RAM and CPU you can give to the application the better it will run. I'd recommend no less that 1GB for the host OS, which allows for about 800MB for the guest running in Hercules. And you need at least 2 CPUs, so that at least 1 can be dedicated the Hercules guest.
For the system to be at all useful you'll need a network configuration. The host will act as a proxy for the guest running under Hercules. All network traffic is tunneled through a CTC (point-to-point; literally "client to client") interface facilitated by the host's TUN adapter.
Initial Host Setup
For this HOWTO I'll refer to the host by the hostname DELTA. Later the guest running within the Hercules emulator will be called TANGO.
Although the host (DELTA) can basically be any relatively recent distribution from a variety of vendors, these instructions refer to the use of SUSE Linux Enterprise Server. Although it is technically possible to use openSUSE for the host, I've never actually tried it.
The installation of the guest (TANGO) requires that the host (DELTA) has the following:
- A properly configured and working network device on the host (DELTA). IPv4 is used in these examples.
- An ftp server, whether pure-ftpd or vsftp, or some other of your choosing. The FTP service must be configure and running. Normally you would simply allow anonymous access, so that the zLinux guest can install without having to provide authentication credentials, but you're welcome to configure it in the way of your choosing.
- A downloaded ISO of the distribution that will be installed on the guest (TANGO). The downloaded ISO will be accessed during the installation via the CTC bridged network, and provided from DELTA via ftp. Be sure to download the s390 ISO. Place the ISO in a directory such as /srv/ftp/iso/ (or in any other appropriate directory where you have space). Loopback mount the ISO to /srv/ftp/install/, such that the content of ISO can be served by DELTA's FTP service.
Install hercules.rpm
The rpm can be downloaded from the Open Build Service. Be sure to download the proper rpm representing your distribution (i.e.: SUSE Linux Enterprise Server 11 SP1, or openSUSE 11.4) and architecture (i.e.: i586, x86_64). Note that OBS doesn't facilitate builds for architectures other than ix86 and x86_64. I have ia64 rpms available; please contact me if you would like to run Hercules on IA64.
Install the hercules rpm on the host (DELTA).
DASD Creation
DASD is IBM's rendition of DISK. You need one or more DASD disk packs in order to install SLES for zLinux. The creation of the DASD disk pack will result in a file on your host (DELTA) linux system that will be mounted by the zLinux system running in your hercules application, and thus will be presented as a normal linux disk device for you to-be-installed guest (TANGO).
To create the DASD, you'll need a large enough amount of free dist on the host (DELTA). To have a 500 GB disk for your guest (TANGO), you'll need to create 500 GB DASD file on the host (DELTA).
DASD disk packs come in a few different flavors. The most common that I have seen used -- and the flavor that I use the most -- is the 3390 model 9. It is roughly 9 GB, which is perfect for the root partition (/) that will contain the bulk of the guest operating system running as TANGO. You can see a page like http://www.bsp-gmbh.com/turnkey/cookbook/hercules/hercload.html to begin to understand the various DASD types and flavors.
Although a swap partition is technically not required for the guest system, you may choose to also create a DASD disk pack file for a swap partition.
As a convenience, create a dedicated directory for your DASD files:
# mkdir /DASD/ # cd /DASD/
Create the DASD files:
# dasdinit -linux -lfs SLES11.3390-9 3390-9 ROOT # dasdinit -linux -lfs SLES11.swap 3390-2 SWAP
The first command will create the file /DASD/SLES11.3390-9 with the proper type (3390-9) that will be readily recognized by the zLinux system running in the Hercules emulator. Its size is roughly 9GB. The trailing "ROOT" argument is a simple 1 to 6 character volume serial identifier (of your choosing). The second command creates a smaller file for the optional swap partition. Its size is about 2GB.
Configuration File
Here's a rudimentary configuration file. (Note that a full sample configuration file -- including annotations -- is included in the rpm that I publish.)
Make sure that you assign RAM (MAINSIZE) and Processors (NUMCPU) that really can be used by the guest. If your host doesn't have that amount RAM or CPUs to spare, hercules will fail to load.
Note that 800 MB of RAM for the zLinux guest is REALLY small.
#------------------------------------------------------------------------------
# CPU Configuration
#------------------------------------------------------------------------------
CPUSERIAL 002623 # CPU serial number
CPUMODEL 2064 # CPU model number
MODEL EMULATOR # STSI returned model
PLANT ZZ # STSI returned plant
MANUFACTURER HRC # STSI returned manufacturer
LPARNAME HERCULES # DIAG 204 returned lparname
CPUVERID FD # CPU Version Identification
MAINSIZE 800 # Main storage size in megabytes
XPNDSIZE 0 # Expanded storage size in megabytes
NUMCPU 2 # Number of CPUs
ARCHMODE ESAME # Architecture mode S/370, ESA/390 or z/Arch
ALRF DISABLE # ASN-and-LX-Reuse facility
ECPSVM NO # VM Assist : NO or Level (20 recommended)
#------------------------------------------------------------------------------
# OS Tailoring
#------------------------------------------------------------------------------
LOADPARM 0150.... # IPL parameter
OSTAILOR LINUX # OS tailoring
SYSEPOCH 1900 # Base year for initial TOD clock
#TZOFFSET 0 # Using UTC (GMT)
TZOFFSET -0700 # Timezone
#------------------------------------------------------------------------------
# Devices
#------------------------------------------------------------------------------
# .-----------------------Device number
# | .-----------------Device type
# | | .---------File name and parameters
# | | |
# V V V
# ---- ---- --------------------
0009 3215-C / noprompt
001F 3270
# Our disk
0140 3390 /DASD/swap.0140
0150 3390 /DASD/SLES11.3390-9
# The Installation DVD
0160 9336 /srv/ftp/iso/SLES-11-SP1-DVD-s390x-GM-DVD1.iso
# The Network
0E20,0E21 3088 CTCI /dev/net/tun 1500 192.168.254.21 192.168.254.20 255.255.255.0
A few important words about the devices listed in the configuration file.
- The 0140 device is the swap device for the Linux guest (TANGO).
- The 0150 device is the primary disk partition that will be used as the root (/) filesystem for Linux guest (TANGO).
- 0E20 and 0E21 are the CTC network devices that form the bridge between the host (DELTA) and the zLinux system running within Hercules. The zLinux device is then presented to the guest (TANGO) as a ctc device, thus bridging the guest (TANGO) to the network via the host's (DELTA's) tunnel device.
Network Configuration
DELTA must be setup with a valid and working network configuration if TANGO is expected to have a working network. All traffic from TANGO to the external network is tunneled over DELTA's NIC device using the tun kernel module and the /dev/net/tun device. On SLE11 (and SLE10) the tun module is installed automatically and the /dev/net/tun device node is created with the installation of the kernel.
To provide the network tunnel from TANGO's perspective, the tunnel is facilitated by a virtual CTC link (client to client). This is essentially peer-to-peer network.
Note that hercules will (should) automatically assign the tun0 (or tunX, where X is an integer representing the Xth tunneled device on DELTA) when the hercules application is started.
Here's a helpful table and graphic to keep track:
A = DELTA's eth0 = 192.168.254.19
B = DELTA's tun0 = 192.168.254.20
C = zLinux-side CTC adapter = 192.168.254.21 = (a.k.a. "PLIP Partner" device)
D = TANGO's ctc0 = 192.168.254.22
+--------------------------------------+
| Linux/x86 Driving System |
| (hostname=DELTA) |
+-------------------+ |
| Hercules | +-----------------+
| Linux Guest | | eth0 |
| (hostname=TANGO) | TCP/IP -------------------------> Network
| *D* | | | 192.168.254.19 |
| 192.168.254.22 | | | *A* |
|-------------------| | +-----------------+
| CTCA | | |
| *C* | *B* |
| 192.168.254.21 | 192.168.254.20 |
| (PLIP Partner) | tun0 |
+----------|--------+ | |
| /dev/tun | |
| | | |
| +------------------+ |
| Virtual CTC link |
| |
+--------------------------------------+
Since DELTA is used to tunnel the network traffic of TANGO, DELTA must be configured to act as a (ipv4) proxy. The manual setting of this forwarding capability can be effectively done with the following command on DELTA:
# echo 1 > /proc/sys/net/ipv4/ip_forward
SYSCTL can also be used to set the kernel parameter automatically on reboot. Add the following line to /etc/sysctl.conf :
net.ipv4.ip_forward=1
Setting this permanently is conceptually easy, but more difficult in reality. The SYSCTL mechanism in SLE should work to set the kernel parameter permanently, but my personal experience is that too often the parameter gets reset to 0 despite the SYSCTL setting. So I find myself having to set /proc/sys/net/ipv4/ip_forward to "1" manually prior to running hercules.
Setting ip_forward'ing on DELTA is something that can be changed either before or after hercules is running.
# route add 192.168.254.22 gw 192.168.254.20 dev tun0
Note that the first IP address listed is that of the Linux guest (*D* in the diagram above), and the second (the value of the gateway argument) is that of tun0 device of the host (*B* in the diagram above).
Also helpful is the ability to proxy the ARP table from DELTA to TANGO (and vice versa). After hercules is running, run the following commands:
# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp# echo 1 > /proc/sys/net/ipv4/conf/tun0/proxy_arp
Since the tun0 device is technically not available until hercules is running, there's no practical reason to set the proxy_arp kernel parameter for the tun0 device prior to running hercules (or in SYSCTL).
Starting Up Hercules
(Just a helpful hint. I always run my Hercules emulator with screen. It makes it much easier than muddling up a normal console, and allows me to move effortlessly back and forth between my host's console and the hercules console.)
Starting the Hercules emulation program is easy, once you've got a viable configuration file.
# hercules -f /etc/hercules/hercules.cnf > /var/log/hercules
Note that this uses /etc/hercules/hercules.cnf as the configuration file, and that all important runtime output is logged in /var/log/hercules. The log file is very important, and can be used to diagnose most problems.
Once started, the emulator takes control of your current console. Thus it's highly recommended that you start hercules using 'screen'. ('screen' provides a variety of advantages, which won't be discussed here.)
The Hercules program, once started, presents to you the Hercules console. A few helpful tips regarding the Hercules console are provided below.
Equally important to starting Hercules is quitting Hercules. At any time that you want or need to stop Hercules -- which will also stop any guest(s) running within the emulator -- simply issue the 'quit' command from the Hercules console.
Another important concept is that of 'stopping' the CPUs that have been allocated to the emulator. Stopping the CPUs allows you to forcefully (and perhaps destructively) stop the guest that is running in the emulator without stopping Hercules itself. The most common way to stop the CPUs is by issuing the "stopall" command from the Hercules console. To restart the CPUs when they are stopped, run "startall".
A Few Helpful Hercules (s390) commands
IPL (Initial Program Load) the SLES Installation
After starting hercules (remember to use 'screen'), you'll need to bootstrap the installation by IPL'ing from the loop-back-mounted ISO that you downloaded to DELTA. At the Hercules console prompt, run the following command:
Command ==> ipl /srv/ftp/install/suse.ins
"ipl" stands for "initial program load", and it takes awhile (depending on the speed of your host) to load the installation program. You'll see a lot of output to your screen. Eventually you'll see something like:
>>> Linuxrc v3.3.81 (Kernel 3.0.13-0.27-default) <<< Main Menu 1) Start Installation 2) Settings 3) Expert 4) Exit or Reboot
To view the settings, press ".2" at the Command ==> prompt. (That's a "period two".) Note that there is no item #0 on this list. If you want to go back to the previous menu, use ".0". If you want to use a default value, use ".<Enter>".
Answering the Installation questions.
From the main menu, press ".1" to start the installation. Press ".1" again to select "1) Start Installation or Update". Press ".2" to select "Network".
You'll likely need to manually configure the network settings.
Setting up network devices can be tricky. This is the thing that gives me the most trouble when I'm doing a fresh install on Hercules. Borrowing from the sample config file above, the "Device address for read channel" is "0.0.0e20", and the "write channel" is "0.0.0e21". (You can just accept the defaults (if they're right) by pressing "." and the Enter key.) When setting the IP address, you must first provide your Linux guest's intended address ("192.168.254.22" by the example), and then installation program will ask for the IP address of the PLIP partner. As indicated in the examples shown above, the value would be "192.168.254.21". If your CTC configuration is successful, you'll see "net.xxxxx: ctc0: Connected with remote side" displayed.
When asked to provide the IP address of the FTP server, remember that you should have setup an anonymous FTP server on the host. Use your host's IP address (such as "192.168.254.19" in the examples above).