Building Stateless/Diskless Images

A stateless, or diskless, provisioned nodes is one where the operating system image is deployed and loaded into memory. The Operating System (OS) does not store its files directly onto persistent storage (i.e. hard disk drive, shared drive, usb, etc) and so subsequent rebooting of the machine results in loss of any state changes that happened while the machine was running.

To deploy stateless compute nodes, you must first create a stateless image. The “netboot” osimages created from copycds in the osimage table are sample osimage definitions that can be used for deploying stateless nodes.

In a homogeneous cluster, the management node is the same hardware architecture and running the same Operating System (OS) as the compute nodes, so genimage can directly be executed from the management node.

The issues arises in a heterogeneous cluster, where the management node is running a different level operating system or hardware architecture as the compute nodes in which to deploy the image. The genimage command that builds stateless images depends on various utilities provided by the base operating system and needs to be run on a node with the same hardware architecture and major Operating System release as the nodes that will be booted from the image.

When running xCAT >= 2.17 on EL >= 8 based management node with x86_64 architecture, qemu-user-static can be used to cross-build ppc64* and aarch64 osimages. Therefore, you don’t need to build images on systems with the target architecture anymore.

Cross-build ppc64*/aarch64 stateless/statelite image on x86_64 management node

Download qemu-user-static binaries for ppc64le and/or aarch64:

wget https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-ppc64le-static -P /usr/bin
wget https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-aarch64-static -P /usr/bin
chmod 755 /usr/bin/qemu-*-static

Configure systemd-binfmt accordingly:

wget https://raw.githubusercontent.com/qemu/qemu/master/scripts/qemu-binfmt-conf.sh
bash qemu-binfmt-conf.sh --systemd 'ALL' --qemu-path '/usr/bin' --qemu-suffix '-static' --persistent 'yes'
systemctl restart systemd-binfmt.service

Make sure osimage.osarch of your image is set to your target architecture eg. ppc64le or aarch64.

Now, genimage will use systemd-binfmt and the qemu-user-static binary to build the image for the specified architecture.

Same Operating System, Different Architecture

The following describes creating stateless images of the same Operating System, but different hardware architecture. The example will use the following nodes:

Management Node: xcatmn (ppc64)
Target node:     n01 (x86_64)

On xCAT management node, xcatmn, select the osimage you want to create from the list of osimage definitions. To list out the osimage definitions:
```
lsdef -t osimage
```
optional: Create a copy of the osimage definition that you want to modify.

To take the sample rhels6.3-x86_64-netboot-compute osimage definition and create a copy called mycomputeimage, run the following command:
```
lsdef -t osimage -z rhels6.3-x86_64-netboot-compute | sed 's/^[^ ]\+:/mycomputeimage:/' | mkdef -z
```

To obtain the genimage command to execute on n01, execute the genimage command with the --dryrun option:

genimage --dryrun mycomputeimage

The result output will look similar to the following:

Generating image:
cd /opt/xcat/share/xcat/netboot/rh;
./genimage -a x86_64 -o rhels6.3 -p compute --permission 755 --srcdir /install/rhels6.3/x86_64 --pkglist \
/opt/xcat/share/xcat/netboot/rh/compute.rhels6.x86_64.pkglist --otherpkgdir /install/post/otherpkgs/rhels6.3/x86_64 --postinstall \
/opt/xcat/share/xcat/netboot/rh/compute.rhels6.x86_64.postinstall --rootimgdir /install/netboot/rhels6.3/x86_64/compute mycomputeimage

Go to the target node, n01 and run the following:
1. mount the /install directory from the xCAT Management Node:
```
mkdir /install
mount -o soft xcatmn:/install /install
```
2. Copy the executable files from the /opt/xcat/share/xcat/netboot from the xCAT Management node to the target node:
```
mkdir -p /opt/xcat/share/xcat/
scp -r xcatmn:/opt/xcat/share/xcat/netboot /opt/xcat/share/xcat/
```

Execute the genimage command obtained from the --dryrun:

cd /opt/xcat/share/xcat/netboot/rh;
./genimage -a x86_64 -o rhels6.3 -p compute --permission 755 --srcdir /install/rhels6.3/x86_64 --pkglist \
 /opt/xcat/share/xcat/netboot/rh/compute.rhels6.x86_64.pkglist --otherpkgdir /install/post/otherpkgs/rhels6.3/x86_64 --postinstall \
 /opt/xcat/share/xcat/netboot/rh/compute.rhels6.x86_64.postinstall --rootimgdir /install/netboot/rhels6.3/x86_64/compute mycomputeimage

If problems creating the stateless image, provide a local directory for –rootimgdir:

mkdir -p /tmp/compute

Rerun genimage, replacing --rootimgdir /tmp/compute:

cd /opt/xcat/share/xcat/netboot/rh;
./genimage -a x86_64 -o rhels6.3 -p compute --permission 755 --srcdir /install/rhels6.3/x86_64 --pkglist \
 /opt/xcat/share/xcat/netboot/rh/compute.rhels6.x86_64.pkglist --otherpkgdir /install/post/otherpkgs/rhels6.3/x86_64 --postinstall \
 /opt/xcat/share/xcat/netboot/rh/compute.rhels6.x86_64.postinstall --rootimgdir /tmp/compute mycomputeimage

Then copy the contents from /tmp/compute to /install/netboot/rhels6.3/compute

Now return to the management node and execute packimage on the osimage and continue provisioning the node
```
packimage mycomputeimage
```