Docker life-cycle management in xCAT

The Docker linux container technology is currently very popular. xCAT can help managing Docker containers. xCAT, as a system management tool has the natural advantage for supporting multiple operating systems, multiple architectures and large scale clusters.

This document describes how to use xCAT for docker management, from Docker Host setup to docker container operations.


This document was verified with:
  • Docker Version 1.10, 1.11
  • Docker API version 1.22
Docker Host has been verified on the following OS/ARCH combinations:
  • Ubuntu 14.04.3,x86_64
  • Ubuntu 15.10,x86_64
  • Ubuntu 16.04,x86_64
  • Ubuntu 16.04,ppc64el

Setting up Docker Host

The Docker Host is the bare metal server or virtual machine where Docker containers can run. It will be called dockerhost in the following sections.

The dockerhost at a minimum must provide the following:

  • An Operating System for running docker daemon
  • The certification related files to be used by Docker service for trusted connection.

Preparing osimage for docker host

The osimage represents the image of the Operating System which will be deployed on the dockerhost.

Copy files out from DVDs/ISOs and generate

[ubuntu x86_64]

copycds ubuntu-xxx-server-amd64.iso

[ubuntu16.04 ppc64el]

copycds ubuntu-16.04-server-ppc64el.iso

Create pkglist and otherpkglist of osimage for dockerhost

The pkglist file should contain the following:

# cat /install/custom/ubuntu/ubuntu.pkglist

The otherpkglist file should contain the following:

[ubuntu x86_64]

# cat /install/custom/ubuntu/ubuntu_docker.pkglist

[ubuntu16.04 ppc64el]

At the time of this writing (February 2016), docker package is not available for ppc64el architecture from You can follow instructions below on how to manually download and install it.

  • Download docker engine for ppc64el:
wget  -O /install/docker_ppc64el/docker.io_1.10.3-0ubuntu4_ppc64el.deb
  • Configure otherpkgdir like this:
  • The otherpkglist file should be:
# cat /install/custom/ubuntu/ubuntu_docker.pkglist

Create the osimage for dockerhost

The osimage for dockerhost will be like this:

[ubuntu x86_64]

# lsdef -t osimage ub14.04.03-x86_64-dockerhost
Object name: ub14.04.03-x86_64-dockerhost
    otherpkgdir= ubuntu-trusty main, trusty main

[ubuntu16.04 ppc64el]

# lsdef -t osimage ub16.04-ppc64el-dockerhost
Object name: ub16.04-ppc64el-dockerhost

Preparing setup trust connection for docker service and create docker network object

Currently, a customer defined network object is needed when create a docker container with static IP address, it can be done with the command:

chdef host01 -p postbootscripts="setupdockerhost <netobj_name>=<subnet>/<netmask>@<gateway>[:nicname]"
  • netobj_name: the network object to be created, it will be used in dockernics when creating docker container
  • subnet/netmask@gateway: the network which the IP address of docker container running on the docker host must be located in. If nicname is specified, the subnet/netmask must be the subnet of the nic nicname located in. And gateway shall be the IP address of the nic nicname.
  • nicname: the physical nic name which will be attached to the network object

For example, a network object mynet0 with subnet and gateway on nic eth0 can be created with the command:

chdef host01 -p postbootscripts="setupdockerhost mynet0="

Start OS provisioning for dockerhost

Reference Initialize the Compute for Deployment for how to finish an OS deployment.

Docker instance management

After the dockerhost is ready, a docker instance can be managed through xCAT commands. In xCAT, a docker instance is represented by a node whose definition can be like this:

# lsdef host01c01
Object name: host01c01

The command mkdef or chdef can be used to create a new docker instance node or change the node attributes. Specify any available unused ip address for ip attribute. mac attribute is optional and if left unset, will be filled in by mkdocker command.

After docker instance node is defined, use command makehosts host01c01 to add node host01c01 and its IP address into /etc/hosts.

Create docker instance

mkdocker <node> [image=<image_name>  [command=<command>] [dockerflag=<docker_flags>]]
  • node - The node object which represents the docker instance
  • image - The image name that the docker instance will use
  • command - The command that the docker will run
  • dockerflag - A JSON string which will be used as parameters to create a docker. Reference docker API v1.22 for more information about which parameters can be specified for “dockerflag”.

To create the docker instance host01c01 with image ubuntu and command /bin/bash, use:

mkdocker host01c01 image=ubuntu command=/bin/bash dockerflag="{\"AttachStdin\":true,\"AttachStdout\":true,\"AttachStderr\":true,\"OpenStdin\":true}"

Remove docker instance

rmdocker <node>

The command rmdocker host01c01 can be used to remove the docker instance host01c01.

List docker information

lsdocker <dockerhost|node> [-l|--logs]

To list all the running docker instances on the dockerhost host01, use lsdocker host01.

To list the info of docker instance host01c01, use lsdocker host01c01.

To get log info of docker instance host01c01, use lsdocker host01c01 –logs.

Start docker instance

rpower <node> start

Stop docker instance

rpower <node> stop

Restart docker instance

rpower <node> restart

Pause all processes within a docker instance

rpower <node> pause

Unpause all processes within a docker instance

rpower <node> unpause

Check docker instance status

rpower <node> state


If things go wrong:

  • After dockerhost node boots, check contents of /var/log/xcat/xcat.log file on the dockerhost for errors.
  • Verify nicname specified in Preparing setup trust connection for docker service and create docker network object section exists on the docker host. Depending on the version of Ubuntu OS and host architecture, it could be eth0, or em1, or eno1, or enp0s1. Verify by running on the dockerhost
ip addr show dev <nicname>
  • Run ps -ef | grep docker to verify docker engine is running with configured options. It should look something like
root      3703     1  0 Apr15 ?        00:12:28 /usr/bin/docker daemon -H unix:///var/run/docker.sock -H tcp://host01:2375 --tls --tlscacert=/root/.docker/ca-cert.pem --tlscert=/root/.docker/dockerhost-cert.pem --tlskey=/root/.docker/dockerhost-cert.pem --tlsverify=true --raw-logs

If the output is missing some options, verify that file /lib/systemd/system/docker.service contains the following lines

ExecStart=/usr/bin/docker daemon $DOCKER_OPTS -H fd://