Configure xCAT

After you installed xCAT packages on management node,you have to configure the management node first .This document introduces how to configure the environment well before you can use xCAT normally.

Here is a summary of steps required for the xCAT management node .

1.Check Site Table
2.Check Networks
3.Configure Password Table
3.Initialize DHCP

Check Site Table

After xCAT is installed , site table should be checked. Please verify following attributes and make sure they are correctly set.

domain: The DNS domain name (exg. cluster.com).

nameservers: A comma delimited list of DNS servers that each node in this network should use. This value will end up in the nameserver settings of the /etc/resolv.conf on each node in this network. If this attribute value is set to the IP address of an xCAT node, make sure DNS is running on it. In a hierarchical cluster, you can also set this attribute to "<xcatmaster>" to mean the DNS server for each node in this network should be the node that is managing it (either its service node or the management node). Used in creating the DHCP network definition, and DNS configuration.

forwarders: The DNS servers at your site that can provide names outside of the cluster. The makedns command will configure the DNS on the management node to forward requests it does not know to these servers.Note that the DNS servers on the service nodes will ignore this value and always be configured to forward requests to the management node.

master: The hostname of the xCAT management node, as known by the nodes.

  1. Before xCAT build is installed, management HostName and management DomainName should be configured in DNS configure file /etc/resolv.conf, after xCAT is installed, nameserver, master, domain and forwarder will be set correctly in site table.

    1.1.Before install xCAT:

    • Modify resolv.conf file like example1:

      cat /etc/resolv.conf
; generated by /usr/sbin/dhclient-script
search <xcat_dom>
nameserver <Management_Node_Ip>
nameserver <Forwarder_ip>

    • Configure hostname setting so that using hostname could get the machine hostname like example2:

      mn:~ # hostname
mn

    • Configure domain name setting so that using hostname -d could get domain name like example3:

      [root@mn ~]# hostname -d
pok.stglabs.ibm.com

    1.2 After xCAT is installed:

    • Using tabdump site to check site table, the outputs will like example4:

      "domain","<xcat_dom>",,
"forwarders","<Forwarder_ip>",,
"master","<Management_Node_Ip>",,
"nameservers","<Management_Node_Ip>",,

  2. If configures in above 1 are not configured before xCAT is installed, the outputs of tabdump site are as following:

    "domain"," ",,
"forwarders",,,
"master","NORESOLUTION",,
"nameservers","NORESOLUTION"
  • In this situation, please configure the /etc/resolv.conf file according to example1. Then using chdef (exg. chdef -t site master=<management_node_ip> ) or tabedit site command to configure the site table according to example4.

3.After site table is configured

  • Please initialize DNS using:

    makedns -n

  • Verify DNS work well using:

    nslookup <Mangement_Node_Hostname>

  • It gives out the Management node hostname and resolved ip. Here is an example:

    c910f04x27v05:~ # nslookup c910f04x27v05
Server:         10.4.27.5
Address:        10.4.27.5#53
Name:   c910f04x27v05.pok.stglabs.ibm.com
Address: 10.4.27.5

Note:

  1. The value of attribute master in site table can be set either management node ip or service node ip.
  2. Setting up name resolution and having the nodes resolved to IP addresses are required in xCAT clusters .
  3. Set site.forwarders to your site-wide DNS servers that can resolve site or public hostnames. The DNS on the MN will forward any requests it can’t answer to these servers.
  4. For more dns explanation please refer to dns_label

Check Networks

Please check networks tables:

tabdump networks

The outputs are as following:

#netname,net,mask,mgtifname,gateway,dhcpserver,tftpserver,nameservers,ntpservers,logservers,dynamicrange,staticrange,staticrangeincrement,nodehostname,ddnsdomain,vlanid,domain,comments,disable
"10_0_0_0-255_0_0_0","10.0.0.0","255.0.0.0","eth0","10.0.0.103",,"10.4.27.5",,,,,,,,,,,,

Note:Networks table will be set after xCAT is installed using default net,default mask and default gateway.

1.If the cluster-facing NICs were not configured when xCAT was installed, or if there are more networks in the cluster that are only available via the service nodes or compute nodes, users can use such options below to create network definitions (exg.50.3.5.5).

1.1(Optinal) How to configured networks table:

  • Using mkdef to update networks table.

    mkdef -t network -o net1 net=9.114.0.0 mask=255.255.255.224 gateway=9.114.113.254

net The  network address.

mask The network mask.

gateway The network gateway.

  • Or using tabedit to modify networks table.

    Tabedit networks

  • Or using command makenetworks to automatically generate networks table entry.

    makenetworks

1.2.Verify networks table similar like:

# tabdump networks
#netname,net,mask,mgtifname,gateway,dhcpserver,tftpserver,nameservers,ntpservers,logservers,dynamicrange,nodehostname,comments,disable
50_0_0_0-255_0_0_0","50.0.0.0","255.0.0.0","eth1","<xcatmaster>",,"50.3.5.5",,,,,,,,,,,,

Note:Domain and nameservers values must be provided either in the network definiton corresponding to the node or in the site definition.

Configure Password Table

The password should be set in the passwd table that will be assigned to root when the node is installed. You can modify this table using tabedit. To change the default password for root on the nodes, change the system line.

tabedit passwd
#key,username,password,cryptmethod,comments,disable
"system","root","cluster",,,
"hmc","hscroot","ABC123",,,

Or

chtab key=system passwd.username=root passwd.password=cluster

Note:

  1. Currently xCAT puts the root password on the node only during install. It is taken from the passwd table where key=system. The new subcluster support requires a unique password for each subcluster to be installed.

  2. The xCAT database needs to contain the proper authentication working with hmc/blade/ipmi userid and password. Example for passwd set up:

    chtab key=hmc passwd.username=hscroot passwd.password=abc123
or
chtab key=blade passwd.username=USERID passwd.password=PASSW0RD
or
chtab key=ipmi passwd.username=USERID passwd.password=PASSW0RD

  3. (Optional)If the BPA passwords are still the factory defaults, you must change them before running any other commands to them.

    rspconfig frame general_passwd=general,<newpd>
rspconfig frame admin_passwd=admin,<newpd>
rspconfig frame HMC_passwd=,<newpd>

  4. (Optional)The username and password for xCAT to access an HMC can also be assigned directly to the HMC node object using the mkdef or chdef commands. This assignment is useful when a specific HMC has a username and/or password that is different from the default one specified in the passwd table. For example, to create an HMC node object and set a unique username or password for it:

    mkdef -t node -o hmc1 groups=hmc,all nodetype=ppc hwtype=hmc mgt=hmc username=hscroot password=abc1234

Or to change it if the HMC definition already exists:

chdef -t node -o hmc1 username=hscroot password=abc1234

  5. (Optional)The REST API calls need to provide a username and password. When this request is passed to xcatd, it will first verify that this user/pw is in the xCAT passwd table, and then xcatd will look in the policy table to see if that user is allowed to do the requested operation.

    • The account which key is xcat will be used for the REST API authentication. The username and password should be passed in with the attirbutes.

      userName: Pass the username of the account
userPW: Pass the password of the account

    • Use non-root account to create new user and setup the password and policy rules.

      useradd wsuser
passwd wsuser     # set the password
tabch key=xcat,username=wsuser passwd.password=cluster
mkdef -t policy 6 name=wsuser rule=allow

    • Use root account:

      tabch key=xcat,username=root passwd.password=<root-pw>

Initialize DHCP

Initialize DHCP service

Create a new dhcp configuration file with a network statement for each network the dhcp daemon should listen on.

makedhcp -n

(Optional)Setup the DHCP interfaces in site table

To set up the site table dhcp interfaces for your system p cluster, identify the correct interfaces that xCAT should listen to on your management node and service nodes.

chdef -t site dhcpinterfaces='pmanagenode|eth1;service|eth0'

makedhcp -n

dhcpinterfaces:  The network interfaces DHCP should listen on.  If it is the same for all nodes, use a simple comma-separated list of NICs.  To specify different NICs for different nodes:xcatmn|eth1,eth2;service|bond0.In this example xcatmn is the name of the xCAT MN, and DHCP there should listen on eth1 and eth2.  On all of the nodes in group 'service' DHCP should listen on the bond0 nic.

Note:To verify makedhcp work well, please check nic,domain-name ,domain-servers in dhcpd.conf , for example:

shared-network nic {
subnet 10.0.0.0 netmask 255.0.0.0 {
authoritative;
max-lease-time 43200;
min-lease-time 43200;
default-lease-time 43200;
option routers  10.2.1.12;
next-server  10.2.1.13;
option log-servers <Management_Node_Ip>;
option ntp-servers <Management_Node_Ip>;
option domain-name "<xcat_dom>";
option domain-name-servers  <Management_Node_Ip>;
option domain-search  "pok.stglabs.ibm.com";
zone pok.stglabs.ibm.com. {
primary 10.2.1.13; key xcat_key;
}