Switch-based Discovery¶
For switch based hardware discovery, the servers are identified through the switches and switchposts they are directly connected to.
In this document, the following configuration is used in the example
Management Node info:
MN Hostname: xcat1
MN NIC info for Management Network(Host network): eth1, 10.0.1.1/16
MN NIC info for Service Network(FSP/BMC nework): eth2, 50.0.1.1/16
Dynamic IP range for Hosts: 10.0.100.1-10.0.100.100
Dynamic IP range for FSP/BMC: 50.0.100.1-50.0.100.100
Compute Node info:
CN Hostname: cn1
Machine type/model: 8247-22L
Serial: 10112CA
IP Address: 10.0.101.1
Root Password: cluster
Desired FSP/BMC IP Address: 50.0.101.1
DHCP assigned FSP/BMC IP Address: 50.0.100.1
FSP/BMC username: ADMIN
FSP/BMC Password: admin
Switch info:
Switch name: switch1
Switch username: xcat
Switch password: passw0rd
Switch IP Address: 10.0.201.1
Switch port for Compute Node: port0
Configure xCAT¶
Configure network table¶
Normally, there will be at least two entries for the two subnet on MN in networks
table after xCAT is installed:
#tabdump networks
#netname,net,mask,mgtifname,gateway,dhcpserver,tftpserver,nameservers,ntpservers,logservers,dynamicrange,staticrange,staticrangeincrement,nodehostname,ddnsdomain,vlanid,domain,comments,disable
"10_0_0_0-255_255_0_0","10.0.0.0","255.255.0.0","eth1","<xcatmaster>",,"10.0.1.1",,,,,,,,,,,,
"50_0_0_0-255_255_0_0","50.0.0.0","255.255.0.0","eth2","<xcatmaster>",,"50.0.1.1",,,,,,,,,,,,
Run the following command to add networks in networks
table if there are no entries in it:
makenetworks
Setup DHCP¶
Set the correct NIC from which DHCP server provide service:
chdef -t site dhcpinterfaces=eth1,eth2
Add dynamic range in purpose of assigning temporary IP adddress for FSP/BMCs and hosts:
chdef -t network 10_0_0_0-255_255_0_0 dynamicrange="10.0.100.1-10.0.100.100"
chdef -t network 50_0_0_0-255_255_0_0 dynamicrange="50.0.100.1-50.0.100.100"
Update DHCP configuration file:
makedhcp -n
makedhcp -a
Config passwd table¶
Set required passwords for xCAT to do hardware management and/or OS provisioning by adding entries to the xCAT passwd
table:
# tabedit passwd
# key,username,password,cryptmethod,authdomain,comments,disable
For hardware management with ipmi, add the following line:
"ipmi","ADMIN","admin",,,,
Verify the genesis packages¶
Genesis packages are used to create the root image for network boot and MUST be installed before doing hardware discovery.
[RH]:
# rpm -qa |grep -i genesis xCAT-genesis-scripts-ppc64-2.10-snap201507240527.noarch xCAT-genesis-base-ppc64-2.10-snap201505172314.noarch
[ubuntu]:
# dpkg -l | grep genesis ii xcat-genesis-base-ppc64 2.10-snap201505172314 all xCAT Genesis netboot image ii xcat-genesis-scripts 2.10-snap201507240105 ppc64el xCAT genesis
Note: If the two packages are not installed, install them first and then run mknb ppc64
to create the network boot root image.
Predefined Nodes¶
In order to differentiate one node from another, the admin needs to predefine node in xCAT database based on the switches information. This consists of two parts:
- predefined_switches_label
- predefined_server_nodes_label
Predefine Switches
The predefined switches will represent devices that the physical servers are connected to. xCAT need to access those switches to get server related information through SNMP v3.
So the admin need to make sure those switches are configured correctly with SNMP v3 enabled. <TODO: The document that Configure Ethernet Switches>
Then, define switch info into xCAT:
nodeadd switch1 groups=switch,all
chdef switch1 ip=10.0.201.1
tabch switch=switch1 switches.snmpversion=3 switches.username=xcat switches.password=passw0rd switches.auth=sha
Add switch into DNS using the following commands:
makehosts switch1
makedns -n
Predefine Server node
After switches are defined, the server node can be predefined with the following commands:
nodeadd cn1 groups=powerLE,all
chdef cn1 mgt=ipmi cons=ipmi ip=10.0.101.1 bmc=50.0.101.1 netboot=petitboot installnic=mac primarynic=mac
chdef cn1 switch=switch1 switchport=0
In order to do BMC configuration during the discovery process, set runcmd=bmcsetup
. For more info about chain, please refer to Chain
chdef cn1 chain="runcmd=bmcsetup"
Add cn1 into DNS:
makehosts cn1
maekdns -n
Discover server and define¶
After environment is ready, and the server is powered, we can start server discovery process. The first thing to do is discovering the FSP/BMC of the server. It is automatically powered on when the physical server is powered.
The following command can be used to discovery BMC within an IP range and write the discovered node definition into xCAT database:
bmcdiscover -s nmap --range 50.0.100.1-100 -t -z -w
The discovered BMC node will be like this:
# lsdef node-8247-42l-10112ca
Object name: node-8247-42l-10112ca
bmc=50.0.100.1
cons=ipmi
groups=all
hwtype=bmc
mgt=ipmi
mtm=8247-42L
nodetype=mp
postbootscripts=otherpkgs
postscripts=syslog,remoteshell,syncfiles
serial=10112CA
- Note:
The BMC node is just used to control the physical during hardware discovery process, it will be deleted after the correct server node object is found.
bmcdiscover will use username/password pair set in
passwd
table with key equal ipmi. If you’d like to use other username/password pair, you can usebmcdiscover -s nmap --range 50.0.100.1-100 -t -z -w -u <username> -p <password>
Start discovery process¶
To start discovery process, just need to power on the PBMC node remotely with the following command, and the discovery process will start automatically after the host is powered on:
rpower node-8247-42l-10112ca on
[Optional] If you’d like to monitor the discovery process, you can use:
makeconservercf node-8247-42l-10112ca
rcons node-8247-42l-10112ca
Verify node definition¶
The following is an example of the server node definition after hardware discovery:
#lsdef cn1
Object name: cn1
arch=ppc64
bmc=50.0.101.1
cons=ipmi
cpucount=192
cputype=POWER8E (raw), altivec supported
groups=powerLE,all
installnic=mac
ip=10.0.101.1
mac=6c:ae:8b:02:12:50
memory=65118MB
mgt=ipmi
mtm=8247-22L
netboot=petitboot
postbootscripts=otherpkgs
postscripts=syslog,remoteshell,syncfiles
primarynic=mac
serial=10112CA
supportedarchs=ppc64
switch=switch1
switchport=0