Hardware RAID
Overview
In many new compute machines, disks have been formatted into RAID oriented format in manufacturer, so admin must create raid arrays using these disks manually before provisioning OS. How to configure raid arrays in unattended way for hundreds of machines turns to be a problem.
IBM has offered a tool iprconfig
to configure raid for IBM power machine. To leverage this tool, xCAT enabled it in xCAT genesis.ppc64 so that admin can use genesis.ppc64 to configure RAID arrays.
There are two commands (diskdiscover
and configraid
) shipped in xCAT-genesis-scripts
package to support RAID arrays configuration using iprconfig
, you can use the runcmd
facility to configure raid in the hardware discovery procedure using them, runcmd
is a facility which will be run in xcat genesis system. You can also use separated manual steps to use configraid
in xcat genesis system shell.
diskdiscover : Scan disk devices in xcat genesis system, give out disks and RAID arrays information.
configraid : Delete RAID arrays, create RAID arrays in xcat genesis system.
Following sections show how to use diskdiscover
and configraid
, we assume cn1
is compute node in all examples.
Discovering disk devices
Command diskdiscover
scans disk devices, it can get the overview of disks and RAID arrays information from compute node; The outputs contain useful information for configraid
to configure RAID arrays, user can get pci_id
, pci_slot_name
, disk names
, RAID arrays
and other information from the outputs. It should be ran in xcat genesis system. It can be executed without input parameter or with pci_id, pci_id includes PCI vendor and device ID. For example, power8 SAS adapter pci_id is 1014:034a
, 1014
is vendor info, 034a
is PCI-E IPR SAS Adapter, more info about pci_id refer to http://pci-ids.ucw.cz/read/PC/1014/
.
Here are steps to use diskdiscover
:
Start xCAT genesis system in compute node, let compute node
cn1
enter xCAT genesis system shell:nodeset cn1 shell rpower cn1 reset
Note
: If user modifydiskdiscover
orconfigraid
scripts, he needs to run themknb <arch>
command beforenodeset
command to update network boot root image.On xcat management node, executing
xdsh
to usediskdiscover
:xdsh cn1 diskdiscover
Or:
xdsh cn1 'diskdiscover <pci_id>'
The outputs format is as following:
# xdsh cn1 diskdiscover cn1: -------------------------------------------------------------------------- cn1: PCI_ID PCI_SLOT_NAME Resource_Path Device Description Status cn1: ------ ------------- ------------- ------ ----------- ---------------- cn1: 1014:034a 0001:08:00.0 0:0:0:0 sg0 Function Disk Active cn1: 1014:034a 0001:08:00.0 0:0:1:0 sg1 0 Array Member Active cn1: ------------------------------------------------------------------- cn1: Get ipr RAID arrays by PCI_SLOT_NAME: 0001:08:00.0 cn1: ------------------------------------------------------------------- cn1: Name PCI/SCSI Location Description Status cn1: ------ ------------------------- ------------------------- ----------------- cn1: sda 0001:08:00.0/0:2:0:0 RAID 0 Disk Array Optimized
Configuring hardware RAID
Command configraid introduction
We can use configraid
to delete RAID arrays or create RAID arrays:
configraid delete_raid=[all|"<raid_array_list>"|null]
stripe_size=[16|64|256]
create_raid="rl#<raidlevel>|[pci_id#<num>|pci_slot_name#<pci_slot_name>|disk_names#<sg0>#..#<sgn>]|disk_num#<number>" ...
Here are the input parameters introduction:
delete_raid : List raid arrays which should be removed.
If its value is all, all raid arrays detected should be deleted.
If its value is a list of raid array names, these raid arrays will be deleted. Raid array names should be separated by
#
.If its value is null or there is no delete_raid, no raid array will be deleted.
If there is no delete_raid, the default value is null.
stripe_size : It is optional used when creating RAID arrays. If stripe size is not specified, it will default to the recommended stripe size for the selected RAID level.
create_raid : To create a raid array, add a line beginning with create_raid, all attributes keys and values are separated by
#
. The formats are as followings:rl
means RAID level, RAID level can be any supported RAID level for the given adapter, such as 0, 10, 5, 6.rl
is a mandatory attribute for every create_raid. Supported RAID level is depend on physical server’s RAID adapter.User can select disks based on following attributes value. User can find these value based on
diskdiscover
outputs as above section described.pci_id
is PCI vendor and device ID.pci_slot_name
is the specified PCI location. If usingpci_slot_name
, this RAID array will be created using disks from it.disk_names
is a list of advanced format disk names. If usingdisk_names
, this RAID array will be created using these disks.
disk_num
is the number of disks this RAID array will contain, default value is all unused disks in its pci slot.
More examples of input parameters:
Delete all original RAID arrays, create one RAID 10 array from pci_id
1014:034a
, it uses the first two available disks:delete_raid=all create_raid="rl#10|pci_id#1014:034a|disk_num#2"Delete original RAID arrays sda and sdb on compute node, create one RAID 0 array from pci slot 0001:08:00.0, its RAID level is 0, it uses first two disks:
delete_raid="sda#sdb" create_raid="rl#0|pci_slot_name#0001:08:00.0|disk_num#2"Create one RAID array from pci_id
1014:034a
, RAID level is 0, stripe_size is 256kb, using first two available disks:stripe_size=256 create_raid="rl#0|pci_id#1014:034a|disk_num#2"Create two RAID arrays, RAID level is 0, one array uses one disks from pci_id 1014:034a, the other array uses two disks from pci_slot_name
0001:08:00.0
:create_raid="rl#0|pci_id#1014:034a|disk_num#1" create_raid="rl#0|pci_slot_name#0001:08:00.0|disk_num#2"Create two RAID arrays, RAID level is 0, one array uses disks sg0 and sg1, the other array uses disks sg2 and sg3:
create_raid="rl#0|disk_names#sg0#sg1" create_raid="rl#0|disk_names#sg2#sg3"
Configuring RAID arrays process
Command configraid
is running in xcat genesis system, its log is saved under /tmp
on compute node genesis system.
Configuring RAID in hardware discovery procedure
Using
runcmd
facility to configure raid in the hardware discovery procedure, after configuring RAID, compute node enter xcat genesis system shell. In the following example,configraid
deletes all original RAID arrays, it creates one RAID 0 array with first two disks from pci_id1014:034a
:nodeset cn1 runcmd="configraid delete_raid=all create_raid=rl#0|pci_id#1014:034a|disk_num#2",shell rpower cn1 reset
Using
rcons
to monitor the process:rcons cn1
Configuring RAID manually in xcat genesis system shell
Starting xCAT genesis system in compute node, let compute node
cn1
enter xCAT genesis system shell:nodeset cn1 shell rpower cn1 reset
On xcat management node, executing
xdsh
to useconfigraid
to configure RAID:xdsh cn1 'configraid delete_raid=all create_raid="rl#0|pci_id#1014:034a|disk_num#2"'
Monitoring and debuging RAID configuration process
Creating some RAID level arrays take very long time, for example, If user creates RAID 10, it will cost tens of minutes or hours. During this period, you can use xCAT xdsh command to monitor the progress of raid configuration.
xdsh cn1 iprconfig -c show-config
Logs for
configraid
is saved undertmp
in compute node genesis system. User can login compute node and checkconfigraid
logs to debug.When configuring RAID in hardware discovery procedure, user can use
rcons
command to monitor or debug the process:rcons cn1