Trouble Shooting

VNC client complains the credentials are not valid

Issue:

While connecting to the hypervisor with VNC, the vnc client complains “Authentication failed”.

Solution:

Check whether the clocks on the hypervisor and headnode are synced

rpower fails with “Error: internal error Process exited while reading console log qemu: Permission denied”

Issue:

#rpower vm1 on
vm1: Error: internal error Process exited while reading console log output: char device redirected to /dev/pts/1
qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vm1.sda.qcow2: Permission denied: internal error Process exited while reading console log output: char device redirected to /dev/pts/1
qemu: could not open disk image /var/lib/xcat/pools/2e66895a-e09a-53d5-74d3-eccdd9746eb5/vm1.sda.qcow2: Permission denied

Solution:

Usually caused by incorrect permission in NFS server/client configuration. NFSv4 is enabled in some Linux distributions such as CentOS6 by default. The solution is simply to disable NFSv4 support on the NFS server by uncommenting the following line in “/etc/sysconfig/nfs”:

RPCNFSDARGS="-N 4"

Then restart the NFS services and try to power on the VM again…

Note: For stateless hypervisor, purge the VM by rmvm -p vm1, reboot the hypervisor and then create the VM.

rpower fails with “Error: internal error: process exited while connecting to monitor qemu: Permission denied”

Issue:

#rpower vm1 on
vm1: Error: internal error: process exited while connecting to monitor: 2016-02-03T08:28:54.104601Z qemu-system-ppc64: -drive file=/var/lib/xcat/pools/c7953a80-89ca-53c7-64fb-2dcfc549bd45/vm1.sda.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none: Could not open '/var/lib/xcat/pools/c7953a80-89ca-53c7-64fb-2dcfc549bd45/vm1.sda.qcow2': Permission denied

Solution:

Usually caused by SELinux policies. The solution is simply to disable SELinux on the vmhost/hypervisor by editing “/etc/selinux/config” and change the SELINUX line to SELINUX=disabled:

SELINUX=disabled

Then reboot the hypervisor…

rmigrate fails with “Error: libvirt error code: 38, message: unable to connect to server at ‘c910f05c35:49152’: No route to host.”

Issue:

#rmigrate vm1 kvmhost2
vm1: Error: libvirt error code: 38, message: unable to connect to server at 'kvmhost2:49152': No route to host: Failed migration of vm1 from kvmhost1 to kvmhost2

Solution:

Usually caused by active firewall. To disable the firewall issue:

systemctl disable firewalld

rmigrate fails with “Error: 38, message: failed to create directory ‘<dir-name>’: File exists: Unknown issue libvirt error code.”

Issue:

#rmigrate vm1 kvmhost2
vm1: Error: 38, message: failed to create directory '<dir-name>': File exists: Unknown issue libvirt error code.

Solution:

Usually happens when nfs: is specified for vmstorage attribute but that NFS directory is no longer mounted. Make sure the directory /var/lib/xcat/pools is empty on the destination kvmhost.

Error: Cannot communicate via libvirt to kvmhost1

Issue:

The kvm related commands complain “Error: Cannot communicate via libvirt to kvmhost1”

Solution:

Usually caused by incorrect ssh configuration between xCAT management node and hypervisor. Make sure it is possible to access the hypervisor from management node via ssh without password.

Fail to ping the installed VM

Issue:

The newly installed stateful VM node is not pingable, the following message can be observed in the console during VM booting:

ADDRCONF(NETDEV_UP): eth0 link is not ready.

Solution:

Usually caused by the incorrect VM NIC model. Try the following steps to specify “virtio”:

rmvm vm1
chdef vm1 vmnicnicmodel=virtio
mkvm vm1