ThinkVirt

As a virtualization consultant, you hear the complaint that "the storage is slow" from nearly every customer. To diagnose this problem, first you start with learning the configuration of the storage network and then move on to lower levels. As a starting point, I glance over the highest disk latencies statistics per host in the VI client and the latency results from esxtop.

If nothing stands out from those statistics, I audit the storage configuration to validate what is documented; are all of the paths really up, is the correct Path Selection Policy and Storage Array Type chosen, is there an over-subscription of the storage links, and at what speed are those links operating?

A combination of the VI client and esxtop will answer all of those questions for Fibre Channel, iSCSI, and NFS storage networks. Well, it will answer all questions the last for Fibre Channel.

The VI client does not have a location (or at least one that is easily accessible) where you could see the Fibre Channel link speed. Instead, you will need to grab an SSH client and get into the Service Console.

Once you're in the Service Console, change directory to "/proc/scsi" and list the contents. In my case, the directory contents are -

[root@ESXBL1847 ~]# cd /proc/scsi/
[root@ESXBL1847 scsi]# ls
device_info hpsa lpfc820 scsi sg

As I am using the "Emulex LightPulse" line of HBAs (Host Bus Adapters), the directory I would go into is "lpfc820."

Within that directory, we have the following contents -

[root@ESXBL1847 lpfc820]# ls
1 2

The files "1" and "2" correspond to the two HBAs installed in the host. Within these files is the information on the link such port and node WWNs (World Wide Names) for the host and the target device along with link state and speed.

The results for the HBA 1 -

Emulex LightPulse Fibre Channel SCSI driver 8.2.0.30.52vmw-3
HP BLc Emulex LPe1105-HP FC Mezz Option Kit on PCI bus 06 device 00 irq 177 port 0
BoardNum: 0
Firmware Version: 2.72A2 (Z3F2.72A2)
Portname: 00:00:00:00:00:00:00:00 Nodename: 00:00:00:00:00:00:00:00

SLI Rev: 3
NPIV Unsupported by Fabric
RPIs max 512 RPIs used 5

Vport List:

Link Up - Ready:
PortID 0xc30900
Fabric
Current speed 2G

Physical Port Discovered Nodes: Count 1
t00 DID 440009 State 06 WWPN 00:00:00:00:00:00:00:00 WWNN 00:00:00:00:00:00:00:00

As we can see, the speed this link is operating on is 2 Gb/s. The results for HBA 2 -

Emulex LightPulse Fibre Channel SCSI driver 8.2.0.30.52vmw-3
HP BLc Emulex LPe1105-HP FC Mezz Option Kit on PCI bus 06 device 01 irq 185 port 1
BoardNum: 1
Firmware Version: 2.72A2 (Z3F2.72A2)
Portname: 00:00:00:00:00:00:00:00 Nodename: 00:00:00:00:00:00:00:00

SLI Rev: 3
NPIV Unsupported by Fabric
RPIs max 512 RPIs used 1

Vport List:

Link Down

As we can see, this link is down which means a good find if this result was unexpected.

Note: WWNs changed to protect the innocent. For QLogic HBAs, the output will be different, although the link state and speed will be identified.