Configuring VMware ESX 5.5 from the command line

Over the past few months I have spent a lot of time building a large ESX and nested ESX infrastructure based on VMware ESX 5.5. As you do this, you quickly realize that configuring ESX from the UI is painful, especially when you need to make sure you have all of the ESX servers exactly the same.  Here are some tips and tricks that I have found to be very helpful.

1. Enable SSH on your ESX server and setup certificate based authentication.  This will greatly ease your work as you can then pipe configuration commands through SSH, which in turn allows you to script the whole configuration (and yes, this all works with the free version of ESXi).  I can now perform all of the configuration for 100+ ESX servers in a few seconds. On the ESX server the public keys for your Linux servers go in a file called:
   /etc/ssh/keys-root/authorized_keys
    
2. I also like to change the motd, shell profile, and ntp.conf at the same time.  I just copy these files over.  The shell profile goes in a file called: /etc/profile.local
   
   My profile.local files looks like this:
   # profile.local
   
   PS1="[\u@\h]:\w-> "
   export PS1
   
   if [ "$TERM" != "dumb" ]; then
       alias ls='ls --color=auto'
       alias ll='ls -l -a --color=auto'
   fi
   
   
3. Configure DNS and Hostname settings
   ssh root@x.x.x.x "esxcli network ip dns server add --server=192.168.0.11"
   ssh root@x.x.x.x "esxcli network ip dns server add --server=192.168.0.11"
   ssh root@x.x.x.x "esxcli system hostname set --host=esxserver01"
   ssh root@x.x.x.x "esxcli system hostname set --domain=mydomain.com"
    
4. Configure NTP Settings
   Copy over a valid ntp.conf file to /etc/ntp.conf
   ssh root@x.x.x.x "esxcli network firewall ruleset set --enabled=true --ruleset-id=ntpClient"
   ssh root@x.x.x.x "chkconfig --add ntpd"
    
5. License ESX
   ssh root@x.x.x.x "vim-cmd vimsvc/license --set xxxxx-xxxxx-xxxxx-xxxxx-xxxx"
    
6. Setup any networking you need.  For my setup, I need to rename the first port group and create a new vswitch with a port group.  You also need to change the failover state as it defaults to non active.  This is how I did that.
   ssh root@x.x.x.x "esxcli network vswitch standard portgroup remove -p \'VM Network\' -v vSwitch0"
   ssh root@x.x.x.x "esxcli network vswitch standard portgroup add -p \'Trusted Network\' -v vSwitch0"
   ssh root@x.x.x.x "esxcli network vswitch standard add -v vSwitch1"
   ssh root@x.x.x.x "esxcli network vswitch standard portgroup add -p \'Client Network\' -v vSwitch1"
   ssh root@x.x.x.x "esxcli network vswitch standard uplink add -u vmnic1 -v vSwitch1"
   ssh root@x.x.x.x "esxcli network vswitch standard policy failover set -a vmnic1 -v vSwitch1"
    
7. Reboot ESX server so all change take effect
   ssh root@x.x.x.x "reboot"
    

As you can see, once you setup certificate based authentication, you could easily script the above commands in bash, perl, python, etc and configure all of you ESX servers at once.  If you do this, I found that you need to add a sleep for 2 seconds statement between setting the DNS hostname and setting the DNS domain. 

Upgrading ESXi from 5.1 to 5.5

I started upgrading my ESX servers tonight and here is the simple version of how to do that. Now if this was VMware's official howto documentation you would be on page 243 by now and would still need to read to page 500 before you got through it all.  Tech writers should not be paid by the word.

1) Shutdown all running VMs

2) Put ESX in maintenance mode:
> vim-cmd /hostsvc/maintenance_mode_enter

3) Enable outbound HTTP client:
> esxcli network firewall ruleset set -e true -r httpClient

4) List avaliable updates.  You want the standard one and it can takes FOREVER for it to complete, say more than 5 minutes with no feedback:
> esxcli software sources profile list -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml | grep ESXi-5.5

5) Perform the upgrade:
> esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-5.5.0-1331820-standard

6) Reboot

7) Exit maintenance mode:
> vim-cmd /hostsvc/maintenance_mode_exit

ESX and LSI MegaRAID

If you have an LSI RAID controller in your ESX host and you would like the LSI RAID health information to show up in vSphere, under Health Status -> Storage, you will need to install the LSI VMware SMIS Provider.  Further, in order to perform CLI RAID commands on the ESX host for the LSI controller, you will need to install the MegaCLI vib package.

I am using the following packages on ESX 5.1 Update 1
VMWare SMIS Provider VIB - MR 5.6
MegaCLI 5.5 P1

And downloaded them from here:
http://www.lsi.com/products/storagecomponents/Pages/MegaRAIDSAS9280-24i4e.aspx

Step 1: Download the vib files from LSI.com.  I found it easier to find them on the product page instead of the LSI download page.

Step 2: scp the vib files over to your ESX host and put them in the /tmp directory

Step 3: SSH to the ESX host and install the vib packages by running the following commands:
/tmp # esxcli software vib install --no-sig-check -v /tmp/vmware-esx-MegaCli-8.07.07.vib
/tmp # esxcli software vib install --no-sig-check -v /tmp/vmware-esx-provider-lsiprovider.vib

Step 4: Shut down any running VMs and Reboot ESX host

Step 5: Power up all your VMs you shut down in Step 4.  NOTE: It will take about 20 minutes for all of the data to show up in the Health Status section.  Also, I had issues with the SMIS Provider randomly stopping on ESX 5.1, and thus upgraded to 5.1 Update 1.

Some useful MegaCLI commands are:
cd /opt/lsi/MegaCLI

Controller information
./MegaCli -AdpAllInfo -aALL
./MegaCli -CfgDsply -aALL

Enclosure information
./MegaCli -EncInfo -aALL

Virtual drive information
./MegaCli -LDInfo -Lall -aALL

Physical drive information
./MegaCli -PDList -aALL

NOTE: If you want to run the MegaRAID Storage Manager from your Windows/Linux system and have it manage the RAID controller in your ESX host, you need to be in the same Layer2 VLAN.  The software uses multicast to find controllers that it can support.  I also found that if your systems are not in DNS, and just have IP addresses, then you MUST add entries for them in the /etc/hosts file or /Windows/system32/drivers/etc/hosts file. If you do not do this then the software will get confused and refer to every device it finds as a NULL string.

Upgrading ESX 5.1 to 5.1 Update 1

A week or so ago, VMware released 5.1 Update 1 (Build Number: 1065491).  This post will show you how to quickly and easily upgrade your system.  If you read VMware's documentation and are still confused, than this should help you.  It is just too bad that VMware has to make everything more complicated than it needs to be.

Step 1: Download zip bundle (update-from-esxi5.1-5.1_update01.zip ) not iso from VMware's download site: http://www.vmware.com/patchmgr/download.portal When you get to the portal, select "ESXi (Embedded and Installable) from the dropdown. 


Step 2: Copy (scp) the update zip file to your datastore (/vmfs/volumes/datastore1)

Step 3: Shut down all running VMs on the host you want to upgrade

Step 4: SSH to the ESX host you are going to upgrade and run the upgrade command
esxcli software vib install --depot /vmfs/volumes/datastore1/update-from-esxi5.1-5.1_update01.zip

Step 5: When the upgrade finishes reboot the ESX host

Step 6: Start all of the VMs that you shutdown in Step 3

Step 7: Upgrade the vmware-tools for each VM that had older versions

Step 8: Upgrade your vSphere application