Upgrade VMWare vSphere 5.5 to Update 3b

To upgrade your ESX 5.5 server to 5.5 Update 3b, you will need to do the following:

1) Shutdown all running VMs

2) Put ESX in maintenance mode
> vim-cmd /hostsvc/maintenance_mode_enter

3) Enable the outbound HTTP client through the ESX firewall
> esxcli network firewall ruleset set -e true -r httpClient

4) Perform the upgrade to 5.5 Update 3b.  As always you can get information about this update or other updates on the VMWare knowledge base here and here
> esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-5.5.0-20151204001-standard

5) Reboot the ESX Server

6) List available updates that came out after the Update3b.  You want the standard one and it can take FOREVER for it to complete, say more than 5-10 minutes with no feedback.  You may see versions that include an "s" at the end of the filename, these profiles only contain security patches, not other bug fixes.  These security only versions would be for those organizations that are restricted by their change control board from patching anything but security fixes.
> esxcli software sources profile list -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml | grep ESXi-5.5 | grep standard | grep 2016

7) You will see from this list that there is another update, as of today.  It is called ESXi-5.5.0-20160104001-standard.  If you wanted more information about this update, you can go to the VMWare Knowledge base and search on "vmware esxi-5.5, patch ESXi-5.5.0-20160104001"

8) Perform this update as well
> esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-5.5.0-20160104001-standard

9) Reboot the ESX Server

7) Exit maintenance mode:
> vim-cmd /hostsvc/maintenance_mode_exit

How to reset the ADMIN password on a Supermicro IPMI device under VMware ESX 5.5

Resetting a forgotten ADMIN password on your Supermicro IPMI device when you are running VMware ESX 5.x is super easy, just follow these simple steps:

1) Download the IPMI tools from Supermicro’s site at: ftp://ftp.supermicro.com/utility/IPMICFG/

2) Extract zip archive and scp the Linux 64bit files over to your ESX server, either in /tmp or make the directory /opt/supermicro and store them there. 

3) Change execute permissions on ipmicfg-linux.x86_64.static to RX

> chmod 750 ipmicfg-linux.x86_64.static

4) Run the following command:

> ./ipmicfg-linux.x86_64.static -user setpwd 2 somenewpassword

Manaully reset VMware VMID values

In VMware ESX if you type the following command at the shell, you will see a list of VMs installed/registered to the ESX host, along with their under-the-covers ID value called the VMID.

[root@esxcore01]:~-> vim-cmd vmsvc/getallvms

Now every time you create a new VM the system will take the next VMID that is not currently being used, however, it does not recycle any of the VMIDs.  Meaning, if you delete a VM that was using VMID = 5, the next VM you create will be VMID 6, then 7, and so forth. 

In my case where I create and delete hundreds of VMs on a daily basis this VMID counter can get really large.  So I got wondering, is there was a way to reset the "working/next" VMID?

What I found is:   

[root@esxcore01]: /etc/vmware/hostd/vmInventory.xml

contains a list of all of the registered VMs on the host, but it calls VMID .  The is also found in: 

[root@esxcore01]: /etc/vmware/hostd/pools.xml

But neither of these helped my issue.  The work around solution I found is the following. Say for example you have 4 Guests:

Server 1 = VMID 1
Server 2 = VMID 2
Server 3 = VMID 11
Server 4 = VMID 20

If you remove server 3 and 4 from your inventory, power down server 1 and 2 and then reboot the ESX server, the VMID table will reset.  So once the ESX server/host comes back up, you can re-add server 3 and 4 back to the inventory and they will have VMID 3 and VMID 4 respectfully.

Failed to connect to VMware lookup service

Over the weekend I had to make some networking changes to my lab. During this process I had to re-IP all of my ESX hosts. Upon completion I brought vCenter 5.5 back online and changed the IP address to the new management network. However, when I tried to log in to the vCenter web-client I I got the following error message, "Failed to connect to VMware Lookup Service"



To solve this issue, do the following:

1. Login into vCenter VA Configuration https://[hostname]:5480
2. Select the Admin tab
3. Click the checkbox to toggle the certificate setting, you will see certificate regeneration enabled change to Yes.
4. Re-boot the Virtual Appliance
5. Force a refresh of your browser to get the new certificate and login into vCenter VA Configuration https://[hostname]:5480 
6. Deselect the checkbox in step 3. 
7. You should now be able to log in with the web-client, just make sure you force reload the browser to get the new certificate. 

Upgrade ESX 5.5 to 5.5 Update 2

See my previous post (here) for more details. 

1) Shutdown all running VMs

2) Put ESX in maintenance mode:
> vim-cmd /hostsvc/maintenance_mode_enter

3) Perform the upgrade:
> esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-5.5.0-20140902001-standard

4) Reboot

5) Exit maintenance mode:
> vim-cmd /hostsvc/maintenance_mode_exit

Configuring VMware ESX 5.5 from the command line

Over the past few months I have spent a lot of time building a large ESX and nested ESX infrastructure based on VMware ESX 5.5. As you do this, you quickly realize that configuring ESX from the UI is painful, especially when you need to make sure you have all of the ESX servers exactly the same.  Here are some tips and tricks that I have found to be very helpful.

1. Enable SSH on your ESX server and setup certificate based authentication.  This will greatly ease your work as you can then pipe configuration commands through SSH, which in turn allows you to script the whole configuration (and yes, this all works with the free version of ESXi).  I can now perform all of the configuration for 100+ ESX servers in a few seconds. On the ESX server the public keys for your Linux servers go in a file called:
   /etc/ssh/keys-root/authorized_keys
    
2. I also like to change the motd, shell profile, and ntp.conf at the same time.  I just copy these files over.  The shell profile goes in a file called: /etc/profile.local
   
   My profile.local files looks like this:
   # profile.local
   
   PS1="[\u@\h]:\w-> "
   export PS1
   
   if [ "$TERM" != "dumb" ]; then
       alias ls='ls --color=auto'
       alias ll='ls -l -a --color=auto'
   fi
   
   
3. Configure DNS and Hostname settings
   ssh root@x.x.x.x "esxcli network ip dns server add --server=192.168.0.11"
   ssh root@x.x.x.x "esxcli network ip dns server add --server=192.168.0.11"
   ssh root@x.x.x.x "esxcli system hostname set --host=esxserver01"
   ssh root@x.x.x.x "esxcli system hostname set --domain=mydomain.com"
    
4. Configure NTP Settings
   Copy over a valid ntp.conf file to /etc/ntp.conf
   ssh root@x.x.x.x "esxcli network firewall ruleset set --enabled=true --ruleset-id=ntpClient"
   ssh root@x.x.x.x "chkconfig --add ntpd"
    
5. License ESX
   ssh root@x.x.x.x "vim-cmd vimsvc/license --set xxxxx-xxxxx-xxxxx-xxxxx-xxxx"
    
6. Setup any networking you need.  For my setup, I need to rename the first port group and create a new vswitch with a port group.  You also need to change the failover state as it defaults to non active.  This is how I did that.
   ssh root@x.x.x.x "esxcli network vswitch standard portgroup remove -p \'VM Network\' -v vSwitch0"
   ssh root@x.x.x.x "esxcli network vswitch standard portgroup add -p \'Trusted Network\' -v vSwitch0"
   ssh root@x.x.x.x "esxcli network vswitch standard add -v vSwitch1"
   ssh root@x.x.x.x "esxcli network vswitch standard portgroup add -p \'Client Network\' -v vSwitch1"
   ssh root@x.x.x.x "esxcli network vswitch standard uplink add -u vmnic1 -v vSwitch1"
   ssh root@x.x.x.x "esxcli network vswitch standard policy failover set -a vmnic1 -v vSwitch1"
    
7. Reboot ESX server so all change take effect
   ssh root@x.x.x.x "reboot"
    

As you can see, once you setup certificate based authentication, you could easily script the above commands in bash, perl, python, etc and configure all of you ESX servers at once.  If you do this, I found that you need to add a sleep for 2 seconds statement between setting the DNS hostname and setting the DNS domain. 

Upgrading ESXi from 5.1 to 5.5

I started upgrading my ESX servers tonight and here is the simple version of how to do that. Now if this was VMware's official howto documentation you would be on page 243 by now and would still need to read to page 500 before you got through it all.  Tech writers should not be paid by the word.

1) Shutdown all running VMs

2) Put ESX in maintenance mode:
> vim-cmd /hostsvc/maintenance_mode_enter

3) Enable outbound HTTP client:
> esxcli network firewall ruleset set -e true -r httpClient

4) List avaliable updates.  You want the standard one and it can takes FOREVER for it to complete, say more than 5 minutes with no feedback:
> esxcli software sources profile list -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml | grep ESXi-5.5

5) Perform the upgrade:
> esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-5.5.0-1331820-standard

6) Reboot

7) Exit maintenance mode:
> vim-cmd /hostsvc/maintenance_mode_exit

ESX and LSI MegaRAID

If you have an LSI RAID controller in your ESX host and you would like the LSI RAID health information to show up in vSphere, under Health Status -> Storage, you will need to install the LSI VMware SMIS Provider.  Further, in order to perform CLI RAID commands on the ESX host for the LSI controller, you will need to install the MegaCLI vib package.

I am using the following packages on ESX 5.1 Update 1
VMWare SMIS Provider VIB - MR 5.6
MegaCLI 5.5 P1

And downloaded them from here:
http://www.lsi.com/products/storagecomponents/Pages/MegaRAIDSAS9280-24i4e.aspx

Step 1: Download the vib files from LSI.com.  I found it easier to find them on the product page instead of the LSI download page.

Step 2: scp the vib files over to your ESX host and put them in the /tmp directory

Step 3: SSH to the ESX host and install the vib packages by running the following commands:
/tmp # esxcli software vib install --no-sig-check -v /tmp/vmware-esx-MegaCli-8.07.07.vib
/tmp # esxcli software vib install --no-sig-check -v /tmp/vmware-esx-provider-lsiprovider.vib

Step 4: Shut down any running VMs and Reboot ESX host

Step 5: Power up all your VMs you shut down in Step 4.  NOTE: It will take about 20 minutes for all of the data to show up in the Health Status section.  Also, I had issues with the SMIS Provider randomly stopping on ESX 5.1, and thus upgraded to 5.1 Update 1.

Some useful MegaCLI commands are:
cd /opt/lsi/MegaCLI

Controller information
./MegaCli -AdpAllInfo -aALL
./MegaCli -CfgDsply -aALL

Enclosure information
./MegaCli -EncInfo -aALL

Virtual drive information
./MegaCli -LDInfo -Lall -aALL

Physical drive information
./MegaCli -PDList -aALL

NOTE: If you want to run the MegaRAID Storage Manager from your Windows/Linux system and have it manage the RAID controller in your ESX host, you need to be in the same Layer2 VLAN.  The software uses multicast to find controllers that it can support.  I also found that if your systems are not in DNS, and just have IP addresses, then you MUST add entries for them in the /etc/hosts file or /Windows/system32/drivers/etc/hosts file. If you do not do this then the software will get confused and refer to every device it finds as a NULL string.

Upgrading ESX 5.1 to 5.1 Update 1

A week or so ago, VMware released 5.1 Update 1 (Build Number: 1065491).  This post will show you how to quickly and easily upgrade your system.  If you read VMware's documentation and are still confused, than this should help you.  It is just too bad that VMware has to make everything more complicated than it needs to be.

Step 1: Download zip bundle (update-from-esxi5.1-5.1_update01.zip ) not iso from VMware's download site: http://www.vmware.com/patchmgr/download.portal When you get to the portal, select "ESXi (Embedded and Installable) from the dropdown. 


Step 2: Copy (scp) the update zip file to your datastore (/vmfs/volumes/datastore1)

Step 3: Shut down all running VMs on the host you want to upgrade

Step 4: SSH to the ESX host you are going to upgrade and run the upgrade command
esxcli software vib install --depot /vmfs/volumes/datastore1/update-from-esxi5.1-5.1_update01.zip

Step 5: When the upgrade finishes reboot the ESX host

Step 6: Start all of the VMs that you shutdown in Step 3

Step 7: Upgrade the vmware-tools for each VM that had older versions

Step 8: Upgrade your vSphere application