Saturday, December 21, 2013

Configuring VMware ESX 5.5 from the command line

Over the past few months I have spent a lot of time building a large ESX and nested ESX infrastructure based on VMware ESX 5.5. As you do this, you quickly realize that configuring ESX from the UI is painful, especially when you need to make sure you have all of the ESX servers exactly the same.  Here are some tips and tricks that I have found to be very helpful.

  1. Enable SSH on your ESX server and setup certificate based authentication.  This will greatly ease your work as you can then pipe configuration commands through SSH, which in turn allows you to script the whole configuration (and yes, this all works with the free version of ESXi).  I can now perform all of the configuration for 100+ ESX servers in a few seconds. On the ESX server the public keys for your Linux servers go in a file called:
    /etc/ssh/keys-root/authorized_keys
     
  2. I also like to change the motd, shell profile, and ntp.conf at the same time.  I just copy these files over.  The shell profile goes in a file called: /etc/profile.local

    My profile.local files looks like this:

    # profile.local

    PS1="[\u@\h]:\w-> "
    export PS1

    if [ "$TERM" != "dumb" ]; then
        alias ls='ls --color=auto'
        alias ll='ls -l -a --color=auto'
    fi

  3. Configure DNS and Hostname settings
    ssh root@x.x.x.x "esxcli network ip dns server add --server=192.168.0.11"

    ssh root@x.x.x.x "esxcli network ip dns server add --server=192.168.0.11"
    ssh root@x.x.x.x "esxcli system hostname set --host=esxserver01"
    ssh root@x.x.x.x "esxcli system hostname set --domain=mydomain.com"
     
  4. Configure NTP Settings
    Copy over a valid ntp.conf file to
    /etc/ntp.conf
    ssh root@x.x.x.x "esxcli network firewall ruleset set --enabled=true --ruleset-id=ntpClient"
    ssh root@x.x.x.x "chkconfig --add ntpd"
     
  5. License ESX
    ssh root@x.x.x.x "vim-cmd vimsvc/license --set xxxxx-xxxxx-xxxxx-xxxxx-xxxx"
     
  6. Setup any networking you need.  For my setup, I need to rename the first port group and create a new vswitch with a port group.  You also need to change the failover state as it defaults to non active.  This is how I did that.
    ssh root@x.x.x.x "esxcli network vswitch standard portgroup remove -p \'VM Network\' -v vSwitch0"
    ssh root@x.x.x.x "esxcli network vswitch standard portgroup add -p \'Trusted Network\' -v vSwitch0"

    ssh root@x.x.x.x "esxcli network vswitch standard add -v vSwitch1"
    ssh root@x.x.x.x "esxcli network vswitch standard portgroup add -p \'Client Network\' -v vSwitch1"
    ssh root@x.x.x.x "esxcli network vswitch standard uplink add -u vmnic1 -v vSwitch1"
    ssh root@x.x.x.x "esxcli network vswitch standard policy failover set -a vmnic1 -v vSwitch1"
     
  7. Reboot ESX server so all change take effect
    ssh root@x.x.x.x "reboot"
     
As you can see, once you setup certificate based authentication, you could easily script the above commands in bash, perl, python, etc and configure all of you ESX servers at once.  If you do this, I found that you need to add a sleep for 2 seconds statement between setting the DNS hostname and setting the DNS domain. 

4 comments:

  1. Is there a way to use ESXCLI to join a domain?

    ReplyDelete
  2. I can't make this NTP config to work... I copied a ntp.conf from a working esxi 5.5 (from my env) to a fresh esxi 5.5 server, used your step above to enable NTP, but it still not working... At least that's what the ESXi client tells me... I also tried to check if changing the ntp.conf file would update the ntp config in the esxi client, but the client show no NTP servers....
    Any advice on that?

    ReplyDelete
    Replies
    1. Here is a very basic ntp.conf example that will work:
      restrict default kod nomodify notrap nopeer
      restrict 127.0.0.1
      server 10.1.1.9
      server 10.1.1.10
      driftfile /etc/ntp.drift

      Did you reboot your ESX Host after making changes? I have not tried setting this outside of my setup script, so I am not sure if it requires a reboot or not for the UI to pick up the change. One thing I have noticed is the UI is not tightly coupled to changes you make on the filesystem. It seems to query them at boot and store them in a local UI database.

      Delete