AutoLab automation uncovered, ESXi builds

Hi, it’s Alastair again, this is the third post uncovering the automation that is in the AutoLab. If you don’t know what the AutoLab is then head to my AutoLab announcement post. If you missed the first article on AutoLab automation then you can head there now, then read the second post before returning here.

A rather important part of the AutoLab are the ESXi servers.  The server basic builds are automated using a PXE server and the Weasel installer that has been part of ESXi since 4.1. Weasel doesn’t require PXE, it is perfectly happy with CDROM or USB boot, the AutoLab uses PXE to allow central management which makes scaling easier. PXE requires two services, a DHCP service and a TFTP service. In the AutoLab both are provided by the DC VM, the DHCP service is the standard Windows DHCP and the TFTP service is the Jounin TFTP server. 

DHCP Setup

The DHCP setup is classic PXE, set two additional option value on the scope and you are done.

1. Option 66 Boot Server Host Name is set to the IP address of the TFTP server
2. Option 067 Bootfile Name is set to pxelinux.0 which is the syslinux PXE boot loader. 

As long as the DHCP server hands out enough information for the PXE client to connect to the boot host, including DNS if you use the hostname rather than IP address, you will be done there. In the AutoLab this is easy as we don’t use VLAN tags on the management network.  In a corporate environment this may be more difficult as DHCP and TFTP use the native VLAN in a VLAN trunked environment.

TFTP Setup

Jounin was selected for it’s freely redistributable license model, there are plenty of other TFTP servers that you can use.  TFTP is simply a lightweight and non-authenticated file transfer service, here it is used to copy files into RAM as the ESXi servers are built. 

On the DC the folder C:\TFTP-Root is made available through the TFTP service.  The file pxelinux.0 is a PXE boot loader, it is part of the SysLinux project at www.syslinux.org where you will also find some usage instructions.  pxelinux.0 expects a pxelinux.cfg folder under the TFTP-Root folder and looks in here for a series of configuration files.  We simply use the last resort configuration file named default, this file controls the menu you see when the ESXi server boot.

Also in the TFTP-Root folder  are version specific folders for each version of ESXi and ESX.  The folders for 4.1 versions contain only the boot files, the install files come from the NAS via NFS, but the ESXi 5.0 folder contains a complete copy of the contents of the ESXi install ISO.

ESXi 5.0

ESXi 50 is designed to allow PXE boot for the full platform which is how AutoDeploy works, We will use the PXE boot to launch the installer for ESXi.  The only change required to the copy of the ESXi installer ISO contents is in the text file boot.cfg.  This file is used to control the boot of the ESXi installer and contains references to each of the archives that are loaded into the RAM disk. Since the ISO contents is copied to a sub directory of the TFTP-Root folder we need to add this path to each file reference.  The modified boot.cfg file will now allow a manual install of ESXi 5.0 via PXE boot, it will behave exactly the same as booting the host from the ESXi installer CDROM.  To have this appear in the PXELinux menu the following three lines are added to the file Default in the pxelinux.0 folder:

        label esx50
        KERNEL /esxi50/mboot.c32
        APPEND -c /esxi50/boot.cfg
        MENU LABEL ESXi-5.0 Installer

To automate the ESXi installation we need to add an answer file, this answers the questions that are asked during the manual install.  The answer filename is added to the boot.cfg file, in the AutoLab I have used separate answer files for each ESXi server as the files contain hostnames and IP addresses.  This means that there need to be boot.cfg files for each ESXi server.  I have named these boot.cfg files as b<ESXi name>.cfg and added to the file Default another section for each ESXi server. The APPEND line being changed to reflect the unique boot.cfg file which in turn calls the unique answer file.

In the AutoLab the answer files are taken from NAS using NFS, rather than the TFTP server, mainly because this is the way prior versions of ESX automated install worked. In a production use you are more likely to take the answer file from the TFTP server as there is no other need for the NAS.

The Answer file itself is fairly simple, in the AutoLab it is just five lines.  The part that is risky in a production environment is the third line, it will repartition the first disk found in the host which will be data destructive.  In the lab this is safe as we don’t use the local disk for anything except booting the ESXi servers and there are no other disks visible at install time.

        vmaccepteula
        rootpw VMware1!
        install –firstdisk –overwritevmfs
        network –bootproto=static –ip=192.168.199.11 –gateway=192.168.199.2 –nameserver=192.168.199.4 –netmask=255.255.255.0 –hostname=host1.lab.local –device=vmnic0 –addvmportgroup=0
        reboot
This gest a base ESXi server built, in the AutoLab we then use a PowerCLI script to add the host to vCenter and do a lot more configuration.

In the real world you may choose to use Host Profiles or scripting to complete the build, but you can use the answer file. The answer file supports having commands run automatically after the ESXi server is installed and rebooted.  This allows further networking, storage and general configuration tasks to be completed automatically.

VMware have a knowledgebase article about unattended ESXi installs and the ever useful Duncan Epping has a post that shows some use of the post install scripting to do a fair amount of configuration automatically.

Adding another ESXi server to the build

Since the question just cam up on twitter, here’s the steps to add another host build to the menus

1. Create a new answer file, in \Automate\Hosts on the build share copy esx1-5.cfg to your new server number i.e. esx3-5.cfg
2. Change the hostname and IP address in the answer file
3. Create a new boot.cfg file, in C:\TFTP-Root\ESXi50 on the DC copy besx1-5.cfg to your new server number i.e. besx3-5.cfg
4. Change the list line of the new file e.g. besx3-5.cfg to point to the new answer file
5. Edit C:\TFTP-Root\pxelinux.cfg\default, copy the whole block that starts Label esx1-5 to after the last ESXi 5.0 build block
6. Update the new block with a new label, boot.cfg file and menu label.
7. Done.