The anatomy of the ESXi 5.0 installation CD - and how to customize it

1. Introduction

With vSphere 5 VMware introduced the Auto Deploy Server and the Image Builder that allow to customize the ESXi installation ISO with partner supplied driver and tools packages.
The Image Builder is a Powershell snapin that comes with the latest version of the PowerCLI package. It allows to add software packages to a pre-defined set of packages (a so-called ImageProfile) and even lets you create an installation ISO from such a baseline making it easier than ever to customize the ESXi installation.

However, doing this is not a straight-forward task. It requires a working installation of the Powershell, plus the PowerCLI software, access to the offline-bundle that makes up the base installation (which is not included with the free version of ESXi!), a custom driver in VIB format, and some guidance on what Powershell-cmdlets you need to use to add the custom driver package and build an ISO from it.
For the developers of custom drivers it requires to supply their packages in VIB format, and it's not trivial and costs extra effort to build such a package (compared to a simple OEM.TGZ file).

I wondered if it is still possible to customize the ESXi 5.0 install ISO with a simple OEM.TGZ file like you can do with ESXi 4.1, e.g. with my ESXi-Customizer script. And yes, it is possible - but it's very different now! I want to provide some background information here on how this works:

2. The contents of the ESXi 5.0 installation ISO

First let's have a look at the root directory of the ESXi 5.0 install ISO:

Contents of the ESXi 4.0 install CD root directory
Unlike the ESXi 4.1 ISO you can see lots of ISO9660-compatible file names here (all capitals and 8.3-format). You can guess from their names that the files with the V00 (and V01, V02, etc.) extensions are device driver archives. The original  type of these files is VGZ, the short form of VMTAR.GZ. That means that they are gzip'ed vmtar-files.

vmtar is a VMware proprietary variant of tar, and you need the vmtar-tool to pack and unpack vmtar archives. It is part of ESXi 5.0 and also ESXi 4.x. Other files have the extensions TGZ and T00 (like TOOLS.T00). These files are gzip'ed standard tar files that the boot loader can also handle. Good.

Comparing with the ESXi 4.1 media you will notice that there is no ddimage.bz2 file any more. In earlier versions of ESXi this is a compressed image that is written to the installation target disk and contains the whole installed ESXi system. Actually you can write this image to a USB key drive to produce a bootable ESXi system without ever booting the install CD. You cannot do this with ESXi 5.0 any more. However, customizing the install CD has become easier this way, because you do not need to add a second copy of your oem.tgz file to this system image.

There are also files named ISOLINUX.BIN and ISOLINUX.CFG in the ISO root. That means that ESXi 5.0 still uses the isolinux boot loader to make the installation CD bootable. If you look into ISOLINUX.CFG it includes a reference to the file BOOT.CFG, and in BOOT.CFG you find references to all the VGZ and TGZ files:
Contents of the BOOT.CFG file
A second copy of the BOOT.CFG file is in the directory \EFI\BOOT. The ESXi 5.0 install ISO (and ESXi 5.0 itself) was built to boot not only on a standard x86 BIOS, but also on new (U)EFI enabled BIOS versions. Just one thing to remember: If you change the one BOOT.CFG you better make the same change to the other.

Now let's have a closer look at a driver VGZ package.

3. What's in a driver's vgz-file?

As mentioned before you need the vmtar-tool to look into a VGZ-file. Since it is only part of ESXi itself you need to have access to an installed copy of ESXi (either 4.1 or 5.0). Luckily you are able to install ESXi 4.1 (and also 5.0!) inside a VMware Workstation 7 VM.
I did this by creating a VM of type "ESX Server 4" with typical settings except for the size of the virtual disk (2GB is enough for ESXi) and installing ESXi 5.0 in it. During installation the driver files from the CD root are uncompressed and copied to the directory /tardisks, so here is where you can find them again. After enabling the local shell (luckily still available with 5.0) I logged in and was finally able to look inside and unpack such a driver archive using the vmtar tool:
Unpacking NET-E100.V00 with the vmtar tool
So there are basically three files in the archive:

1. The driver binary module (with no file name extension, e1000 in this example) that will be unpacked to the well known location /usr/lib/vmware/vmkmod.

2. A text file that maps PCI device IDs to the included driver:
Contents of /etc/vmware/driver.map.d/e1000.map
3. Another text file that maps PCI IDs to vendor and device descriptive names:
Contents of /usr/share/hwdata/driver.pciids.d/e1000.ids

It is good to know that the PCI ID mapping files are now separated by driver. In ESXi 4.1 there is a single pci.ids file and a single simple.map file for all drivers which raised the potential of having conflicting copies of these files in case you merged multiple OEM drivers into the image.

It looks easy now to add a custom driver to the install CD: Just create a tgz-file containing the three files mentioned above, copy it to the ISO root directory and add its name to the two BOOT.CFG files. And yes, this will indeed work for the CD boot! The custom driver will be loaded and you will be able to install ESXi, ... but the installation routine will not copy the tgz-file to the install media, and if you boot the installed system the first time it will behave like a regular install without the custom driver.

So, there is more to it...

4. The image database IMGDB.TGZ

There is a file named IMGDB.TGZ in the root directory of the CD that is also listed in the BOOT.CFG files and has the following contents:
Unpacking the IMGDB.TGZ file
It contains files that will be unpacked to the directory /var/db/esximg. For each driver (or other software package) an XML-file is created under the vibs sub directory. There are a lot more of these files than shown here (I fiddled the output with "..."), one example is net-e1000--925314997.xml for the e1000 driver. Let's look into this file:
The contents of net-e1000--925314997.xml
The xml-file contains information about the package including possible dependencies on other packages and a list of all included files. Its file name ("net-e1000--925314997.xml") consists of the name element plus a (probably) unique number with 9 or 10 digits. The list of payloads is the list of included archive files (either of type vgz or tgz), in most cases it's just one. The name of the payload is limited to 8 characters ("net-e100" in this case) and is the name of the corresponding file in the CD's root directory. The extension of this file is expected to be ".v00" if the file is of type vgz and ".t00" if the file is of type tgz. If there are name conflicts with other packages the number in the extension is counted up. E.g. the payload file for the e1000e driver is "net-e100.v01".

Then there is the host image profile XML file in the directory /var/db/esximg/profiles. In our example this is the file ESXi-5.0.0-381646-standard1293795055. Let's look into this one:

... ... ... (lot more <vib></vib> entries cutted) ... ... ...
Contents of the host image profile XML file
Here we find a list of all vib-packages that make up the currently installed system. Please note that the vib-id of a package strictly corresponds to the element values that are in the associated vib xml file (see picture before), it is composed the following way:
<vendor>_<type>_<name>_<version>
So the vib-id element of the net-e1000 driver e.g. is
VMware_bootbank_net-e1000_8.0.3.1-2vmw.0.0.383646

The payload names that are listed in the image profile file are the same as in the distinct vib xml files with the exception that here the exact file names (e.g. "net-e100.v00") are listed rather than just the file type (vgz or tgz).

Conclusion: If we want to add a custom driver to the install CD we need to do the following (in addition to the steps described in section 3.): modify the contents of IMGDB.TGZ, add a vib xml file for the driver (similar to net-e1000...xml) to it and update the contained image profile file to include the driver as an additional <vib>-entry.

There is another particular XML element in both the vib files and image profile file that we need to take care of: the <acceptancelevel>. VMware distinguishes four different acceptance levels: VMwareCertified, VMwareAcceptedPartnerSupported and CommunitySupported, in the XML files they are coded as certified, vmwarepartner and community. The names are pretty self-explanatory, and one can easily guess that certified is stricter than vmware that is stricter than partner that in turn is stricter than community. In other words: If the host image profile is of acceptance level certified only packages of the same acceptance level can be part of it. If it is of acceptance level vmware only VMware certified and VMware accepted packages can be installed. If it is of acceptance level partner (and this is the default!) partner supported packages can be installed in addition to that. The least restrictive level is community that would accept all four types of packages.
My expectation is that custom drivers for whitebox hardware are community supported (unless they are published by a hardware vendor company). However, if the driver's vib file contains the acceptance level community the image profile's acceptance level must also be changed to community. Otherwise the installation of the package will fail.

5. Can we automate it?

Yes, we can! The latest version of ESXi-Customizer does automate all the steps described here to add custom drivers in tgz-format to an ESXi 5.0 install ISO. You only need to feed it with a tgz-file that contains the three files listed in section 3 of this post.

Please note: Packages made for earlier ESXi versions will not work with ESXi 5.0, not only because the directory structure has changed, but also because the earlier versions' driver modules won't be loaded by the new version! And - at the time of this writing - there are probably no oem.tgz-style driver packages available that are compatible with ESXi 5.0!
Hopefully, this will soon change. If you are looking for a driver of a device that does not work out-of-the-box with ESXi 5.0 check the Unofficial Whitebox HCL at vm-help.com.


12 comments:

  1. Are there a list of included drivers in ESXi5 and the exactly name of the binary module?

    And if not, how can i investigate the correct name of a driver if i know the driver is included in esxi5?

    For example i know there is a jmicron jmb362/368 driver and i guess that these driver work also with jmicron 361/363 Controller.

    ReplyDelete
  2. Great write-up!

    If I understand what you are saying, I should be able to repackage a working oem.tgz from ESXi 4.1 with the directory structure shown in step 3 to use on my ESXi5.0 install but after install, when I boot the machine my custom drivers will not be there?

    ReplyDelete
  3. Hi Aaron,
    you can merge ESXi 4.1 drivers into the ESXi 5.0 install ISO this way (the easiest way to do this would be using my ESXi-Customizer script), but it just makes no sense, because ESXi 5.0 is not able to use drivers made for ESXi 4.1. See my note at the end of the above post.
    - Andreas

    ReplyDelete
  4. Wouahaou !! Great reverse of the structure... THANKS !!

    Anyway I've just tried your script and a little thing is going wrong somewhere... ;) Cause I success loading my brand new drivers using the ones from :http://www.vm-help.com/forum/viewtopic.php?f=24&t=3489

    But if they are cleanly pushed during the setup (I can "ls" them in the directories) they are not loaded. When I run
    # esxcfg-module -e 8139T00

    It doesn't complain, so it seems to be the first time it's called ?!

    Anyway my network card is still unrecognized and the install sequence abort... this led to my second question: How could I relaunch the setup process without rebooting my server ??

    Thanks a lot for your answers !!!!

    Ben

    ReplyDelete
  5. ben,
    I do not fully understand your issue ... Please send me (mail to [email protected])
    a) the oem.tgz that you are trying to apply (or a download link to it)
    b) The log-file of ESXi-Customizer
    c) A description of what is going wrong when installing and/or booting the server for the first time.
    Thanks - Andreas

    ReplyDelete
  6. Andreas,
    thanks for your note.
    As far as you know is there a method to replace a driver on ISO ?
    I have a LSI megasas 9240. ESXi 5 installation CD stop during scan and I have to:
    1)remove card
    2)install ESXi on USB stick
    3)upgrade driver using esxcli software vib install
    4)instal card
    5) ok now it recognize CARD
    As I would like to install sys & data on RAID,
    what is the best way?

    Thanks in advance

    ReplyDelete
  7. bsa,
    the latest version of my ESXi-Customizer script is able to apply a vib-file to the ESXi 5.0 install ISO. If an older version of the driver already exists in the install ISO it will be updated/replaced.
    - Andreas

    ReplyDelete
  8. Where can we obtain the e1000, e1000.map and e1000.ids files?

    Is it OK to include these in your ESXi-Customizer extraction?

    ReplyDelete
  9. DirtyWilly,
    you can extract these files from any computer that has been installed with ESXi 5.0.
    But what do you want to achieve with these files?
    - Andreas

    ReplyDelete
  10. Andreas, thank for all the information!

    Is it possible, after installation, to replace the e1000 and e1000e drivers of ESXi 5.0 with the ones that came with the 4.0 version?

    Im dealing with a Wake-on-lan issue...
    The hardware used to work OK in the 4.0 version, but after upgrading to ESXi 5.0 the WOL function gives no response..

    Any idea?

    Thanks in advance.

    ReplyDelete
  11. Hello.
    Thank you for your support, tools and help.
    I am a newbie :-)


    I have a problem I can't understand and perhaps you can help me.

    I have created a new image for the installation with a network card driver (vib file)
    I try to install it and when the installation loads all the drivers, it fails and say "no network adapters were detected"...

    lspci:
    000:002:00.0 Network controller: Marvell Technologies, Inc. 88E8055 PCI-E Gigabit Ethernet Controller

    lspci -v:
    000:002:00.0 Network controller: Marvell Technologies, Inc. 88E8055 PCI-E Gigabit Ethernet Controller
    class: 0200: 11ab:4363

    that it is correct, due is the same information in Fedora 16 and windows 7 for this device.

    I can load the driver with vmkload_mod and I have network link, but I don't know how to launch the installation again or how can I do to autoload this driver.

    So I am unable (and frustrated) to continue installing.
    Do you have suggestions?
    Thanks a lot.

    (sorry for my english)

    ReplyDelete
    Replies
    1. Please stop asking support questions in the comments of this or any other post, but use e-mail instead!

      Jota, I will come back to you by e-mail.

      For this post I will now disable comments.

      Thanks
      Andreas

      Delete