ImageBuilder Deep Dive, Part 2: A closer look and advanced functions

In the first part of this Image Builder Deep Dive I introduced a script that you can use to build your own customized installation ISO and can easily be adapted to your own needs. In this second part we will take a closer look at the basic cmdlets we used and get to know some more cmdlets with advanced functionality.

To make best use of the following explanations I suggest that you load the complete script that we go through in this part into the PowerShell ISE and run it line by line while reading the post. You can also experiment with the various code samples by pasting them into the command prompt of the ISE.

Using Online depots with proxy servers

At the beginning of the first part's script we added two Online depots to the ImageBuilder session using the Add-EsxSoftwareDepot cmdlet. I stated that this is not possible when using a proxy server, and  - after some testing - I need to correct this statement: I got it working with a proxy server that does not require authentication. With a proxy server requiring authentication it worked on the first try and failed when adding the second online depot (?!). Originally I had a proxy auto-configuration URL entered in the Internet Explorer settings, and this made adding an Online depot consequently fail. Your mileage may vary ...

By default PowerCLI will use the Windows proxy settings (that means what you have configured in Internet Explorer). You can change that by using the Set-PowerCLIConfiguration cmdlet. However, the possibilities are somewhat limited. You can either use the system proxy settings or no proxy at all:
# Load the VimAutomation Snapin
Add-PSSnapin VMware.VimAutomation.Core
#
# Set and get the proxy configuration
Set-PowerCLIConfiguration -ProxyPolicy NoProxy
Set-PowerCLIConfiguration -ProxyPolicy UseSystemProxy
Get-PowerCLIConfiguration
#
Please note: You need to add the VMware.VimAutomation.Core snapin first (if not already added) to use these cmdlets. To display the current setting use the Get-PowerCLIConfiguration cmdlet.

How to "download" the VMware Online base depot

Now you may wonder: What if your computer has no Internet connection (or only via a proxy configuration that causes the above mentioned problems)? Can I somewhat "download" an Online depot and re-use it later on the same or another computer? Yes, you can!

To be exact: You cannot create an offline copy of a complete Online depot, but you can export a specific ImageProfile not only to an installation ISO, but also into a re-usable Offline bundle:
# Load the ImageBuilder Snapin
Add-PSSnapin VMware.ImageBuilder
#
# Reference the VMware ESXi base depot
$baseDepot = Add-EsxSoftwareDepot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
#
# List available ImageProfiles sorted by name (filtered on updated standard profiles)
Get-EsxImageProfile "ESXi-5.0.0-2012*-standard" | Sort-Object -Descending Name
#
# The previous command outputs "ESXi-5.0.0-20120504001-standard" as the newest profile (as of May 5th 2012)
# Export this Imageprofile into an Offline bundle zip file
Export-EsxImageProfile -ImageProfile ESXi-5.0.0-20120504001-standard -ExportToBundle -FilePath 'U:\ESXi-5.0.0-20120504001-standard.zip'
#
Now you can re-use this exported zip file any time and on any other computer by adding it with the Add-EsxSoftwareDepot cmdlet. Obviously this Offline bundle zip file only contains the ImageProfile that you exported and only the packages that are part of this ImageProfile.

Please note: The HP Online depot cannot be "downloaded" in this way, because it does not expose any ImageProfiles that you can export. However, in this special case you can also open the http://vibsdepot.hp.com URL in a browser and download the Offline bundles that it includes from the linked web pages.

Looking closer at the ImageProfile object

Apropos ImageProfiles: So far we only looked at the names of these objects. Obviously VMware codes the release date of an ImageProfile (= patch level) into its name using the format YYYYMMDD. Hopefully they will stick with this naming standard - it makes it easy to identify the newest ImageProfile at first sight and programmatically.

The ImageProfile object has more interesting attributes though:
# Examining the ImageProfile's attributes
$ip = Get-EsxImageProfile ESXi-5.0.0-20120504001-standard
$ip | format-list
$ip.VibList
#
By piping the object through the format-list cmdlet you can display all its attributes in a nicely formatted list view:
Name            : ESXi-5.0.0-20120504001-standard
Vendor          : VMware, Inc.
Author          : 
Description     : For more information, see http://kb.vmware.com/kb/2019863.
CreationTime    : 30.04.2012 21:47:06
ModifiedTime    : 30.04.2012 21:47:06
ReadOnly        : False
VibList         : {ata-pata-atiixp 0.4.6-3vmw.500.0.0.469512, net-nx-nic 4.0.557-3vmw.500.1.11.623860, scsi-rste 2.0.2.0088-1vmw.500.1.11.623860, net-e1000 8.0.3.1-2vmw.500.0.7.515841...}
AcceptanceLevel : PartnerSupported
Guid            : 076da48a930e11e1b58d0017a477682c
Rules           : 
StatelessReady  : True
The Description attribute is interesting, because its value includes a URL to a VMware KB article with further information about this ESXi 5.0 patch release.

The VibList attribute is an array of all included software packages. The list is shortened in the formatted object view to retain readability of the output. The third command above ($ip.VibList) will output the complete list of packages with names and versions.

The AcceptanceLevel attribute is something that you normally don't need to take care of, but we will come back to it at the end of this post.

How to compare two ImageProfiles

If you want to quickly check the differences of two ImageProfiles you could list their attributes one after the other like shown before and compare the outputs, especially the VibLists. However, there is a more elegant way to do this - you can use the Compare-EsxImageProfile cmdlet:
# Comparing two ImageProfiles
Compare-EsxImageProfile ESXi-5.0.0-20120504001-standard -ReferenceProfile ESXi-5.0.0-20120404001-standard
#
This command will create and output an ImageProfile comparison object. In this example it looks like this:
Equal               : False
PackagesEqual       : False
RefAcceptanceLevel  : PartnerSupported
CompAcceptanceLevel : PartnerSupported
OnlyInRef           : {}
OnlyInComp          : {}
UpgradeFromRef      : {VMware_bootbank_esx-base_5.0.0-1.13.702118}
DowngradeFromRef    : {}
What do we see here? The two ImageProfiles are not equal (Equal = False), they do not contain the exact same list of packages (PackagesEqual = False), but they have the same AcceptanceLevel (PartnerSupported). The remaining four attributes summarize the differences in the software package list of the two ImageProfiles. Their names are pretty self-explanatory, and in this example the only difference is that the comparison profile (ESXi-5.0.0-20120504001-standard) contains a newer version of the esx-base package than the reference profile (ESXi-5.0.0-20120404001-standard).

Examining, adding and removing single software packages from a depot

In the first part of this series of posts we used a pipeline of commands to add all packages of a depot to our custom profile. Now we are going to take a closer look at a single package using the Get-EsxSoftwarePackage cmdlet:
# Reference downloaded HP offline bundle for be2net driver
$be2net = Add-EsxSoftwareDepot "U:\HP-ESXi5-Drivers\be2net-4.0.355.1-offline_bundle-487292.zip"
#
# Look at all versions of a single software package (net-be2net)
Get-EsxSoftwarePackage net-be2net
# Look at all properties of a package's specific version
Get-EsxSoftwarePackage net-be2net -Version 4.0.88.0-1vmw.500.0.7.515841 | fl
#
The output of the command in line 34 looks like this:
Name                     Version                        Vendor     Release Date    
----                     -------                        ------     ------------    
net-be2net               4.0.88.0-1vmw.500.0.7.515841   VMware     15.12.2011 00...
net-be2net               4.0.88.0-1vmw.500.0.0.469512   VMware     19.08.2011 01...
net-be2net               4.0.355.1-1OEM.500.0.0.406165  Emulex     16.09.2011 00...
That means that there are three different versions of the net-ne2net package, the first two are in the VMware base depot, and the third one is in the Offline bundle we downloaded from HP.

In line 36 we examine the properties of a specific version of a package. We will see that the SoftwarePackage object has some interesting attributes:
Name            : net-be2net
Version         : 4.0.88.0-1vmw.500.0.7.515841
Vendor          : VMware
Summary         : Updates the ESX 5.0.0 net-be2net
Description     : For build informatin, see KB  http://kb.vmware.com/kb/2007675
ReleaseDate     : 15.12.2011 00:00:00
Depends         : {vmkapi_2_0_0_0, com.vmware.driverAPI-9.2.0.0}
Conflicts       : {}
Replaces        : {}
AcceptanceLevel : VMwareCertified
MaintenanceMode : False
LiveInstallOk   : False
LiveRemoveOk    : False
SourceUrls      : {https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/net-be2net/VMware_bootbank_net-be2net_4.0.88.0-1vmw.500.0.7.515841.vib}
Guid            : VMware_bootbank_net-be2net_4.0.88.0-1vmw.500.0.7.515841
StatelessReady  : True
Tags            : {driver, module, category:bugfix, severity:general}
Like with the ImageProfile object the Description property contains a link to a VMware KB article with further information on this package. The SourceUrl indicates from where the package will be downloaded. And the package's Tags provides additional classification categories.

The three properties Depends, Conflicts and Replaces describe relationships to other packages (or features that are provided by other packages). Their names are self-explanatory, and ImageBuilder will take care of them for us. You will e.g. not be allowed to add a package that conflicts with another package that is already part of the profile.

Now let's add this single package to a custom profile using the Add-EsxSoftwarePackage cmdlet:
# Create a new ImageProfile
$MyProfile = New-EsxImageProfile -CloneProfile ESXi-5.0.0-20120504001-standard -Name MyProfile
#
# Add an older version of the net-be2net package to the custom profile
Add-EsxSoftwarePackage -ImageProfile $MyProfile "net-be2net 4.0.88.0-1vmw.500.0.7.515841"
# Add the neweset net-be2net package to the custom profile
Add-EsxSoftwarePackage -ImageProfile $MyProfile net-be2net
#
If you just specify the name of the software package (net-be2net in this example), and there are multiple versions of this package available then Add-EsxSoftwarePackage will automatically add the newest version. If you want to add an older version then you can specify it like shown in line 42. Whenever the ImageProfile already includes a package of the same name, but a different version, it will be replaced by the added package, even if it is older! If the ImageProfile already includes the exact same version then the command will fail, or in other words: you cannot add the same version twice.

Finally, you can also remove a package from an ImageProfile using the Remove-EsxSoftwarePackage cmdlet:
# Remove a package from the custom profile
Remove-EsxSoftwarePackage -ImageProfile $MyProfile net-bnx2
#
You won't be able to remove packages that are required by other packages of the ImageProfile. In some case you can override this dependency check by using the parameter -Force, but you really shouldn't do that, because it will most likely result in an invalid ImageProfile.

Adding community developed packages

If you have ever used my ESXi-Customizer script to add a Community developed software package to an ESXi 5.0 installation ISO then you may wonder: Can you also do that with ImageBuilder? Yes, you can, and here is how:
# Add an Offline Bundle with a community developed software package
Add-EsxSoftwareDepot 'U:\$Download\fwenable-ntpd-1.2.0-0-offline_bundle.zip'
# Change the AcceptanceLevel of the custom profile to "CommunitySupported"
$MyProfile.AcceptanceLevel = "CommunitySupported"
# Add the package to the profile
Add-EsxSoftwarePackage -ImageProfile $MyProfile fwenable-ntpd
In this example we add a software package that includes a custom firewall rule to allow incoming NTP queries (this way you can use the ESXi host as NTP server. Read this post for background information). The Offline bundle was created using my ESXi5 Community Packaging Tools by converting a standard tar.gz file to a VIB file first, and then packaging the VIB file into an Offline bundle zip. The project page of the tools contains detailed instructions on how to do this.

There is only one particularity that you need to be aware off when adding Community developed packages this way: Software packages that were created (and certified) by VMware or trusted partners contain an electronic signature that is checked when adding (or installing) the package. This qualifies them for one of the following three so-called AcceptanceLevels: VMwareCertified, VMwareAccepted or PartnerSupported. Of course, Community developed packages do not have trusted electronic signatures, which means that you can only install them if they have the least restrictive (and least trustworthy) AcceptanceLevel called CommunitySupported. The ImageProfile object also has an AcceptanceLevel, and that must be lowered to the same level before you are able to add a Community developed package. This happens in line 52.

This ends the second part of the ImageBuilder Deep Dive series. The next and last part will not deal with the ImageBuilder snapin specifically, but with Powershell coding in general. We will then develop the script from the first part to a more general and versatile solution.


No comments:

Post a Comment

***** All comments will be moderated! *****
- Please post only comments or questions that are related to this post's contents!
- Advertising and link spamming will not be tolerated!