VMware, please fix IPv6 support in ESXi!

In a recent blog post I wrote about my efforts to implement IPv6 in my hosted virtual lab. This is working all fine for my VMs now, but to complete the picture I also wanted to configure IPv6 for the public management / VMkernel interface of the physical ESXi host (Please note: you do not need IPv6 support for the management interface if you just want to use IPv6 with your VMs - these are really two different networks!)

To come to the point: I am currently not able to make my ESXi host use IPv6!

The main reason for this is that my hosting provider uses a link-local address (fe80::1 in my case) for the IPv6 gateway. This is not a problem with any other Operating System that I stumbled over so far, but ESXi 5.1 refuses this:

Broken link-local support in VMware ESXi 5
It looks like it is not really uncommon to use link-local addresses for gateways in IPv6 network implementations. In fact I am in the same boat as network giant Cisco - they also complain about this ESXi limitation in their white paper Deploying IPv6 in the Internet Edge.

Unfortunately the work around that Cisco recommends - using router advertisements to auto-configure the IPv6 address and gateway - does not work for me either, because my hosting provider does not provide router advertisements, but requires statically defined addresses and routers.

So I am stuck for now ... but that's not all: When taking a closer look at the IPv6 implementation in ESXi I found even more limitations that you won't find in other Operating Systems:
  • The default gateway address must be on the same subnet as the interface's address
  • You cannot define static routes for a specific interface (if you have multiple vmknics that are IPv6 enabled)
  • When adding a static route you must provide an address for a gateway. Other OSs will allow you to bind a route to just a specific interface and omit the gateway. This is useful for cases where another IPv6 address is directly reachable although it is on a different subnet.
VMware, please fix this! I have started a thread in the VMware Communities Product & Feature Suggestions forum to address my request. If you are also concerned by this then please support it by posting there. If you have a valid support contract for ESXi then you can also file a support request to make yourself heard. Thank you!


7 comments:

  1. But you never route LLA - ULA is for NAT .

    This is where some may disagree, but you use one 64bit subnet (FD:::::) for your internal network.

    This change will require much thought as we rely on subnet for security and this was never the intent with ipv6.

    Originally: Everyone will have a public IP address (fail)
    Now: Everyone will have a ULA (nat/routable) if they do not like the public IP idea.

    Perhaps CIDR or IP based VLAN's are the solution but ipv6 folks never intended you to have more than 1 subnet. 64 bits is huge!

    your LLA IP will go away as soon you give the the interface another IP. I mean do you leave your servers interfaces with 169.254.. ip's? heck no that is a huge security risk. Just as bad as leaving vlan 0 around - hacker will come along and find a way to exploit your switch.



    ReplyDelete
    Replies
    1. I wonder why who misunderstood my post completely ...

      I'm not talking about routing my LLA or ULA, and I'm not going for NAT.
      I am just required to route the regular public IPv6 address of my ESXi host over a gateway that is to be addressed by a LLA (fe80::1). This is just for the first hop. Of course the router itself also has public addresses that it uses to talk to external routing partners.

      I admit that this looks weird at first sight, but it is a perfectly valid and commonly used setup.

      Delete
  2. Idea:
    Since:
    - You cannot set fe80::1 as such
    - You also cannot add an e.g. fe80::2/64 address to vmk0, it will complain
    Try this:
    - Add an address it does not recognize as Link Local (outside fe80::/10), but with a larger netmask so as to encompass fe80::1, for example feff::2/8
    - Then it lets you add fe80::1 as default gateway
    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    ~ # esxcli network ip interface ipv6 address list
    Interface Address Netmask Type Status
    --------- ------------------------- ------- ------ ---------
    vmk0 fe80::4261:86ff:fe2b:8639 64 STATIC PREFERRED
    vmk0 feff::2 8 STATIC PREFERRED
    vmk0 2a01:4f8:101:xxxx::ffff 64 STATIC PREFERRED
    ~ # esxcli network ip route ipv6 list
    Network Netmask Gateway Interface
    ------------------- --------------------- ------- ---------
    default :: fe80::1 vmk0
    2a01:4f8:101:xxxx:: ffff:ffff:ffff:ffff:: :: vmk0
    fe00:: ff00:: :: vmk0
    fe80:: ffff:ffff:ffff:ffff:: fe80::1 lo0
    fe80:: ffff:ffff:ffff:ffff:: :: vmk0
    ff01:: ffff:ffff:: ::1 lo0
    ff01:: ffff:ffff:: :: vmk0
    ff02:: ffff:ffff:: ::1 lo0
    ff02:: ffff:ffff:: :: vmk0
    ~ # esxcli network ip neighbor list
    Neighbor Mac Address Vmknic Expiry State
    ------------------------- ----------------- ------ ------- ---------
    188.40.xxxxxx 00:21:59:c2:0e:c8 vmk0 751 sec
    2a01:4f8:101:xxxx::ffff 40:61:86:2b:86:39 vmk0 0 sec Reachable
    fe80::1 00:21:59:c2:0e:c8 vmk0 28 sec Reachable
    fe80::4261:86ff:fe2b:8639 40:61:86:2b:86:39 vmk0 0 sec Reachable
    feff::2 40:61:86:2b:86:39 vmk0 0 sec Reachable
    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    - Unsurprisingly, the IP pings, as it does with a global unicast gateway:
    ~ # ping fe80::1
    PING fe80::1 (fe80::1): 56 data bytes
    64 bytes from fe80::1: icmp_seq=0 time=0.783 ms
    - But:
    ~ # ping6 ipv6.google.com
    PING ipv6.google.com (2a00:1450:4001:804::1002): 56 data bytes
    sendto() failed (No route to host)

    Whyyyyy? The routing table is correct after all? 2a00:... matches the default route. Which gives fe80::1 as the gateway on vmk0.... And the gateway pings alright. There definitely is a route to the host.
    :-((

    ReplyDelete
    Replies
    1. I guess it tries to go through the loopback interface lo0, because you have
      fe80:: ffff:ffff:ffff:ffff:: fe80::1 lo0
      listed first in the routes table.

      Are you trying this on an ESXi host that is hosted at Hetzner?

      Delete
    2. Absolutely right ;-) I now invested 1 EUR/month for an additional IPv4 address. It's a shame, as there are billions of IPv6 addresses for free.

      Delete
  3. From what I've heard at a couple conferences and read in multiple forums/communities, this is supposed to be fixed with ESXi 5.5 U3. I'm not sure about ESXi 5.1. It's been broken for far too long. Too late for me (former 7-figure VMware customer).

    ReplyDelete
    Replies
    1. Well, it will definitely be fixed in 6.0, but I would not expect anything for 5.x versions.

      Delete

***** All comments will be moderated! *****
- Please post only comments or questions that are related to this post's contents!
- Advertising and link spamming will not be tolerated!