I’m a big fan of simple vSphere network configurations, one vSwitch with a small number of fast uplinks. This has been my stance since ESX 3 and I think it’s still a good starting point for a network design. I have had some concerns about the vNetwork Distributed switch (vDS) with it’s management through vCentre and the inherent ability to make a mistake that break the vDS for every ESXi server that uses the vDS. In the past I couldn’t be unreserved about my suggestion that there be a single vDS as the whole network configuration for group of ESXi servers as the ability to break a lot of hosts with one mistake is a big risk. An example would be changing the VLAN ID of the VMkernel portgroup (used to connect to vCentre) to the wrong value, rendering every ESXi server unmanageable with a single error. With great power comes great responsibility. Combine this with the difficulty of managing the vDS from the ESXi shell made me more confortable with a hybrid environment, a vNetwork Standard Switch (vSS) for the management VMkernel portgroup and a vDS for all VM and IP storage traffic. Recovering a broken vSS from the ESXi command line is a pretty well documented process of esxcfg-vswitch and esxcfg-vmknic commands.
With the release of vSphere 5.1 the vDS acquired an awesome new ability, the ability to automatically back out a toxic change. If a vDS configuration change on a host removes the ability of that host to connect to vCentre then the change is undone on the host. This removes my objection to putting the management VMkernel port on a vDS and makes me happy with a return to my one vSwitch policy even when that one switch is a vDS. With the huge manageability benefits of vDS this makes me very happy.
From what I’ve read any vDS change that results in the ESXi server being unable to talk to vCentre for 20 seconds is automatically undone. So far I haven’t been able to test the rollback by making toxic changes on purpose, as soon as I get the vSphere 5.1 AutoLab build that Grant Orchard has put together tested I plan on some serious wreckage to make sure this works how it sounds.