Follow Me Icons

 

Follow @KendrickColeman on TwitterConnect on LinkedInWatch My Videos on YouTubeFollow me on FacebookCheck Out My Projects on GitHubStay Up To Date with RSS

Search

BSA 728x90 Center Banner

vSphere Host NIC Design - 12 NICs

EDIT: 11/29/2011

I have no updated these to reflect vSphere 5 changes!

VMware vSphere 5 Host Network Design Layout and Configuration

 

I was approached by someone via LinkedIn and the VMTN Community Forums this past week about possible ways to design a vSphere Host NIC layout. The customer has 12 NICs for each server (talk about making it easy), has Enterprise Plus licensing, but wasn't planning on using features such as Fault Tolerance or Virtual Distributed Switches. With that said, here is how I came up with a few designs.


Table Key
2 on-board NICs = vmnic0, vmnic1
4 expansion NICs = vmnic2, vmnic3, vmnic4, vmnic5
4 expansion NICs = vmnic6, vmnic7, vmnic8, vmnic9
2 expansion NICs = vmnic10, vmnic11


Option 1 easier and less configuration:
vSwitch0

 

  • Service Console
    • vmnic0 Active
    • vmnic6 Stand-by
    • VMKernel for Service Console= IP Address in VLAN 50
  • vMotion
    • vmnic2 Active
    • vmnic8 Stand-by
    • VMKernel for vMotion = IP Address in VLAN 51
  • This is dedicating 2 NICs to each option. I do not combine Active/Active because you don't gain any additional benefits unless LACP is implemented
vSwitch1
  • Virtual Machine Network
    • vmnic1,3,4,8,10 all Active
vSwitch2
Option 1a (With Fault Tolerance network)
Do everything the same except remove vmnic4 and 11 from respective vSwitches add:
vSwitch3
  • FT Network
    • vmnic11 Active
    • vmnic4 Standby
    • VMKernel for FT = IP Address in VLAN 53


Option 2
vSwitch0

  • Service Console
    • vmnic0 Active
    • vmnic6 Stand-by
    • VMKernel for Service Console= IP Address in VLAN 50

vSwitch1

vSwitch2
  • Virtual Machine Network
    • vmnic1,3,4,8,10 all Active
vSwitch3


Option 2a

vSwitch4

  • FT Network
    • vmnic11 Active
    • vmnic4 Standby
    • VMKernel for FT = IP Address in VLAN 53

Networking (use your own VLANs, this is just an example)
Service Console - VLAN 50
vMotion - VLAN 51
iSCSI - VLAN 52
FT - VLAN 53
Virtual Machine Network (any vlans needed by your VMs)

You can tag VLANs in 2 places. On the physical switch itself, or on the port group within a vSwitch inside of VMware.
You can tag VLANs within VMware for any of these, but you have to set your switches up to use 801.2q trunking on each port. I would also suggest specifying the VLANs on the configuration to only allow those specific VLANs to pass traffic.

You can also tag vlans on the physical switch itself for everything above (except the Virtual Machine Network if you have more than 1 VLAN or a DMZ too), in lieu of Port Group VLAN tagging.

I would personally tag Service Console, vMotion, iSCSI, and FT all at the physical switch. Then use 801.2q trunking for my Virtual Machine Network. More often than not, you will have to add new VLANs for a VM Network and nothing else. Of course, this is your own cup of tea, so if you want to set every single switch port up to be trunked and specify the VLAN on the port group, you can do that as well.

 

It goes without saying that you can use a Virtual Distributed Switch instead of a vSwitch in any scenario. I would keep Service Console and vMotion on vSwitches and move everything else to a vDS. That's your call.

 

The design and mismatch of vmnics was done with a purpose. To alleviate NIC failure, NIC expansion card failure, or On-board NIC failure. If you end up losing a 4 port expansion card, you don't want to have all your iSCSI traffic flowing over 2 NICs from the same card.

 

I would personally do configuration Option 2a. It's a alot more configuration, but it should yield optimum results. They might not want to use Fault Tolerance, but an extra NIC in the Virtual Machine Network and the iSCSI vSwitches aren't going to give you any benefits besides another layer of redundancy. I setup the FT network here at my work, but we don't use any FT VMs. Having it all designed out correctly from the beginning so you're not having to backtrack will make your life easier. Initial design matters most.

 

Please take a look at my other posts for more information and diagrams:

vSphere Host NIC Design - 6 NICs

vSphere Host NIC Design - 10 NICs

you have to think NIC failure, NIC expansion card failure, or On-board NIC failure. If you end up losing a 4 port expansion card, you don't want to have all your iSCSI traffic flowing over 2 NICs from the same card.

Related Items

Related Tags