This post comes because I am just as confused as the rest of you when it comes to the options available in some of these new vSphere interfaces. I figure it best to take a look at some of the options, and figure when it is best to move away from defaults and start tuning things just right. Today those options are VMDirectPath, and Paravirtual SCSI.
Note: This is what I get for complaining about writers(bloggers?) block on Twitter. This one is another post that originated from the great Mr. @rogerlund. (the picture is by webhamster) More notes! I asked to be called on it, and was. There is some discussion in the comments about what exactly VMDirectPath will do for us, and we found some ‘oddities’ in the VMware docs. Read on!
Specifically Mr. Lund asked me to answer the following three questions: “Which applications would be good for Paravirtual SCSI? Which for VMDirectPath? And; How do we choose?” Because it is hard to decide when you don’t know what does what, let us start with definitions! Yes they are boring, and no there will not be a test.
Paravirtual SCSI Adapters (PVSCSI)
The ‘vSphere Basic System Administration’ guide says this (p116):
“Paravirtual SCSI (PVSCSI) adapters are high-performance storage adapters that can result in greater throughput and lower CPU utilization. Paravirtual SCSI adapters are best suited for high performance storage environments. Paravirtual SCSI adapters are not suited for DAS environments. VMware recommends that you create a primary adapter (LSI Logic by default) for use with a disk that will host the system software (boot disk) and a separate PVSCSI adapter for the disk that will store user data, such as a database.”
We had to jump to page 116 to get this one for VMDirectPath:
“VMDirectPath I/O allows a guest operating system on a virtual machine to directly access physical PCI and PCIe devices connected to a host. Each virtual machine can be connected to up to two PCI devices. PCI devices connected to a host can be marked as available for passthrough from the Hardware Advanced Settings in the Configuration tab for the host.”
However, we find this in the Configuration Examples doc for VMDirectPath seems to conflict a bit:
VMDirectPath allows guest operating systems to directly access an I/O device, bypassing the virtualization layer. This direct path, or passthrough can improve performance for VMware ESXTM systems that utilize high‐speed I/O devices, such as 10 Gigabit Ethernet.
What then is VMDirectPath? It seems that it will allow us to connect an I/O PCI device up to a VM and make things better.
So now that we know what these are… we can start to draw some differences in what they do, and when they might be required.
When & Why?
I figure one of these questions is closely related to the other. Thus it makes sense that we will discuss them together. At least, it makes sense to me. Let’s go! (This pic was borrowed from Pete Reed)
Paravirtual SCSI (PVSCSI)
Part (most) of the when & why for PVSCSI is provided in the quote from above. You use PVSCSI when you need a high performance virtual storage adapter. What does this mean? It means you will not use this for your AD server, or print server. You also generally will not use this on local storage, or DAS (Direct Attached Storage).
When would you use it? Well, I’m glad you asked. Remember that database that you were not virtualizing, because of it’s high IO requirement? That graphics rendering app for marketing that let the magic blue smoke out of your last SAN array? These are good candidates for PVSCSI.
This here, is where we answer the when and why, for VMDirectPath. VMDirectPath is quite a bit different than PVSCSI, but no less cool. It allows VMs to directly access PCI(e) I/O devices. Up to two per VM. Now why would you want to do that? Wasn’t hardware abstraction one of the beauties of virtualizing your environment? Direct access to HBA or 10gb NICs? Those go here. Why? It reduces the overhead of actually virtualizing these operations.
That actually covered a bit of the when as well as the why.
You knew these were coming didn’t you? I know you did. Each of these wonderful technologies have some requirements and some caveats (Who is cav, and what is he eating?) to their use.
Paravirtual SCSI (PVSCSI)
- vSphere (really… it does)
- One Happy Administrator (it does not work when angry, trust me)
- OS: Win2k3/2k8 or RHEL5
- Hardware version 7
Caveats (there is that cav guy… eating again, always eating):
- No Boot disks
- No record/replay
- No FT
- No MSCS
That about covers it, now onto VMDirectPath!
This one needs… well:
- vSphere (well… common)
- Intel Virtualization Technology for Directed I/O (VT-d)
- AMD IP Virtualization Technology (IOMMU)
- Devices must be connected to the host and marked available for pass through
- VMs require hardware version 7
When using VMDirectPath you lose following features:
- Storage VMotion
- Device hot add
- Suspend and resume
- Record and replay
I think this about covers it, but I am quite sure I left something out. Feel free to call me on it in the comments or via Twitter. I am not entirely sold on it being strictly for I/O only. I’ve poked a few contacts at VMware for some clarification.