If you’ve not been living on the moon, you’ve probably heard of the concept of “Nested Virtualization”.
This is a method of running a Virtual Machine inside a Virtual Machine.
This concept is not limited to VMware, but since it’s the most popular Hypervisor, I’ll use it as my example. We will talk about the ‘Why’, use-cases, Performance, how-to, and a recent example.
Let’s start with some questions:
The first thing which popped into your mind when you were introduced to nested virtualization is probably:
- “Why would you ever want to do such a scary thing?”
- “Isn’t normal Virtualization good enough anymore?”
- “It’s going to be extremely slow, isn’t it?!”
And I’ll answer:
- Great benefit in certain use cases!
- It’s great!! But not for all scenarios.
- Potentially it would be a little slower, but not much if you do it right, and I’ll explain more next.
But Why? What use cases am I talking about?
– Practice/Modeling Upgrade procedure of your VSphere infrastructure
o Upgrading you Entire Production VSphere infrastructure before you’ve done this before, could be very risky!
Especially with the new surprises every version!
o Trying the upgrade procedure on a nested environment could save you a bunch of troubles and downtime in production infrastructure, especially if you don’t have ‘spare’ ESX hosts to play with.
o Also, you can enjoy all the ‘fun stuff’ that comes with VMware: like Snapshot if you have an issue/ found a bug.
- Testing/ Lab environments.
o In cases when you need something to ‘mess around’ with.
o We are all aware of the situation when you just need a quick setup to try an idea you have, but can’t wait for IT to create that setup for you.
o Small, Non-Production, Temp, testing purposes.
- Demo/ Proof of concept
o If you’re Integrating with the VMware VSphere product family and want to do a Demo/POC without the costs of expensive hardware/datacenter cabling and gear.
- Education
o Teaching/ Learning VSphere Product Family without the time and high costs, while enjoying all the VMware Features (Students can Snapshot/ revert / clone ETC)
Let’s try it ourselves:
ESXi 5.x Virtual machine inside a psychical ESXi 5.x.
Two ways to enable nested virtualization support:
- Via the nested machine’s .vmx file:
o VSphere 5.0 = “vhv.allow = true” (or in /etc/vmware/config)
o VSphere 5.x = “vhv.enable = true”.
- Or the recommended way, Via VCenter Web Client:
Also, to enable Network Communication, Enable “promisc” on the nested machine’s nic.
This is in order to overcome VMware’s default behavior, which by default only forwarding packets from the v-switch to VM’s Mac address.
If you like, Optional VMware Tools for nested ESXi 5.x could be downloaded from VMware ‘flings’:
https://labs.vmware.com/flings/vmware-tools-for-nested-esxi
How about Performance?
When you run a ‘regular’ Virtual Machine, you are technically running a process in the hypervisor.
If you have 10 VMs in the hypervisor you have 10 processes, and so on.
The more processes you have, the more time you spend on context-switching.
Physical processor scheduling and ready time will increase, and potentially the less performance you’ll get.
Today’s Hypervisors handles these situations well.
Now, imagine one of those VMs processes, ‘hides’ inside 10 mores VMs (nested).
Will it be slower? Yes.
Much slower? Not necessarily. Depends on your devices and configurations.
For example: I did a small test, and the nested machine got 10-15% less ‘overall performance’ score, comparing to its parent machine.
Which is not a big price to pay, if you’re not doing production on those nested machine, and the benefit could be great!
A Recent use-case
Recently, I had to use this in order to create a ‘blueprint’ in CloudShare’s TeamLabs, of our Entire integration lab.
Because CloudShare uses many advanced features like Hypervisor API/ Storage vendor API/ complex networking and Security; this hardcore blueprint includes a nested ESXi 5.x and VCenter, Storage communication, and a top of the line Linux box with all the fun stuff of ‘vconfig’ (vlan and trunk configuration in Linux) and brctl (bridge\switch configuration in Linux).
The bottom line is:
We’ve provided R&D and QA a blueprint they can now work, develop, and test on with CloudShare’s awesome features and capabilities!
So Nested virtualization? VERY COOL!
