guides: Notes to setup DevStack with Nested KVM Virtualization

Add a document with procedure to configure KVM-based nested
virtualization on the physical host and to configure DevStack (in a VM)
to take advantage of it.

Current topics:

   - Configure nested virt on Intel hosts
   - Configure nested virt on AMD hosts
   - Expose virt extensions to DevStack VM
   - Ensure DevStack VM is using KVM

Change-Id: Ibe6fa482cc0d51183438d99680a0e10d0da652cb

doc/source/guides/devstack-with-nested-kvm.rst

@@ -0,0 +1,139 @@
+=======================================================
+Configure DevStack with KVM-based Nested Virtualization
+=======================================================
+
+When using virtualization technologies like KVM, one can take advantage
+of "Nested VMX" (i.e. the ability to run KVM on KVM) so that the VMs in
+cloud (Nova guests) can run relatively faster than with plain QEMU
+emulation.
+
+Kernels shipped with Linux distributions doesn't have this enabled by
+default. This guide outlines the configuration details to enable nested
+virtualization in KVM-based environments. And how to setup DevStack
+(that'll run in a VM) to take advantage of this.
+
+
+Nested Virtualization Configuration
+===================================
+
+Configure Nested KVM for Intel-based Machines
+---------------------------------------------
+
+Procedure to enable nested KVM virtualization on AMD-based machines.
+
+Check if the nested KVM Kernel parameter is enabled:
+
+::
+
+    cat /sys/module/kvm_intel/parameters/nested
+    N
+
+Temporarily remove the KVM intel Kernel module, enable nested
+virtualization to be persistent across reboots and add the Kernel
+module back:
+
+::
+
+    sudo rmmod kvm-intel
+    sudo sh -c "echo 'options kvm-intel nested=y' >> /etc/modprobe.d/dist.conf"
+    sudo modprobe kvm-intel
+
+Ensure the Nested KVM Kernel module parameter for Intel is enabled on
+the host:
+
+::
+
+    cat /sys/module/kvm_intel/parameters/nested
+    Y
+
+    modinfo kvm_intel | grep nested
+    parm:           nested:bool
+
+Start your VM, now it should have KVM capabilities -- you can verify
+that by ensuring `/dev/kvm` character device is present.
+
+
+Configure Nested KVM for AMD-based Machines
+--------------------------------------------
+
+Procedure to enable nested KVM virtualization on AMD-based machines.
+
+Check if the nested KVM Kernel parameter is enabled:
+
+::
+
+    cat /sys/module/kvm_amd/parameters/nested
+    0
+
+
+Temporarily remove the KVM AMD Kernel module, enable nested
+virtualization to be persistent across reboots and add the Kernel module
+back:
+
+::
+
+    sudo rmmod kvm-amd
+    sudo sh -c "echo 'options amd nested=1' >> /etc/modprobe.d/dist.conf"
+    sudo modprobe kvm-amd
+
+Ensure the Nested KVM Kernel module parameter for AMD is enabled on the
+host:
+
+::
+
+    cat /sys/module/kvm_amd/parameters/nested
+    1
+
+    modinfo kvm_amd | grep -i nested
+    parm:           nested:int
+
+To make the above value persistent across reboots, add an entry in
+/etc/modprobe.ddist.conf so it looks as below::
+
+    cat /etc/modprobe.d/dist.conf
+    options kvm-amd nested=y
+
+
+Expose Virtualization Extensions to DevStack VM
+-----------------------------------------------
+
+Edit the VM's libvirt XML configuration via `virsh` utility:
+
+::
+
+    sudo virsh edit devstack-vm
+
+Add the below snippet to expose the host CPU features to the VM:
+
+::
+
+    <cpu mode='host-passthrough'>
+    </cpu>
+
+
+Ensure DevStack VM is Using KVM
+-------------------------------
+
+Before invoking ``stack.sh`` in the VM, ensure that KVM is enabled. This
+can be verified by checking for the presence of the file `/dev/kvm` in
+your VM. If it is present, DevStack will default to using the config
+attribute `virt_type = kvm` in `/etc/nova.conf`; otherwise, it'll fall
+back to `virt_type=qemu`, i.e. plain QEMU emulation.
+
+Optionally, to explicitly set the type of virtualization, to KVM, by the
+libvirt driver in Nova, the below config attribute can be used in
+DevStack's ``local.conf``:
+
+::
+
+    LIBVIRT_TYPE=kvm
+
+
+Once DevStack is configured succesfully, verify if the Nova instances
+are using KVM by noticing the QEMU CLI invoked by Nova is using the
+parameter `accel=kvm`, e.g.:
+
+::
+
+    ps -ef | grep -i qemu
+    root     29773     1  0 11:24 ?        00:00:00 /usr/bin/qemu-system-x86_64 -machine accel=kvm [. . .]

doc/source/index.rst

@@ -66,6 +66,7 @@ Walk through various setups used by stackers
    guides/single-machine
    guides/multinode-lab
    guides/neutron
+   guides/devstack-with-nested-kvm
 
 All-In-One Single VM
 --------------------
@@ -94,6 +95,13 @@ Building a DevStack cluster with :doc:`Neutron Networking <guides/neutron>`.
 This guide is meant for building lab environments with a dedicated
 control node and multiple compute nodes.
 
+DevStack with KVM-based Nested Virtualization
+---------------------------------------------
+
+Procedure to setup :doc:`DevStack with KVM-based Nested Virtualization
+<guides/devstack-with-nested-kvm>`. With this setup, Nova instances
+will be more performant than with plain QEMU emulation.
+
 DevStack Documentation
 ======================