From 75afd6d33c93f510fb9f62abd443eac4131480e6 Mon Sep 17 00:00:00 2001 From: Kashyap Chamarthy Date: Tue, 20 Jan 2015 17:39:25 +0100 Subject: [PATCH] guides: Notes to setup DevStack with Nested KVM Virtualization Add a document with procedure to configure KVM-based nested virtualization on the physical host and to configure DevStack (in a VM) to take advantage of it. Current topics: - Configure nested virt on Intel hosts - Configure nested virt on AMD hosts - Expose virt extensions to DevStack VM - Ensure DevStack VM is using KVM Change-Id: Ibe6fa482cc0d51183438d99680a0e10d0da652cb --- .../guides/devstack-with-nested-kvm.rst | 139 ++++++++++++++++++ doc/source/index.rst | 8 + 2 files changed, 147 insertions(+) create mode 100644 doc/source/guides/devstack-with-nested-kvm.rst diff --git a/doc/source/guides/devstack-with-nested-kvm.rst b/doc/source/guides/devstack-with-nested-kvm.rst new file mode 100644 index 0000000000..2538c8d031 --- /dev/null +++ b/doc/source/guides/devstack-with-nested-kvm.rst @@ -0,0 +1,139 @@ +======================================================= +Configure DevStack with KVM-based Nested Virtualization +======================================================= + +When using virtualization technologies like KVM, one can take advantage +of "Nested VMX" (i.e. the ability to run KVM on KVM) so that the VMs in +cloud (Nova guests) can run relatively faster than with plain QEMU +emulation. + +Kernels shipped with Linux distributions doesn't have this enabled by +default. This guide outlines the configuration details to enable nested +virtualization in KVM-based environments. And how to setup DevStack +(that'll run in a VM) to take advantage of this. + + +Nested Virtualization Configuration +=================================== + +Configure Nested KVM for Intel-based Machines +--------------------------------------------- + +Procedure to enable nested KVM virtualization on AMD-based machines. + +Check if the nested KVM Kernel parameter is enabled: + +:: + + cat /sys/module/kvm_intel/parameters/nested + N + +Temporarily remove the KVM intel Kernel module, enable nested +virtualization to be persistent across reboots and add the Kernel +module back: + +:: + + sudo rmmod kvm-intel + sudo sh -c "echo 'options kvm-intel nested=y' >> /etc/modprobe.d/dist.conf" + sudo modprobe kvm-intel + +Ensure the Nested KVM Kernel module parameter for Intel is enabled on +the host: + +:: + + cat /sys/module/kvm_intel/parameters/nested + Y + + modinfo kvm_intel | grep nested + parm: nested:bool + +Start your VM, now it should have KVM capabilities -- you can verify +that by ensuring `/dev/kvm` character device is present. + + +Configure Nested KVM for AMD-based Machines +-------------------------------------------- + +Procedure to enable nested KVM virtualization on AMD-based machines. + +Check if the nested KVM Kernel parameter is enabled: + +:: + + cat /sys/module/kvm_amd/parameters/nested + 0 + + +Temporarily remove the KVM AMD Kernel module, enable nested +virtualization to be persistent across reboots and add the Kernel module +back: + +:: + + sudo rmmod kvm-amd + sudo sh -c "echo 'options amd nested=1' >> /etc/modprobe.d/dist.conf" + sudo modprobe kvm-amd + +Ensure the Nested KVM Kernel module parameter for AMD is enabled on the +host: + +:: + + cat /sys/module/kvm_amd/parameters/nested + 1 + + modinfo kvm_amd | grep -i nested + parm: nested:int + +To make the above value persistent across reboots, add an entry in +/etc/modprobe.ddist.conf so it looks as below:: + + cat /etc/modprobe.d/dist.conf + options kvm-amd nested=y + + +Expose Virtualization Extensions to DevStack VM +----------------------------------------------- + +Edit the VM's libvirt XML configuration via `virsh` utility: + +:: + + sudo virsh edit devstack-vm + +Add the below snippet to expose the host CPU features to the VM: + +:: + + + + + +Ensure DevStack VM is Using KVM +------------------------------- + +Before invoking ``stack.sh`` in the VM, ensure that KVM is enabled. This +can be verified by checking for the presence of the file `/dev/kvm` in +your VM. If it is present, DevStack will default to using the config +attribute `virt_type = kvm` in `/etc/nova.conf`; otherwise, it'll fall +back to `virt_type=qemu`, i.e. plain QEMU emulation. + +Optionally, to explicitly set the type of virtualization, to KVM, by the +libvirt driver in Nova, the below config attribute can be used in +DevStack's ``local.conf``: + +:: + + LIBVIRT_TYPE=kvm + + +Once DevStack is configured succesfully, verify if the Nova instances +are using KVM by noticing the QEMU CLI invoked by Nova is using the +parameter `accel=kvm`, e.g.: + +:: + + ps -ef | grep -i qemu + root 29773 1 0 11:24 ? 00:00:00 /usr/bin/qemu-system-x86_64 -machine accel=kvm [. . .] diff --git a/doc/source/index.rst b/doc/source/index.rst index 7188f6ecf4..1aaab6e2d6 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -66,6 +66,7 @@ Walk through various setups used by stackers guides/single-machine guides/multinode-lab guides/neutron + guides/devstack-with-nested-kvm All-In-One Single VM -------------------- @@ -94,6 +95,13 @@ Building a DevStack cluster with :doc:`Neutron Networking `. This guide is meant for building lab environments with a dedicated control node and multiple compute nodes. +DevStack with KVM-based Nested Virtualization +--------------------------------------------- + +Procedure to setup :doc:`DevStack with KVM-based Nested Virtualization +`. With this setup, Nova instances +will be more performant than with plain QEMU emulation. + DevStack Documentation ======================