libvirt: support SASL authentication

In Kolla Ansible OpenStack deployments, by default, libvirt is
configured to allow read-write access via an unauthenticated,
unencrypted TCP connection, using the internal API network.  This is to
facilitate migration between hosts.

By default, Kolla Ansible does not use encryption for services on the
internal network (and did not support it until Ussuri). However, most
other services on the internal network are at least authenticated
(usually via passwords), ensuring that they cannot be used by anyone
with access to the network, unless they have credentials.

The main issue here is the lack of authentication. Any client with
access to the internal network is able to connect to the libvirt TCP
port and make arbitrary changes to the hypervisor. This could include
starting a VM, modifying an existing VM, etc. Given the flexibility of
the domain options, it could be seen as equivalent to having root access
to the hypervisor.

Kolla Ansible supports libvirt TLS [1] since the Train release, using
client and server certificates for mutual authentication and encryption.
However, this feature is not enabled by default, and requires
certificates to be generated for each compute host.

This change adds support for libvirt SASL authentication, and enables it
by default. This provides base level of security. Deployments requiring
further security should use libvirt TLS.

[1] https://docs.openstack.org/kolla-ansible/latest/reference/compute/libvirt-guide.html#libvirt-tls

Depends-On: https://review.opendev.org/c/openstack/kolla/+/833021
Closes-Bug: #1964013
Change-Id: Ia91ceeb609e4cdb144433122b443028c0278b71e

ansible/roles/nova-cell/defaults/main.yml

@@ -529,6 +529,14 @@ migration_hostname: "{{ ansible_facts.nodename }}"
 # It does not change that often (in fact, most likely never ever).
 qemu_user_gid: 42427
 
+# Whether to enable libvirt SASL authentication.
+libvirt_enable_sasl: true
+# Username for libvirt SASL.
+libvirt_sasl_authname: "nova"
+# List of enabled libvirt SASL authentication mechanisms.
+libvirt_sasl_mech_list:
+  - "{{ 'SCRAM-SHA-256' if libvirt_tls | bool else 'DIGEST-MD5' }}"
+
 ####################
 # Kolla
 ####################

ansible/roles/nova-cell/handlers/main.yml

@@ -93,6 +93,7 @@
   vars:
     service_name: "nova-libvirt"
     service: "{{ nova_cell_services[service_name] }}"
+    nova_libvirt_notify: "{{ ['Create libvirt SASL user'] if libvirt_enable_sasl | bool else [] }}"
   become: true
   kolla_docker:
     action: "recreate_or_restart_container"
@@ -112,6 +113,20 @@
   until: restart_nova_libvirt is success
   when:
     - kolla_action != "config"
+  notify: "{{ nova_libvirt_notify }}"
+
+# The SASL user needs to exist in order for nova-compute to start successfully.
+- name: Create libvirt SASL user
+  become: true
+  shell:
+    cmd: >
+      set -o pipefail &&
+      echo {{ libvirt_sasl_password }} |
+      docker exec -i nova_libvirt
+      saslpasswd2 -c -p -a libvirt {{ libvirt_sasl_authname }}
+    executable: /bin/bash
+  changed_when: true
+  no_log: true
 
 - name: Restart nova-compute container
   vars:

ansible/roles/nova-cell/tasks/config.yml

@@ -97,6 +97,26 @@
     - libvirt_tls | bool
     - libvirt_tls_manage_certs | bool
 
+- name: Copying over libvirt SASL configuration
+  become: true
+  vars:
+    service_name: "{{ item.service }}"
+    service: "{{ nova_cell_services[service_name] }}"
+  template:
+    src: "{{ item.src }}"
+    dest: "{{ node_config_directory }}/{{ service_name }}/{{ item.dest }}"
+    mode: "0660"
+  when:
+    - libvirt_enable_sasl | bool
+    - inventory_hostname in groups[service.group]
+    - service.enabled | bool
+  with_items:
+    - { src: "auth.conf.j2", dest: "auth.conf", service: "nova-compute" }
+    - { src: "auth.conf.j2", dest: "auth.conf", service: "nova-libvirt" }
+    - { src: "sasl.conf.j2", dest: "sasl.conf", service: "nova-libvirt" }
+  notify:
+    - Restart {{ service_name }} container
+
 - name: Copying files for nova-ssh
   become: true
   vars:

ansible/roles/nova-cell/templates/auth.conf.j2

@@ -0,0 +1,6 @@
+[credentials-default]
+authname={{ libvirt_sasl_authname }}
+password={{ libvirt_sasl_password }}
+
+[auth-libvirt-default]
+credentials=default

ansible/roles/nova-cell/templates/libvirtd.conf.j2

@@ -5,10 +5,11 @@ tls_port = "{{ nova_libvirt_port }}"
 key_file = "/etc/pki/libvirt/private/serverkey.pem"
 cert_file = "/etc/pki/libvirt/servercert.pem"
 ca_file = "/etc/pki/CA/cacert.pem"
+auth_tls = "{{ 'sasl' if libvirt_enable_sasl | bool else 'none' }}"
 {% else %}
 listen_tcp = 1
 listen_tls = 0
-auth_tcp = "none"
+auth_tcp = "{{ 'sasl' if libvirt_enable_sasl | bool else 'none' }}"
 tcp_port = "{{ nova_libvirt_port }}"
 ca_file = ""
 {% endif %}

ansible/roles/nova-cell/templates/nova-compute.json.j2

@@ -55,7 +55,13 @@
             "owner": "nova",
             "perm": "0600",
             "optional": true
-        }
+        }{% if nova_compute_virt_type in ['kvm', 'qemu'] and libvirt_enable_sasl | bool %},
+        {
+            "source": "{{ container_config_directory }}/auth.conf",
+            "dest": "/var/lib/nova/.config/libvirt/auth.conf",
+            "owner": "nova",
+            "perm": "0600"
+        }{% endif %}
     ],
     "permissions": [
         {

ansible/roles/nova-cell/templates/nova-libvirt.json.j2

@@ -55,6 +55,18 @@
             "dest": "/etc/ceph/ceph.conf",
             "owner": "nova",
             "perm": "0600"
+        }{% endif %}{% if libvirt_enable_sasl | bool %},
+        {
+            "source": "{{ container_config_directory }}/sasl.conf",
+            "dest": "/etc/sasl2/libvirt.conf",
+            "owner": "root",
+            "perm": "0600"
+        },
+        {
+            "source": "{{ container_config_directory }}/auth.conf",
+            "dest": "/root/.config/libvirt/auth.conf",
+            "owner": "root",
+            "perm": "0600"
         }{% endif %}
     ]
 }

ansible/roles/nova-cell/templates/sasl.conf.j2

@@ -0,0 +1,2 @@
+mech_list: {{ libvirt_sasl_mech_list | join(' ') }}
+sasldb_path: /etc/libvirt/passwd.db

doc/source/reference/compute/libvirt-guide.rst

@@ -1,5 +1,3 @@
-.. libvirt-tls-guide:
-
 ====================================
 Libvirt - Nova Virtualisation Driver
 ====================================
@@ -23,16 +21,39 @@ hardware virtualisation (e.g. Virtualisation Technology (VT) BIOS configuration
 on Intel systems), ``qemu`` may be used to provide less performant
 software-emulated virtualisation.
 
+SASL Authentication
+===================
+
+The default configuration of Kolla Ansible is to run libvirt over TCP,
+authenticated with SASL. This should not be considered as providing a secure,
+encrypted channel, since the username/password SASL mechanisms available for
+TCP are no longer considered cryptographically secure. However, it does at
+least provide some authentication for the libvirt API. For a more secure
+encrypted channel, use :ref`libvirt TLS <libvirt-tls>`.
+
+SASL is enabled according to the ``libvirt_enable_sasl`` flag, which defaults
+to ``true``.
+
+The username is configured via ``libvirt_sasl_authname``, and defaults to
+``kolla``. The password is configured via ``libvirt_sasl_password``, and is
+generated with other passwords using and stored in ``passwords.yml``.
+
+The list of enabled authentication mechanisms is configured via
+``libvirt_sasl_mech_list``, and defaults to ``["SCRAM-SHA-256"]`` if libvirt
+TLS is enabled, or ``["DIGEST-MD5"]`` otherwise.
+
+.. libvirt-tls:
+
 Libvirt TLS
 ===========
 
 The default configuration of Kolla Ansible is to run libvirt over TCP, with
-authentication disabled. As long as one takes steps to protect who can access
+SASL authentication. As long as one takes steps to protect who can access
-the port this works well. However, in the case where you want live-migration to
+the network this works well. However, in a less trusted environment one may
-be allowed across hypervisors one may want to either add some level of
+want to use encryption when accessing the libvirt API. To do this we can enable
-authentication to the connections or make sure VM data is passed between
+TLS for libvirt and make nova use it. Mutual TLS is configured, providing
-hypervisors in a secure manner. To do this we can enable TLS for libvirt and
+authentication of clients via certificates. SASL authentication provides a
-make nova use it.
+further level of security.
 
 Using libvirt TLS
 ~~~~~~~~~~~~~~~~~

etc/kolla/passwords.yml

@@ -253,3 +253,8 @@ keystone_federation_openid_crypto_password:
 # Ceph RadosGW options
 ####################
 ceph_rgw_keystone_password:
+
+##################
+# libvirt options
+##################
+libvirt_sasl_password:

releasenotes/notes/libvirt-sasl-404199143610fb75.yaml

@@ -0,0 +1,27 @@
+---
+features:
+  - |
+    Adds support for libvirt SASL authentication. It is enabled by default.
+    `LP#1964013 <https://bugs.launchpad.net/kolla-ansible/+bug/1964013>`__
+security:
+  - |
+    Fixes an issue where the default configuration of libvirt did not use
+    authentication for the API exposed over TCP on the internal API network.
+    This allowed anyone with access to the internal API network read-write
+    access to libvirt. While the internal API network is typically trusted,
+    other services on this network generally at least require authentication.
+
+    SASL authentication is now enabled for libvirt by default. Kolla Ansible
+    supports libvirt TLS since the Train release, and this is recommended to
+    provide a higher level of security. `LP#1964013
+    <https://bugs.launchpad.net/kolla-ansible/+bug/1964013>`__
+upgrade:
+  - |
+    The addition of libvirt SASL authentication requires a new password in
+    ``passwords.yml``, ``libvirt_sasl_password``. This may be generated using
+    the existing ``kolla-genpwd`` and ``kolla-mergepwd`` tooling.
+  - |
+    The addition of libvirt SASL authentication requires both the
+    ``nova_libvirt`` and ``nova_compute`` containers to be updated
+    simultaneously, using new images with the necessary Cyrus SASL
+    dependencies, as well as configuration containing the SASL credentials.