Migrate to using keystoneauth Sessions

We currently construct Keystone client objects directly, which
is no longer the preferred way.  Instead, we should be using Sessions
which allows use of different auth plugins.  This change attempts to
migrate our Keystone usage to this model.

Additionally, we currently rely on the imported keystonemiddleware
auth_token's configuration for all of the Keystone credentials used
by the Ironic service user.  This is bad, as that config is internal
to that library and may change at any time.  Also, the service user
may be using different credentials than the token validator.

This refactors the keystone module to use Sessions.
It attempts to provide some backward compat for users
who have not yet updated their config,
by falling back to the authtoken config section when required.

Operators impact:

- Authentification parameters for each service now should specified in
  the corresponding config section for this service ([glance], [neutron]
  [swift], [inspector]).
  This includes providing both Keystone session-related options
  (timeout, SSL-related ones) and authentification options
  (`auth_type`, `auth_url` and proper options for the auth plugin).

- New config section `service_catalog` for Ironic service user
  credentials, used to resolve Ironic API URL from Keystone catalog.

- If loading from the service config section fails, an attempt is made
  to use respective options from [keystone_authtoken] section as a
  fall-back for backward compatibility.

Implementation details:

- using keystoneauth1 library instead of keystoneclient

- For each service the keystone session is created only once and is
  reused further. This lowers the number of authentification requests
  made to Keystone but implies that only auth plugins that can
  re-authentificate themselves can be used (so no *Token plugins).

This patch does not update the DevStack plugin, in order to test
backwards compatibility with old config options.
DevStack plugin will be modified in a subsequent patch.

Change-Id: I166eebefc1e1335a1a7b632149cf6441512e9d5e
Closes-Bug: #1422632
Related-Bug: #1418341
Related-Bug: #1494776
Co-Authored-By: Adam Gandelman <adamg@ubuntu.com>

etc/ironic/ironic.conf.sample

@@ -977,9 +977,141 @@
 # value)
 #allowed_direct_url_schemes =
 
-# The secret token given to Swift to allow temporary URL
+# Authentication URL (string value)
-# downloads. Required for temporary URLs. (string value)
+#auth_url = <None>
-#swift_temp_url_key = <None>
+
+# Authentication strategy to use when connecting to glance.
+# (string value)
+# Allowed values: keystone, noauth
+#auth_strategy = keystone
+
+# Authentication type to load (string value)
+# Deprecated group/name - [glance]/auth_plugin
+#auth_type = <None>
+
+# PEM encoded Certificate Authority to use when verifying
+# HTTPs connections. (string value)
+#cafile = <None>
+
+# PEM encoded client certificate cert file (string value)
+#certfile = <None>
+
+# Optional domain ID to use with v3 and v2 parameters. It will
+# be used for both the user and project domain in v3 and
+# ignored in v2 authentication. (string value)
+#default_domain_id = <None>
+
+# Optional domain name to use with v3 API and v2 parameters.
+# It will be used for both the user and project domain in v3
+# and ignored in v2 authentication. (string value)
+#default_domain_name = <None>
+
+# Domain ID to scope to (string value)
+#domain_id = <None>
+
+# Domain name to scope to (string value)
+#domain_name = <None>
+
+# Allow to perform insecure SSL (https) requests to glance.
+# (boolean value)
+#glance_api_insecure = false
+
+# A list of the glance api servers available to ironic. Prefix
+# with https:// for SSL-based glance API servers. Format is
+# [hostname|IP]:port. (list value)
+#glance_api_servers = <None>
+
+# Optional path to a CA certificate bundle to be used to
+# validate the SSL certificate served by glance. It is used
+# when glance_api_insecure is set to False. (string value)
+#glance_cafile = <None>
+
+# Default glance hostname or IP address. (string value)
+#glance_host = $my_ip
+
+# Number of retries when downloading an image from glance.
+# (integer value)
+#glance_num_retries = 0
+
+# Default glance port. (port value)
+# Minimum value: 0
+# Maximum value: 65535
+#glance_port = 9292
+
+# Default protocol to use when connecting to glance. Set to
+# https for SSL. (string value)
+# Allowed values: http, https
+#glance_protocol = http
+
+# Verify HTTPS connections. (boolean value)
+#insecure = false
+
+# PEM encoded client certificate key file (string value)
+#keyfile = <None>
+
+# User's password (string value)
+#password = <None>
+
+# Domain ID containing project (string value)
+#project_domain_id = <None>
+
+# Domain name containing project (string value)
+#project_domain_name = <None>
+
+# Project ID to scope to (string value)
+# Deprecated group/name - [glance]/tenant-id
+#project_id = <None>
+
+# Project name to scope to (string value)
+# Deprecated group/name - [glance]/tenant-name
+#project_name = <None>
+
+# The account that Glance uses to communicate with Swift. The
+# format is "AUTH_uuid". "uuid" is the UUID for the account
+# configured in the glance-api.conf. Required for temporary
+# URLs when Glance backend is Swift. For example:
+# "AUTH_a422b2-91f3-2f46-74b7-d7c9e8958f5d30". Swift temporary
+# URL format:
+# "endpoint_url/api_version/[account/]container/object_id"
+# (string value)
+#swift_account = <None>
+
+# The Swift API version to create a temporary URL for.
+# Defaults to "v1". Swift temporary URL format:
+# "endpoint_url/api_version/[account/]container/object_id"
+# (string value)
+#swift_api_version = v1
+
+# The Swift container Glance is configured to store its images
+# in. Defaults to "glance", which is the default in glance-
+# api.conf. Swift temporary URL format:
+# "endpoint_url/api_version/[account/]container/object_id"
+# (string value)
+#swift_container = glance
+
+# The "endpoint" (scheme, hostname, optional port) for the
+# Swift URL of the form
+# "endpoint_url/api_version/[account/]container/object_id". Do
+# not include trailing "/". For example, use
+# "https://swift.example.com". If using RADOS Gateway,
+# endpoint may also contain /swift path; if it does not, it
+# will be appended. Required for temporary URLs. (string
+# value)
+#swift_endpoint_url = <None>
+
+# This should match a config by the same name in the Glance
+# configuration file. When set to 0, a single-tenant store
+# will only use one container to store all images. When set to
+# an integer value between 1 and 32, a single-tenant store
+# will use multiple containers to store images, and this value
+# will determine how many containers are created. (integer
+# value)
+#swift_store_multiple_containers_seed = 0
+
+# Whether to cache generated Swift temporary URLs. Setting it
+# to true is only useful when an image caching proxy is used.
+# Defaults to False. (boolean value)
+#swift_temp_url_cache_enabled = false
 
 # The length of time in seconds that the temporary URL will be
 # valid for. Defaults to 20 minutes. If some deploys get a 401
@@ -989,11 +1121,6 @@
 # swift_temp_url_expected_download_start_delay (integer value)
 #swift_temp_url_duration = 1200
 
-# Whether to cache generated Swift temporary URLs. Setting it
-# to true is only useful when an image caching proxy is used.
-# Defaults to False. (boolean value)
-#swift_temp_url_cache_enabled = false
-
 # This is the delay (in seconds) from the time of the deploy
 # request (when the Swift temporary URL is generated) to when
 # the IPA ramdisk starts up and URL is used for the image
@@ -1007,47 +1134,9 @@
 # Minimum value: 0
 #swift_temp_url_expected_download_start_delay = 0
 
-# The "endpoint" (scheme, hostname, optional port) for the
+# The secret token given to Swift to allow temporary URL
-# Swift URL of the form
+# downloads. Required for temporary URLs. (string value)
-# "endpoint_url/api_version/[account/]container/object_id". Do
+#swift_temp_url_key = <None>
-# not include trailing "/". For example, use
-# "https://swift.example.com". If using RADOS Gateway,
-# endpoint may also contain /swift path; if it does not, it
-# will be appended. Required for temporary URLs. (string
-# value)
-#swift_endpoint_url = <None>
-
-# The Swift API version to create a temporary URL for.
-# Defaults to "v1". Swift temporary URL format:
-# "endpoint_url/api_version/[account/]container/object_id"
-# (string value)
-#swift_api_version = v1
-
-# The account that Glance uses to communicate with Swift. The
-# format is "AUTH_uuid". "uuid" is the UUID for the account
-# configured in the glance-api.conf. Required for temporary
-# URLs when Glance backend is Swift. For example:
-# "AUTH_a422b2-91f3-2f46-74b7-d7c9e8958f5d30". Swift temporary
-# URL format:
-# "endpoint_url/api_version/[account/]container/object_id"
-# (string value)
-#swift_account = <None>
-
-# The Swift container Glance is configured to store its images
-# in. Defaults to "glance", which is the default in glance-
-# api.conf. Swift temporary URL format:
-# "endpoint_url/api_version/[account/]container/object_id"
-# (string value)
-#swift_container = glance
-
-# This should match a config by the same name in the Glance
-# configuration file. When set to 0, a single-tenant store
-# will only use one container to store all images. When set to
-# an integer value between 1 and 32, a single-tenant store
-# will use multiple containers to store images, and this value
-# will determine how many containers are created. (integer
-# value)
-#swift_store_multiple_containers_seed = 0
 
 # Type of endpoint to use for temporary URLs. If the Glance
 # backend is Swift, use "swift"; if it is CEPH with RADOS
@@ -1055,41 +1144,30 @@
 # Allowed values: swift, radosgw
 #temp_url_endpoint_type = swift
 
-# Default glance hostname or IP address. (string value)
+# Tenant ID (string value)
-#glance_host = $my_ip
+#tenant_id = <None>
 
-# Default glance port. (port value)
+# Tenant Name (string value)
-# Minimum value: 0
+#tenant_name = <None>
-# Maximum value: 65535
-#glance_port = 9292
 
-# Default protocol to use when connecting to glance. Set to
+# Timeout value for http requests (integer value)
-# https for SSL. (string value)
+#timeout = <None>
-# Allowed values: http, https
-#glance_protocol = http
 
-# A list of the glance api servers available to ironic. Prefix
+# Trust ID (string value)
-# with https:// for SSL-based glance API servers. Format is
+#trust_id = <None>
-# [hostname|IP]:port. (list value)
-#glance_api_servers = <None>
 
-# Allow to perform insecure SSL (https) requests to glance.
+# User's domain id (string value)
-# (boolean value)
+#user_domain_id = <None>
-#glance_api_insecure = false
 
-# Number of retries when downloading an image from glance.
+# User's domain name (string value)
-# (integer value)
+#user_domain_name = <None>
-#glance_num_retries = 0
 
-# Authentication strategy to use when connecting to glance.
+# User id (string value)
-# (string value)
+#user_id = <None>
-# Allowed values: keystone, noauth
-#auth_strategy = keystone
 
-# Optional path to a CA certificate bundle to be used to
+# Username (string value)
-# validate the SSL certificate served by glance. It is used
+# Deprecated group/name - [glance]/user-name
-# when glance_api_insecure is set to False. (string value)
+#username = <None>
-#glance_cafile = <None>
 
 
 [iboot]
@@ -1189,10 +1267,63 @@
 # From ironic
 #
 
+# Authentication URL (string value)
+#auth_url = <None>
+
+# Authentication type to load (string value)
+# Deprecated group/name - [inspector]/auth_plugin
+#auth_type = <None>
+
+# PEM encoded Certificate Authority to use when verifying
+# HTTPs connections. (string value)
+#cafile = <None>
+
+# PEM encoded client certificate cert file (string value)
+#certfile = <None>
+
+# Optional domain ID to use with v3 and v2 parameters. It will
+# be used for both the user and project domain in v3 and
+# ignored in v2 authentication. (string value)
+#default_domain_id = <None>
+
+# Optional domain name to use with v3 API and v2 parameters.
+# It will be used for both the user and project domain in v3
+# and ignored in v2 authentication. (string value)
+#default_domain_name = <None>
+
+# Domain ID to scope to (string value)
+#domain_id = <None>
+
+# Domain name to scope to (string value)
+#domain_name = <None>
+
 # whether to enable inspection using ironic-inspector (boolean
 # value)
 #enabled = false
 
+# Verify HTTPS connections. (boolean value)
+#insecure = false
+
+# PEM encoded client certificate key file (string value)
+#keyfile = <None>
+
+# User's password (string value)
+#password = <None>
+
+# Domain ID containing project (string value)
+#project_domain_id = <None>
+
+# Domain name containing project (string value)
+#project_domain_name = <None>
+
+# Project ID to scope to (string value)
+# Deprecated group/name - [inspector]/tenant-id
+#project_id = <None>
+
+# Project name to scope to (string value)
+# Deprecated group/name - [inspector]/tenant-name
+#project_name = <None>
+
 # ironic-inspector HTTP endpoint. If this is not set, the
 # ironic-inspector client default (http://127.0.0.1:5050) will
 # be used. (string value)
@@ -1202,6 +1333,31 @@
 # (integer value)
 #status_check_period = 60
 
+# Tenant ID (string value)
+#tenant_id = <None>
+
+# Tenant Name (string value)
+#tenant_name = <None>
+
+# Timeout value for http requests (integer value)
+#timeout = <None>
+
+# Trust ID (string value)
+#trust_id = <None>
+
+# User's domain id (string value)
+#user_domain_id = <None>
+
+# User's domain name (string value)
+#user_domain_name = <None>
+
+# User id (string value)
+#user_id = <None>
+
+# Username (string value)
+# Deprecated group/name - [inspector]/user-name
+#username = <None>
+
 
 [ipmi]
 
@@ -1631,21 +1787,8 @@
 # From ironic
 #
 
-# URL for connecting to neutron. (string value)
+# Authentication URL (string value)
-#url = http://$my_ip:9696
+#auth_url = <None>
-
-# Timeout value for connecting to neutron in seconds. (integer
-# value)
-#url_timeout = 30
-
-# Delay value to wait for Neutron agents to setup sufficient
-# DHCP configuration for port. (integer value)
-# Minimum value: 0
-#port_setup_delay = 0
-
-# Client retries in the case of a failed request. (integer
-# value)
-#retries = 3
 
 # Authentication strategy to use when connecting to neutron.
 # Running neutron in noauth mode (related to but not affected
@@ -1654,17 +1797,111 @@
 # Allowed values: keystone, noauth
 #auth_strategy = keystone
 
+# Authentication type to load (string value)
+# Deprecated group/name - [neutron]/auth_plugin
+#auth_type = <None>
+
+# PEM encoded Certificate Authority to use when verifying
+# HTTPs connections. (string value)
+#cafile = <None>
+
+# PEM encoded client certificate cert file (string value)
+#certfile = <None>
+
 # Neutron network UUID for the ramdisk to be booted into for
 # cleaning nodes. Required for "neutron" network interface. It
 # is also required if cleaning nodes when using "flat" network
 # interface or "neutron" DHCP provider. (string value)
 #cleaning_network_uuid = <None>
 
+# Optional domain ID to use with v3 and v2 parameters. It will
+# be used for both the user and project domain in v3 and
+# ignored in v2 authentication. (string value)
+#default_domain_id = <None>
+
+# Optional domain name to use with v3 API and v2 parameters.
+# It will be used for both the user and project domain in v3
+# and ignored in v2 authentication. (string value)
+#default_domain_name = <None>
+
+# Domain ID to scope to (string value)
+#domain_id = <None>
+
+# Domain name to scope to (string value)
+#domain_name = <None>
+
+# Verify HTTPS connections. (boolean value)
+#insecure = false
+
+# PEM encoded client certificate key file (string value)
+#keyfile = <None>
+
+# User's password (string value)
+#password = <None>
+
+# Delay value to wait for Neutron agents to setup sufficient
+# DHCP configuration for port. (integer value)
+# Minimum value: 0
+#port_setup_delay = 0
+
+# Domain ID containing project (string value)
+#project_domain_id = <None>
+
+# Domain name containing project (string value)
+#project_domain_name = <None>
+
+# Project ID to scope to (string value)
+# Deprecated group/name - [neutron]/tenant-id
+#project_id = <None>
+
+# Project name to scope to (string value)
+# Deprecated group/name - [neutron]/tenant-name
+#project_name = <None>
+
 # Neutron network UUID for the ramdisk to be booted into for
 # provisioning nodes. Required for "neutron" network
 # interface. (string value)
 #provisioning_network_uuid = <None>
 
+# Client retries in the case of a failed request. (integer
+# value)
+#retries = 3
+
+# Tenant ID (string value)
+#tenant_id = <None>
+
+# Tenant Name (string value)
+#tenant_name = <None>
+
+# Timeout value for http requests (integer value)
+#timeout = <None>
+
+# Trust ID (string value)
+#trust_id = <None>
+
+# URL for connecting to neutron. Default value translates to
+# 'http://$my_ip:9696' when auth_strategy is 'noauth', and to
+# discovery from Keystone catalog when auth_strategy is
+# 'keystone'. (string value)
+#url = <None>
+
+# Timeout value for connecting to neutron in seconds. (integer
+# value)
+#url_timeout = 30
+
+# User's domain id (string value)
+#user_domain_id = <None>
+
+# User's domain name (string value)
+#user_domain_name = <None>
+
+# User id (string value)
+#user_id = <None>
+
+# Username (string value)
+# Deprecated group/name - [neutron]/user-name
+#username = <None>
+
 
 [oneview]
 
@@ -2213,6 +2450,91 @@
 #action_timeout = 10
 
 
+[service_catalog]
+
+#
+# From ironic
+#
+
+# Authentication URL (string value)
+#auth_url = <None>
+
+# Authentication type to load (string value)
+# Deprecated group/name - [service_catalog]/auth_plugin
+#auth_type = <None>
+
+# PEM encoded Certificate Authority to use when verifying
+# HTTPs connections. (string value)
+#cafile = <None>
+
+# PEM encoded client certificate cert file (string value)
+#certfile = <None>
+
+# Optional domain ID to use with v3 and v2 parameters. It will
+# be used for both the user and project domain in v3 and
+# ignored in v2 authentication. (string value)
+#default_domain_id = <None>
+
+# Optional domain name to use with v3 API and v2 parameters.
+# It will be used for both the user and project domain in v3
+# and ignored in v2 authentication. (string value)
+#default_domain_name = <None>
+
+# Domain ID to scope to (string value)
+#domain_id = <None>
+
+# Domain name to scope to (string value)
+#domain_name = <None>
+
+# Verify HTTPS connections. (boolean value)
+#insecure = false
+
+# PEM encoded client certificate key file (string value)
+#keyfile = <None>
+
+# User's password (string value)
+#password = <None>
+
+# Domain ID containing project (string value)
+#project_domain_id = <None>
+
+# Domain name containing project (string value)
+#project_domain_name = <None>
+
+# Project ID to scope to (string value)
+# Deprecated group/name - [service_catalog]/tenant-id
+#project_id = <None>
+
+# Project name to scope to (string value)
+# Deprecated group/name - [service_catalog]/tenant-name
+#project_name = <None>
+
+# Tenant ID (string value)
+#tenant_id = <None>
+
+# Tenant Name (string value)
+#tenant_name = <None>
+
+# Timeout value for http requests (integer value)
+#timeout = <None>
+
+# Trust ID (string value)
+#trust_id = <None>
+
+# User's domain id (string value)
+#user_domain_id = <None>
+
+# User's domain name (string value)
+#user_domain_name = <None>
+
+# User id (string value)
+#user_id = <None>
+
+# Username (string value)
+# Deprecated group/name - [service_catalog]/user-name
+#username = <None>
+
+
 [snmp]
 
 #
@@ -2285,10 +2607,88 @@
 # From ironic
 #
 
+# Authentication URL (string value)
+#auth_url = <None>
+
+# Authentication type to load (string value)
+# Deprecated group/name - [swift]/auth_plugin
+#auth_type = <None>
+
+# PEM encoded Certificate Authority to use when verifying
+# HTTPs connections. (string value)
+#cafile = <None>
+
+# PEM encoded client certificate cert file (string value)
+#certfile = <None>
+
+# Optional domain ID to use with v3 and v2 parameters. It will
+# be used for both the user and project domain in v3 and
+# ignored in v2 authentication. (string value)
+#default_domain_id = <None>
+
+# Optional domain name to use with v3 API and v2 parameters.
+# It will be used for both the user and project domain in v3
+# and ignored in v2 authentication. (string value)
+#default_domain_name = <None>
+
+# Domain ID to scope to (string value)
+#domain_id = <None>
+
+# Domain name to scope to (string value)
+#domain_name = <None>
+
+# Verify HTTPS connections. (boolean value)
+#insecure = false
+
+# PEM encoded client certificate key file (string value)
+#keyfile = <None>
+
+# User's password (string value)
+#password = <None>
+
+# Domain ID containing project (string value)
+#project_domain_id = <None>
+
+# Domain name containing project (string value)
+#project_domain_name = <None>
+
+# Project ID to scope to (string value)
+# Deprecated group/name - [swift]/tenant-id
+#project_id = <None>
+
+# Project name to scope to (string value)
+# Deprecated group/name - [swift]/tenant-name
+#project_name = <None>
+
 # Maximum number of times to retry a Swift request, before
 # failing. (integer value)
 #swift_max_retries = 2
 
+# Tenant ID (string value)
+#tenant_id = <None>
+
+# Tenant Name (string value)
+#tenant_name = <None>
+
+# Timeout value for http requests (integer value)
+#timeout = <None>
+
+# Trust ID (string value)
+#trust_id = <None>
+
+# User's domain id (string value)
+#user_domain_id = <None>
+
+# User's domain name (string value)
+#user_domain_name = <None>
+
+# User id (string value)
+#user_id = <None>
+
+# Username (string value)
+# Deprecated group/name - [swift]/user-name
+#username = <None>
+
 
 [virtualbox]
 

ironic/cmd/conductor.py

@@ -22,12 +22,40 @@ The Ironic Management Service
 import sys
 
 from oslo_config import cfg
+from oslo_log import log
 from oslo_service import service
 
+from ironic.common.i18n import _LW
 from ironic.common import service as ironic_service
+from ironic.conf import auth
 
 CONF = cfg.CONF
 
+LOG = log.getLogger(__name__)
+
+SECTIONS_WITH_AUTH = (
+    'service_catalog', 'neutron', 'glance', 'swift', 'inspector')
+
+
+# TODO(pas-ha) remove this check after deprecation period
+def _check_auth_options(conf):
+    missing = []
+    for section in SECTIONS_WITH_AUTH:
+            if not auth.load_auth(conf, section):
+                missing.append('[%s]' % section)
+    if missing:
+        link = "http://docs.openstack.org/releasenotes/ironic/newton.html"
+        LOG.warning(_LW("Failed to load authentification credentials from "
+                        "%(missing)s config sections. "
+                        "The corresponding service users' credentials "
+                        "will be loaded from [%(old)s] config section, "
+                        "which is deprecated for this purpose. "
+                        "Please update the config file. "
+                        "For more info see %(link)s."),
+                    dict(missing=", ".join(missing),
+                         old=auth.LEGACY_SECTION,
+                         link=link))
+
 
 def main():
     # Parse config file and command line options, then start logging
@@ -37,6 +65,8 @@ def main():
                                     'ironic.conductor.manager',
                                     'ConductorManager')
 
+    _check_auth_options(CONF)
+
     launcher = service.launch(CONF, mgr)
     launcher.wait()
 

ironic/common/image_service.py

@@ -35,9 +35,14 @@ from ironic.conf import CONF
 
 IMAGE_CHUNK_SIZE = 1024 * 1024  # 1mb
 
-# TODO(rama_y): This import should be removed,
+_GLANCE_SESSION = None
-# once https://review.openstack.org/#/c/309070 is merged.
+
-CONF.import_opt('my_ip', 'ironic.netconf')
+
+def _get_glance_session():
+    global _GLANCE_SESSION
+    if not _GLANCE_SESSION:
+        _GLANCE_SESSION = keystone.get_session('glance')
+    return _GLANCE_SESSION
 
 
 def import_versioned_module(version, submodule=None):
@@ -52,7 +57,8 @@ def GlanceImageService(client=None, version=1, context=None):
     service_class = getattr(module, 'GlanceImageService')
     if (context is not None and CONF.glance.auth_strategy == 'keystone'
         and not context.auth_token):
-        context.auth_token = keystone.get_admin_auth_token()
+            session = _get_glance_session()
+            context.auth_token = keystone.get_admin_auth_token(session)
     return service_class(client, version, context)
 
 

ironic/common/keystone.py

@@ -12,132 +12,125 @@
 # License for the specific language governing permissions and limitations
 # under the License.
 
-from keystoneclient import exceptions as ksexception
+"""Central place for handling Keystone authorization and service lookup."""
-from oslo_concurrency import lockutils
+
-from six.moves.urllib import parse
+from keystoneauth1 import exceptions as kaexception
+from keystoneauth1 import loading as kaloading
+from oslo_log import log as logging
+import six
+from six.moves.urllib import parse  # for legacy options loading only
 
 from ironic.common import exception
 from ironic.common.i18n import _
+from ironic.common.i18n import _LE
+from ironic.conf import auth as ironic_auth
 from ironic.conf import CONF
 
-CONF.import_group('keystone_authtoken', 'keystonemiddleware.auth_token')
 
-_KS_CLIENT = None
+LOG = logging.getLogger(__name__)
 
 
+# FIXME(pas-ha): for backward compat with legacy options loading only
 def _is_apiv3(auth_url, auth_version):
-    """Checks if V3 version of API is being used or not.
+    """Check if V3 version of API is being used or not.
 
     This method inspects auth_url and auth_version, and checks whether V3
     version of the API is being used or not.
-
+    When no auth_version is specified and auth_url is not a versioned
+    endpoint, v2.0 is assumed.
     :param auth_url: a http or https url to be inspected (like
         'http://127.0.0.1:9898/').
     :param auth_version: a string containing the version (like 'v2', 'v3.0')
+                         or None
     :returns: True if V3 of the API is being used.
     """
     return auth_version == 'v3.0' or '/v3' in parse.urlparse(auth_url).path
 
 
-def _get_ksclient(token=None):
+def ks_exceptions(f):
-    auth_url = CONF.keystone_authtoken.auth_uri
+    """Wraps keystoneclient functions and centralizes exception handling."""
-    if not auth_url:
+    @six.wraps(f)
-        raise exception.KeystoneFailure(_('Keystone API endpoint is missing'))
+    def wrapper(*args, **kwargs):
-
-    auth_version = CONF.keystone_authtoken.auth_version
-    api_v3 = _is_apiv3(auth_url, auth_version)
-
-    if api_v3:
-        from keystoneclient.v3 import client
-    else:
-        from keystoneclient.v2_0 import client
-
-    auth_url = get_keystone_url(auth_url, auth_version)
         try:
-        if token:
+            return f(*args, **kwargs)
-            return client.Client(token=token, auth_url=auth_url)
+        except kaexception.EndpointNotFound:
-        else:
+            service_type = kwargs.get('service_type', 'baremetal')
-            params = {'username': CONF.keystone_authtoken.admin_user,
+            endpoint_type = kwargs.get('endpoint_type', 'internal')
-                      'password': CONF.keystone_authtoken.admin_password,
+            raise exception.CatalogNotFound(
-                      'tenant_name': CONF.keystone_authtoken.admin_tenant_name,
+                service_type=service_type, endpoint_type=endpoint_type)
-                      'region_name': CONF.keystone.region_name,
+        except (kaexception.Unauthorized, kaexception.AuthorizationFailure):
-                      'auth_url': auth_url}
-            return _get_ksclient_from_conf(client, **params)
-    except ksexception.Unauthorized:
             raise exception.KeystoneUnauthorized()
-    except ksexception.AuthorizationFailure as err:
+        except (kaexception.NoMatchingPlugin,
-        raise exception.KeystoneFailure(_('Could not authorize in Keystone:'
+                kaexception.MissingRequiredOptions) as e:
-                                          ' %s') % err)
+            raise exception.ConfigInvalid(six.text_type(e))
+        except Exception as e:
+            LOG.exception(_LE('Keystone request failed: %(msg)s'),
+                          {'msg': six.text_type(e)})
+            raise exception.KeystoneFailure(six.text_type(e))
+    return wrapper
 
 
-@lockutils.synchronized('keystone_client', 'ironic-')
+@ks_exceptions
-def _get_ksclient_from_conf(client, **params):
+def get_session(group):
-    global _KS_CLIENT
+    auth = ironic_auth.load_auth(CONF, group) or _get_legacy_auth()
-    # NOTE(yuriyz): use Keystone client default gap, to determine whether the
+    if not auth:
-    # given token is about to expire
+        msg = _("Failed to load auth from either [%(new)s] or [%(old)s] "
-    if _KS_CLIENT is None or _KS_CLIENT.auth_ref.will_expire_soon():
+                "config sections.")
-        _KS_CLIENT = client.Client(**params)
+        raise exception.ConfigInvalid(message=msg, new=group,
-    return _KS_CLIENT
+                                      old=ironic_auth.LEGACY_SECTION)
+    session = kaloading.load_session_from_conf_options(
+        CONF, group, auth=auth)
+    return session
 
 
-def get_keystone_url(auth_url, auth_version):
+# FIXME(pas-ha) remove legacy path after deprecation
-    """Gives an http/https url to contact keystone.
+def _get_legacy_auth():
+    """Load auth from keystone_authtoken config section
 
-    Given an auth_url and auth_version, this method generates the url in
+    Used only to provide backward compatibility with old configs.
-    which keystone can be reached.
-
-    :param auth_url: a http or https url to be inspected (like
-        'http://127.0.0.1:9898/').
-    :param auth_version: a string containing the version (like v2, v3.0, etc)
-    :returns: a string containing the keystone url
     """
-    api_v3 = _is_apiv3(auth_url, auth_version)
+    conf = getattr(CONF, ironic_auth.LEGACY_SECTION)
-    api_version = 'v3' if api_v3 else 'v2.0'
+    legacy_loader = kaloading.get_plugin_loader('password')
-    # NOTE(lucasagomes): Get rid of the trailing '/' otherwise urljoin()
+    auth_params = {
-    #   fails to override the version in the URL
+        'auth_url': conf.auth_uri,
-    return parse.urljoin(auth_url.rstrip('/'), api_version)
+        'username': conf.admin_user,
+        'password': conf.admin_password,
+        'tenant_name': conf.admin_tenant_name
+    }
+    api_v3 = _is_apiv3(conf.auth_uri, conf.auth_version)
+    if api_v3:
+        # NOTE(pas-ha): mimic defaults of keystoneclient
+        auth_params.update({
+            'project_domain_id': 'default',
+            'user_domain_id': 'default',
+        })
+    return legacy_loader.load_from_options(**auth_params)
 
 
-def get_service_url(service_type='baremetal', endpoint_type='internal'):
+@ks_exceptions
+def get_service_url(session, service_type='baremetal',
+                    endpoint_type='internal'):
     """Wrapper for get service url from keystone service catalog.
 
-    Given a service_type and an endpoint_type, this method queries keystone
+    Given a service_type and an endpoint_type, this method queries
-    service catalog and provides the url for the desired endpoint.
+    keystone service catalog and provides the url for the desired
+    endpoint.
 
     :param service_type: the keystone service for which url is required.
     :param endpoint_type: the type of endpoint for the service.
     :returns: an http/https url for the desired endpoint.
     """
-    ksclient = _get_ksclient()
+    return session.get_endpoint(service_type=service_type,
-
+                                interface_type=endpoint_type,
-    if not ksclient.has_service_catalog():
+                                region=CONF.keystone.region_name)
-        raise exception.KeystoneFailure(_('No Keystone service catalog '
-                                          'loaded'))
-
-    try:
-        endpoint = ksclient.service_catalog.url_for(
-            service_type=service_type,
-            endpoint_type=endpoint_type,
-            region_name=CONF.keystone.region_name)
-
-    except ksexception.EndpointNotFound:
-        raise exception.CatalogNotFound(service_type=service_type,
-                                        endpoint_type=endpoint_type)
-
-    return endpoint
 
 
-def get_admin_auth_token():
+@ks_exceptions
-    """Get an admin auth_token from the Keystone."""
+def get_admin_auth_token(session):
-    ksclient = _get_ksclient()
+    """Get admin token.
-    return ksclient.auth_token
 
-
+    Currently used for inspector, glance and swift clients.
-def token_expires_soon(token, duration=None):
+    Only swift client does not actually support using sessions directly,
-    """Determines if token expiration is about to occur.
+    LP #1518938, others will be updated in ironic code.
-
-    :param duration: time interval in seconds
-    :returns: boolean : true if expiration is within the given duration
     """
-    ksclient = _get_ksclient(token=token)
+    return session.get_token()
-    return ksclient.auth_ref.will_expire_soon(stale_duration=duration)

ironic/common/neutron.py

@@ -24,29 +24,49 @@ from ironic.conf import CONF
 
 LOG = log.getLogger(__name__)
 
+DEFAULT_NEUTRON_URL = 'http://%s:9696' % CONF.my_ip
+
+_NEUTRON_SESSION = None
+
+
+def _get_neutron_session():
+    global _NEUTRON_SESSION
+    if not _NEUTRON_SESSION:
+        _NEUTRON_SESSION = keystone.get_session('neutron')
+    return _NEUTRON_SESSION
+
 
 def get_client(token=None):
-    params = {
+    params = {'retries': CONF.neutron.retries}
-        'timeout': CONF.neutron.url_timeout,
+    url = CONF.neutron.url
-        'retries': CONF.neutron.retries,
-        'insecure': CONF.keystone_authtoken.insecure,
-        'ca_cert': CONF.keystone_authtoken.certfile,
-    }
-
     if CONF.neutron.auth_strategy == 'noauth':
-        params['endpoint_url'] = CONF.neutron.url
+        params['endpoint_url'] = url or DEFAULT_NEUTRON_URL
         params['auth_strategy'] = 'noauth'
+        params.update({
+            'timeout': CONF.neutron.url_timeout or CONF.neutron.timeout,
+            'insecure': CONF.neutron.insecure,
+            'ca_cert': CONF.neutron.cafile})
+    else:
+        session = _get_neutron_session()
+        if token is None:
+            params['session'] = session
+            # NOTE(pas-ha) endpoint_override==None will auto-discover
+            # endpoint from Keystone catalog.
+            # Region is needed only in this case.
+            # SSL related options are ignored as they are already embedded
+            # in keystoneauth Session object
+            if url:
+                params['endpoint_override'] = url
             else:
-        params['endpoint_url'] = (
-            CONF.neutron.url or
-            keystone.get_service_url(service_type='network'))
-        params['username'] = CONF.keystone_authtoken.admin_user
-        params['tenant_name'] = CONF.keystone_authtoken.admin_tenant_name
-        params['password'] = CONF.keystone_authtoken.admin_password
-        params['auth_url'] = (CONF.keystone_authtoken.auth_uri or '')
-        if CONF.keystone.region_name:
                 params['region_name'] = CONF.keystone.region_name
+        else:
             params['token'] = token
+            params['endpoint_url'] = url or keystone.get_service_url(
+                session, service_type='network')
+            params.update({
+                'timeout': CONF.neutron.url_timeout or CONF.neutron.timeout,
+                'insecure': CONF.neutron.insecure,
+                'ca_cert': CONF.neutron.cafile})
 
     return clientv20.Client(**params)
 

ironic/common/service.py

@@ -108,7 +108,6 @@ def prepare_service(argv=None):
                                          'qpid.messaging=INFO',
                                          'oslo_messaging=INFO',
                                          'sqlalchemy=WARNING',
-                                         'keystoneclient=INFO',
                                          'stevedore=INFO',
                                          'eventlet.wsgi.server=INFO',
                                          'iso8601=WARNING',

ironic/common/swift.py

@@ -14,6 +14,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
+import six
 from six.moves import http_client
 from six.moves.urllib import parse
 from swiftclient import client as swift_client
@@ -25,60 +26,39 @@ from ironic.common.i18n import _
 from ironic.common import keystone
 from ironic.conf import CONF
 
-CONF.import_opt('admin_user', 'keystonemiddleware.auth_token',
+
-                group='keystone_authtoken')
+_SWIFT_SESSION = None
-CONF.import_opt('admin_tenant_name', 'keystonemiddleware.auth_token',
+
-                group='keystone_authtoken')
+
-CONF.import_opt('admin_password', 'keystonemiddleware.auth_token',
+def _get_swift_session():
-                group='keystone_authtoken')
+    global _SWIFT_SESSION
-CONF.import_opt('auth_uri', 'keystonemiddleware.auth_token',
+    if not _SWIFT_SESSION:
-                group='keystone_authtoken')
+        _SWIFT_SESSION = keystone.get_session('swift')
-CONF.import_opt('auth_version', 'keystonemiddleware.auth_token',
+    return _SWIFT_SESSION
-                group='keystone_authtoken')
-CONF.import_opt('insecure', 'keystonemiddleware.auth_token',
-                group='keystone_authtoken')
-CONF.import_opt('cafile', 'keystonemiddleware.auth_token',
-                group='keystone_authtoken')
-CONF.import_opt('region_name', 'keystonemiddleware.auth_token',
-                group='keystone_authtoken')
 
 
 class SwiftAPI(object):
     """API for communicating with Swift."""
 
-    def __init__(self,
+    def __init__(self):
-                 user=None,
+        # TODO(pas-ha): swiftclient does not support keystone sessions ATM.
-                 tenant_name=None,
+        # Must be reworked when LP bug #1518938 is fixed.
-                 key=None,
+        session = _get_swift_session()
-                 auth_url=None,
+        params = {
-                 auth_version=None,
+            'retries': CONF.swift.swift_max_retries,
-                 region_name=None):
+            'preauthurl': keystone.get_service_url(
-        """Constructor for creating a SwiftAPI object.
+                session,
-
+                service_type='object-store'),
-        :param user: the name of the user for Swift account
+            'preauthtoken': keystone.get_admin_auth_token(session)
-        :param tenant_name: the name of the tenant for Swift account
+        }
-        :param key: the 'password' or key to authenticate with
+        # NOTE(pas-ha):session.verify is for HTTPS urls and can be
-        :param auth_url: the url for authentication
+        # - False (do not verify)
-        :param auth_version: the version of api to use for authentication
+        # - True (verify but try to locate system CA certificates)
-        :param region_name: the region used for getting endpoints of swift
+        # - Path (verify using specific CA certificate)
-        """
+        verify = session.verify
-        user = user or CONF.keystone_authtoken.admin_user
+        params['insecure'] = not verify
-        tenant_name = tenant_name or CONF.keystone_authtoken.admin_tenant_name
+        if verify and isinstance(verify, six.string_types):
-        key = key or CONF.keystone_authtoken.admin_password
+            params['cacert'] = verify
-        auth_url = auth_url or CONF.keystone_authtoken.auth_uri
-        auth_version = auth_version or CONF.keystone_authtoken.auth_version
-        auth_url = keystone.get_keystone_url(auth_url, auth_version)
-        params = {'retries': CONF.swift.swift_max_retries,
-                  'insecure': CONF.keystone_authtoken.insecure,
-                  'cacert': CONF.keystone_authtoken.cafile,
-                  'user': user,
-                  'tenant_name': tenant_name,
-                  'key': key,
-                  'authurl': auth_url,
-                  'auth_version': auth_version}
-        region_name = region_name or CONF.keystone_authtoken.region_name
-        if region_name:
-            params['os_options'] = {'region_name': region_name}
 
         self.connection = swift_client.Connection(**params)
 
@@ -131,8 +111,7 @@ class SwiftAPI(object):
             raise exception.SwiftOperationError(operation=operation,
                                                 error=e)
 
-        storage_url, token = self.connection.get_auth()
+        parse_result = parse.urlparse(self.connection.url)
-        parse_result = parse.urlparse(storage_url)
         swift_object_path = '/'.join((parse_result.path, container, object))
         temp_url_key = account_info['x-account-meta-temp-url-key']
         url_path = swift_utils.generate_temp_url(swift_object_path, timeout,

ironic/conf/__init__.py

@@ -38,6 +38,7 @@ from ironic.conf import metrics_statsd
 from ironic.conf import neutron
 from ironic.conf import oneview
 from ironic.conf import seamicro
+from ironic.conf import service_catalog
 from ironic.conf import snmp
 from ironic.conf import ssh
 from ironic.conf import swift
@@ -68,6 +69,7 @@ metrics_statsd.register_opts(CONF)
 neutron.register_opts(CONF)
 oneview.register_opts(CONF)
 seamicro.register_opts(CONF)
+service_catalog.register_opts(CONF)
 snmp.register_opts(CONF)
 ssh.register_opts(CONF)
 swift.register_opts(CONF)

ironic/conf/auth.py

@@ -0,0 +1,79 @@
+# Copyright 2016 Mirantis Inc
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+import copy
+
+from keystoneauth1 import exceptions as kaexception
+from keystoneauth1 import loading as kaloading
+from oslo_config import cfg
+
+
+LEGACY_SECTION = 'keystone_authtoken'
+OLD_SESSION_OPTS = {
+    'certfile': [cfg.DeprecatedOpt('certfile', LEGACY_SECTION)],
+    'keyfile': [cfg.DeprecatedOpt('keyfile', LEGACY_SECTION)],
+    'cafile': [cfg.DeprecatedOpt('cafile', LEGACY_SECTION)],
+    'insecure': [cfg.DeprecatedOpt('insecure', LEGACY_SECTION)],
+    'timeout': [cfg.DeprecatedOpt('timeout', LEGACY_SECTION)],
+}
+
+# FIXME(pas-ha) remove import of auth_token section after deprecation period
+cfg.CONF.import_group(LEGACY_SECTION, 'keystonemiddleware.auth_token')
+
+
+def load_auth(conf, group):
+    try:
+        auth = kaloading.load_auth_from_conf_options(conf, group)
+    except kaexception.MissingRequiredOptions:
+        auth = None
+    return auth
+
+
+def register_auth_opts(conf, group):
+    """Register session- and auth-related options
+
+    Registers only basic auth options shared by all auth plugins.
+    The rest are registered at runtime depending on auth plugin used.
+    """
+    kaloading.register_session_conf_options(
+        conf, group, deprecated_opts=OLD_SESSION_OPTS)
+    kaloading.register_auth_conf_options(conf, group)
+
+
+def add_auth_opts(options):
+    """Add auth options to sample config
+
+    As these are dynamically registered at runtime,
+    this adds options for most used auth_plugins
+    when generating sample config.
+    """
+    def add_options(opts, opts_to_add):
+        for new_opt in opts_to_add:
+            for opt in opts:
+                if opt.name == new_opt.name:
+                    break
+            else:
+                opts.append(new_opt)
+
+    opts = copy.deepcopy(options)
+    opts.insert(0, kaloading.get_auth_common_conf_options()[0])
+    # NOTE(dims): There are a lot of auth plugins, we just generate
+    # the config options for a few common ones
+    plugins = ['password', 'v2password', 'v3password']
+    for name in plugins:
+        plugin = kaloading.get_plugin_loader(name)
+        add_options(opts, kaloading.get_auth_plugin_conf_options(plugin))
+    add_options(opts, kaloading.get_session_conf_options())
+    opts.sort(key=lambda x: x.name)
+    return opts

ironic/conf/glance.py

@@ -18,6 +18,7 @@
 from oslo_config import cfg
 
 from ironic.common.i18n import _
+from ironic.conf import auth
 
 opts = [
     cfg.ListOpt('allowed_direct_url_schemes',
@@ -145,3 +146,8 @@ opts = [
 
 def register_opts(conf):
     conf.register_opts(opts, group='glance')
+    auth.register_auth_opts(conf, 'glance')
+
+
+def list_opts():
+    return auth.add_auth_opts(opts)

ironic/conf/inspector.py

@@ -15,6 +15,7 @@
 from oslo_config import cfg
 
 from ironic.common.i18n import _
+from ironic.conf import auth
 
 opts = [
     cfg.BoolOpt('enabled', default=False,
@@ -31,3 +32,8 @@ opts = [
 
 def register_opts(conf):
     conf.register_opts(opts, group='inspector')
+    auth.register_auth_opts(conf, 'inspector')
+
+
+def list_opts():
+    return auth.add_auth_opts(opts)

ironic/conf/neutron.py

@@ -17,11 +17,15 @@
 from oslo_config import cfg
 
 from ironic.common.i18n import _
+from ironic.conf import auth
 
 opts = [
     cfg.StrOpt('url',
-               default='http://$my_ip:9696',
+               help=_("URL for connecting to neutron. "
-               help=_('URL for connecting to neutron.')),
+                      "Default value translates to 'http://$my_ip:9696' "
+                      "when auth_strategy is 'noauth', "
+                      "and to discovery from Keystone catalog "
+                      "when auth_strategy is 'keystone'.")),
     cfg.IntOpt('url_timeout',
                default=30,
                help=_('Timeout value for connecting to neutron in seconds.')),
@@ -55,3 +59,8 @@ opts = [
 
 def register_opts(conf):
     conf.register_opts(opts, group='neutron')
+    auth.register_auth_opts(conf, 'neutron')
+
+
+def list_opts():
+    return auth.add_auth_opts(opts)

ironic/conf/opts.py

@@ -45,25 +45,26 @@ _opts = [
     ('database', ironic.conf.database.opts),
     ('deploy', ironic.conf.deploy.opts),
     ('dhcp', ironic.conf.dhcp.opts),
-    ('glance', ironic.conf.glance.opts),
+    ('glance', ironic.conf.glance.list_opts()),
     ('iboot', ironic.conf.iboot.opts),
     ('ilo', ironic.conf.ilo.opts),
-    ('inspector', ironic.conf.inspector.opts),
+    ('inspector', ironic.conf.inspector.list_opts()),
     ('ipmi', ironic.conf.ipmi.opts),
     ('irmc', ironic.conf.irmc.opts),
     ('iscsi', ironic.drivers.modules.iscsi_deploy.iscsi_opts),
     ('keystone', ironic.conf.keystone.opts),
-    ('neutron', ironic.conf.neutron.opts),
     ('metrics', ironic.conf.metrics.opts),
     ('metrics_statsd', ironic.conf.metrics_statsd.opts),
+    ('neutron', ironic.conf.neutron.list_opts()),
     ('oneview', ironic.conf.oneview.opts),
     ('pxe', itertools.chain(
         ironic.drivers.modules.iscsi_deploy.pxe_opts,
         ironic.drivers.modules.pxe.pxe_opts)),
     ('seamicro', ironic.conf.seamicro.opts),
+    ('service_catalog', ironic.conf.service_catalog.list_opts()),
     ('snmp', ironic.conf.snmp.opts),
     ('ssh', ironic.conf.ssh.opts),
-    ('swift', ironic.conf.swift.opts),
+    ('swift', ironic.conf.swift.list_opts()),
     ('virtualbox', ironic.conf.virtualbox.opts),
 ]
 

ironic/conf/service_catalog.py

@@ -0,0 +1,33 @@
+# Copyright 2016 Mirantis Inc
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from oslo_config import cfg
+
+from ironic.common.i18n import _
+from ironic.conf import auth
+
+SERVCIE_CATALOG_GROUP = cfg.OptGroup(
+    'service_catalog',
+    title='Access info for Ironic service user',
+    help=_('Holds credentials and session options to access '
+           'Keystone catalog for Ironic API endpoint resolution.'))
+
+
+def register_opts(conf):
+    auth.register_auth_opts(conf, SERVCIE_CATALOG_GROUP.name)
+
+
+def list_opts():
+    return auth.add_auth_opts([])

ironic/conf/swift.py

@@ -17,6 +17,7 @@
 from oslo_config import cfg
 
 from ironic.common.i18n import _
+from ironic.conf import auth
 
 opts = [
     cfg.IntOpt('swift_max_retries',
@@ -28,3 +29,8 @@ opts = [
 
 def register_opts(conf):
     conf.register_opts(opts, group='swift')
+    auth.register_auth_opts(conf, 'swift')
+
+
+def list_opts():
+    return auth.add_auth_opts(opts)

ironic/drivers/modules/deploy_utils.py

@@ -86,6 +86,38 @@ warn_about_unsafe_shred_parameters()
 # All functions are called from deploy() directly or indirectly.
 # They are split for stub-out.
 
+_IRONIC_SESSION = None
+
+
+def _get_ironic_session():
+    global _IRONIC_SESSION
+    if not _IRONIC_SESSION:
+        _IRONIC_SESSION = keystone.get_session('service_catalog')
+    return _IRONIC_SESSION
+
+
+def get_ironic_api_url():
+    """Resolve Ironic API endpoint
+
+    either from config of from Keystone catalog.
+    """
+    ironic_api = CONF.conductor.api_url
+    if not ironic_api:
+        try:
+            ironic_session = _get_ironic_session()
+            ironic_api = keystone.get_service_url(ironic_session)
+        except (exception.KeystoneFailure,
+                exception.CatalogNotFound,
+                exception.KeystoneUnauthorized) as e:
+            raise exception.InvalidParameterValue(_(
+                "Couldn't get the URL of the Ironic API service from the "
+                "configuration file or keystone catalog. Keystone error: "
+                "%s") % six.text_type(e))
+    # NOTE: we should strip '/' from the end because it might be used in
+    # hardcoded ramdisk script
+    ironic_api = ironic_api.rstrip('/')
+    return ironic_api
+
 
 def discovery(portal_address, portal_port):
     """Do iSCSI discovery on portal."""
@@ -998,10 +1030,8 @@ def build_agent_options(node):
     :returns: a dictionary containing the parameters to be passed to
         agent ramdisk.
     """
-    ironic_api = (CONF.conductor.api_url or
-                  keystone.get_service_url()).rstrip('/')
     agent_config_opts = {
-        'ipa-api-url': ironic_api,
+        'ipa-api-url': get_ironic_api_url(),
         'ipa-driver-name': node.driver,
         # NOTE: The below entry is a temporary workaround for bug/1433812
         'coreos.configdrive': 0,

ironic/drivers/modules/inspector.py

@@ -40,6 +40,15 @@ client = importutils.try_import('ironic_inspector_client')
 
 INSPECTOR_API_VERSION = (1, 0)
 
+_INSPECTOR_SESSION = None
+
+
+def _get_inspector_session():
+    global _INSPECTOR_SESSION
+    if not _INSPECTOR_SESSION:
+        _INSPECTOR_SESSION = keystone.get_session('inspector')
+    return _INSPECTOR_SESSION
+
 
 class Inspector(base.InspectInterface):
     """In-band inspection via ironic-inspector project."""
@@ -165,7 +174,8 @@ def _check_status(task):
 
     # NOTE(dtantsur): periodic tasks do not have proper tokens in context
     if CONF.auth_strategy == 'keystone':
-        task.context.auth_token = keystone.get_admin_auth_token()
+        session = _get_inspector_session()
+        task.context.auth_token = keystone.get_admin_auth_token(session)
 
     try:
         status = _call_inspector(client.get_status, node.uuid, task.context)

ironic/drivers/modules/iscsi_deploy.py

@@ -25,7 +25,6 @@ from six.moves.urllib import parse
 from ironic.common import dhcp_factory
 from ironic.common import exception
 from ironic.common.i18n import _
-from ironic.common import keystone
 from ironic.common import states
 from ironic.common import utils
 from ironic.conductor import task_manager
@@ -388,16 +387,8 @@ def validate(task):
              catalog.
     :raises: MissingParameterValue if no ports are enrolled for the given node.
     """
-    try:
     # TODO(lucasagomes): Validate the format of the URL
-        CONF.conductor.api_url or keystone.get_service_url()
+    deploy_utils.get_ironic_api_url()
-    except (exception.KeystoneFailure,
-            exception.CatalogNotFound,
-            exception.KeystoneUnauthorized) as e:
-        raise exception.InvalidParameterValue(_(
-            "Couldn't get the URL of the Ironic API service from the "
-            "configuration file or keystone catalog. Keystone error: %s") % e)
-
     # Validate the root device hints
     deploy_utils.parse_root_device_hints(task.node)
     deploy_utils.parse_instance_info(task.node)

ironic/tests/unit/common/test_image_service.py

@@ -25,7 +25,6 @@ from six.moves import http_client
 from ironic.common import exception
 from ironic.common.glance_service.v1 import image_service as glance_v1_service
 from ironic.common import image_service
-from ironic.common import keystone
 from ironic.tests import base
 
 if six.PY3:
@@ -254,56 +253,59 @@ class FileImageServiceTestCase(base.TestCase):
 
 class ServiceGetterTestCase(base.TestCase):
 
-    @mock.patch.object(keystone, 'get_admin_auth_token', autospec=True)
+    @mock.patch.object(image_service, '_get_glance_session')
     @mock.patch.object(glance_v1_service.GlanceImageService, '__init__',
                        return_value=None, autospec=True)
-    def test_get_glance_image_service(self, glance_service_mock, token_mock):
+    def test_get_glance_image_service(self, glance_service_mock,
+                                      session_mock):
         image_href = 'image-uuid'
         self.context.auth_token = 'fake'
         image_service.get_image_service(image_href, context=self.context)
         glance_service_mock.assert_called_once_with(mock.ANY, None, 1,
                                                     self.context)
-        self.assertFalse(token_mock.called)
+        self.assertFalse(session_mock.called)
 
-    @mock.patch.object(keystone, 'get_admin_auth_token', autospec=True)
+    @mock.patch.object(image_service, '_get_glance_session')
     @mock.patch.object(glance_v1_service.GlanceImageService, '__init__',
                        return_value=None, autospec=True)
     def test_get_glance_image_service_url(self, glance_service_mock,
-                                          token_mock):
+                                          session_mock):
         image_href = 'glance://image-uuid'
         self.context.auth_token = 'fake'
         image_service.get_image_service(image_href, context=self.context)
         glance_service_mock.assert_called_once_with(mock.ANY, None, 1,
                                                     self.context)
-        self.assertFalse(token_mock.called)
+        self.assertFalse(session_mock.called)
 
-    @mock.patch.object(keystone, 'get_admin_auth_token', autospec=True)
+    @mock.patch.object(image_service, '_get_glance_session')
     @mock.patch.object(glance_v1_service.GlanceImageService, '__init__',
                        return_value=None, autospec=True)
     def test_get_glance_image_service_no_token(self, glance_service_mock,
-                                               token_mock):
+                                               session_mock):
         image_href = 'image-uuid'
         self.context.auth_token = None
-        token_mock.return_value = 'admin-token'
+        sess = mock.Mock()
+        sess.get_token.return_value = 'admin-token'
+        session_mock.return_value = sess
         image_service.get_image_service(image_href, context=self.context)
         glance_service_mock.assert_called_once_with(mock.ANY, None, 1,
                                                     self.context)
-        token_mock.assert_called_once_with()
+        sess.get_token.assert_called_once_with()
         self.assertEqual('admin-token', self.context.auth_token)
 
-    @mock.patch.object(keystone, 'get_admin_auth_token', autospec=True)
+    @mock.patch.object(image_service, '_get_glance_session')
     @mock.patch.object(glance_v1_service.GlanceImageService, '__init__',
                        return_value=None, autospec=True)
     def test_get_glance_image_service_token_not_needed(self,
                                                        glance_service_mock,
-                                                       token_mock):
+                                                       session_mock):
         image_href = 'image-uuid'
         self.context.auth_token = None
         self.config(auth_strategy='noauth', group='glance')
         image_service.get_image_service(image_href, context=self.context)
         glance_service_mock.assert_called_once_with(mock.ANY, None, 1,
                                                     self.context)
-        self.assertFalse(token_mock.called)
+        self.assertFalse(session_mock.called)
         self.assertIsNone(self.context.auth_token)
 
     @mock.patch.object(image_service.HttpImageService, '__init__',

ironic/tests/unit/common/test_keystone.py

@@ -12,174 +12,138 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-from keystoneclient import exceptions as ksexception
+from keystoneauth1 import exceptions as ksexception
+from keystoneauth1 import loading as kaloading
 import mock
+from oslo_config import cfg
+from oslo_config import fixture
 
 from ironic.common import exception
 from ironic.common import keystone
+from ironic.conf import auth as ironic_auth
 from ironic.tests import base
 
 
-class FakeCatalog(object):
-    def url_for(self, **kwargs):
-        return 'fake-url'
-
-
-class FakeAccessInfo(object):
-    def will_expire_soon(self):
-        pass
-
-
-class FakeClient(object):
-    def __init__(self, **kwargs):
-        self.service_catalog = FakeCatalog()
-        self.auth_ref = FakeAccessInfo()
-
-    def has_service_catalog(self):
-        return True
-
-
 class KeystoneTestCase(base.TestCase):
 
     def setUp(self):
         super(KeystoneTestCase, self).setUp()
-        self.config(group='keystone_authtoken',
+        self.config(region_name='fake_region',
-                    auth_uri='http://127.0.0.1:9898/',
+                    group='keystone')
-                    admin_user='fake', admin_password='fake',
+        self.test_group = 'test_group'
-                    admin_tenant_name='fake')
+        self.cfg_fixture.conf.register_group(cfg.OptGroup(self.test_group))
-        self.config(group='keystone', region_name='fake')
+        ironic_auth.register_auth_opts(self.cfg_fixture.conf, self.test_group)
-        keystone._KS_CLIENT = None
+        self.config(auth_type='password',
+                    group=self.test_group)
+        # NOTE(pas-ha) this is due to auth_plugin options
+        # being dynamically registered on first load,
+        # but we need to set the config before
+        plugin = kaloading.get_plugin_loader('password')
+        opts = kaloading.get_auth_plugin_conf_options(plugin)
+        self.cfg_fixture.register_opts(opts, group=self.test_group)
+        self.config(auth_url='http://127.0.0.1:9898',
+                    username='fake_user',
+                    password='fake_pass',
+                    project_name='fake_tenant',
+                    group=self.test_group)
 
-    def test_failure_authorization(self):
+    def _set_config(self):
-        self.assertRaises(exception.KeystoneFailure, keystone.get_service_url)
+        self.cfg_fixture = self.useFixture(fixture.Config())
+        self.addCleanup(cfg.CONF.reset)
 
-    @mock.patch.object(FakeCatalog, 'url_for', autospec=True)
+    def test_get_url(self):
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
-    def test_get_url(self, mock_ks, mock_uf):
         fake_url = 'http://127.0.0.1:6385'
-        mock_uf.return_value = fake_url
+        mock_sess = mock.Mock()
-        mock_ks.return_value = FakeClient()
+        mock_sess.get_endpoint.return_value = fake_url
-        res = keystone.get_service_url()
+        res = keystone.get_service_url(mock_sess)
         self.assertEqual(fake_url, res)
 
-    @mock.patch.object(FakeCatalog, 'url_for', autospec=True)
+    def test_get_url_failure(self):
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
+        exc_map = (
-    def test_url_not_found(self, mock_ks, mock_uf):
+            (ksexception.Unauthorized, exception.KeystoneUnauthorized),
-        mock_uf.side_effect = ksexception.EndpointNotFound
+            (ksexception.EndpointNotFound, exception.CatalogNotFound),
-        mock_ks.return_value = FakeClient()
+            (ksexception.EmptyCatalog, exception.CatalogNotFound),
-        self.assertRaises(exception.CatalogNotFound, keystone.get_service_url)
+            (ksexception.Unauthorized, exception.KeystoneUnauthorized),
+        )
+        for kexc, irexc in exc_map:
+            mock_sess = mock.Mock()
+            mock_sess.get_endpoint.side_effect = kexc
+            self.assertRaises(irexc, keystone.get_service_url, mock_sess)
 
-    @mock.patch.object(FakeClient, 'has_service_catalog', autospec=True)
+    def test_get_admin_auth_token(self):
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
+        mock_sess = mock.Mock()
-    def test_no_catalog(self, mock_ks, mock_hsc):
+        mock_sess.get_token.return_value = 'fake_token'
-        mock_hsc.return_value = False
+        self.assertEqual('fake_token',
-        mock_ks.return_value = FakeClient()
+                         keystone.get_admin_auth_token(mock_sess))
-        self.assertRaises(exception.KeystoneFailure, keystone.get_service_url)
 
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
+    def test_get_admin_auth_token_failure(self):
-    def test_unauthorized(self, mock_ks):
+        mock_sess = mock.Mock()
-        mock_ks.side_effect = ksexception.Unauthorized
+        mock_sess.get_token.side_effect = ksexception.Unauthorized
         self.assertRaises(exception.KeystoneUnauthorized,
-                          keystone.get_service_url)
+                          keystone.get_admin_auth_token, mock_sess)
 
-    def test_get_service_url_fail_missing_auth_uri(self):
+    @mock.patch.object(ironic_auth, 'load_auth')
-        self.config(group='keystone_authtoken', auth_uri=None)
+    def test_get_session(self, auth_get_mock):
-        self.assertRaises(exception.KeystoneFailure,
+        auth_mock = mock.Mock()
-                          keystone.get_service_url)
+        auth_get_mock.return_value = auth_mock
+        session = keystone.get_session(self.test_group)
+        self.assertEqual(auth_mock, session.auth)
 
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
+    @mock.patch.object(keystone, '_get_legacy_auth', return_value=None)
-    def test_get_service_url_versionless_v2(self, mock_ks):
+    @mock.patch.object(ironic_auth, 'load_auth', return_value=None)
-        mock_ks.return_value = FakeClient()
+    def test_get_session_fail(self, auth_get_mock, legacy_get_mock):
-        self.config(group='keystone_authtoken', auth_uri='http://127.0.0.1')
+        self.assertRaisesRegexp(
-        expected_url = 'http://127.0.0.1/v2.0'
+            exception.KeystoneFailure,
-        keystone.get_service_url()
+            "Failed to load auth from either",
-        mock_ks.assert_called_once_with(username='fake', password='fake',
+            keystone.get_session, self.test_group)
-                                        tenant_name='fake',
-                                        region_name='fake',
-                                        auth_url=expected_url)
 
-    @mock.patch('keystoneclient.v3.client.Client', autospec=True)
+    @mock.patch('keystoneauth1.loading.load_auth_from_conf_options')
-    def test_get_service_url_versionless_v3(self, mock_ks):
+    @mock.patch('ironic.common.keystone._get_legacy_auth')
-        mock_ks.return_value = FakeClient()
+    def test_get_session_failed_new_auth(self, legacy_get_mock, load_mock):
-        self.config(group='keystone_authtoken', auth_version='v3.0',
+        legacy_mock = mock.Mock()
-                    auth_uri='http://127.0.0.1')
+        legacy_get_mock.return_value = legacy_mock
-        expected_url = 'http://127.0.0.1/v3'
+        load_mock.side_effect = [None, ksexception.MissingRequiredOptions]
-        keystone.get_service_url()
+        self.assertEqual(legacy_mock,
-        mock_ks.assert_called_once_with(username='fake', password='fake',
+                         keystone.get_session(self.test_group).auth)
-                                        tenant_name='fake',
-                                        region_name='fake',
-                                        auth_url=expected_url)
 
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
-    def test_get_service_url_version_override(self, mock_ks):
-        mock_ks.return_value = FakeClient()
-        self.config(group='keystone_authtoken',
-                    auth_uri='http://127.0.0.1/v2.0/')
-        expected_url = 'http://127.0.0.1/v2.0'
-        keystone.get_service_url()
-        mock_ks.assert_called_once_with(username='fake', password='fake',
-                                        tenant_name='fake',
-                                        region_name='fake',
-                                        auth_url=expected_url)
 
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
+@mock.patch('keystoneauth1.loading._plugins.identity.generic.Password.'
-    def test_get_admin_auth_token(self, mock_ks):
+            'load_from_options')
-        fake_client = FakeClient()
+class KeystoneLegacyTestCase(base.TestCase):
-        fake_client.auth_token = '123456'
+    def setUp(self):
-        mock_ks.return_value = fake_client
+        super(KeystoneLegacyTestCase, self).setUp()
-        self.assertEqual('123456', keystone.get_admin_auth_token())
+        self.test_group = 'test_group'
+        self.cfg_fixture.conf.register_group(cfg.OptGroup(self.test_group))
+        self.config(group=ironic_auth.LEGACY_SECTION,
+                    auth_uri='http://127.0.0.1:9898',
+                    admin_user='fake_user',
+                    admin_password='fake_pass',
+                    admin_tenant_name='fake_tenant')
+        ironic_auth.register_auth_opts(self.cfg_fixture.conf, self.test_group)
+        self.config(group=self.test_group,
+                    auth_type=None)
+        self.expected = dict(
+            auth_url='http://127.0.0.1:9898',
+            username='fake_user',
+            password='fake_pass',
+            tenant_name='fake_tenant')
 
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
+    def _set_config(self):
-    def test_get_region_name_v2(self, mock_ks):
+        self.cfg_fixture = self.useFixture(fixture.Config())
-        mock_ks.return_value = FakeClient()
+        self.addCleanup(cfg.CONF.reset)
-        self.config(group='keystone', region_name='fake_region')
-        expected_url = 'http://127.0.0.1:9898/v2.0'
-        expected_region = 'fake_region'
-        keystone.get_service_url()
-        mock_ks.assert_called_once_with(username='fake', password='fake',
-                                        tenant_name='fake',
-                                        region_name=expected_region,
-                                        auth_url=expected_url)
 
-    @mock.patch('keystoneclient.v3.client.Client', autospec=True)
+    @mock.patch.object(ironic_auth, 'load_auth', return_value=None)
-    def test_get_region_name_v3(self, mock_ks):
+    def test_legacy_loading_v2(self, load_auth_mock, load_mock):
-        mock_ks.return_value = FakeClient()
+        keystone.get_session(self.test_group)
-        self.config(group='keystone', region_name='fake_region')
+        load_mock.assert_called_once_with(**self.expected)
-        self.config(group='keystone_authtoken', auth_version='v3.0')
-        expected_url = 'http://127.0.0.1:9898/v3'
-        expected_region = 'fake_region'
-        keystone.get_service_url()
-        mock_ks.assert_called_once_with(username='fake', password='fake',
-                                        tenant_name='fake',
-                                        region_name=expected_region,
-                                        auth_url=expected_url)
 
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
+    @mock.patch.object(ironic_auth, 'load_auth', return_value=None)
-    def test_cache_client_init(self, mock_ks):
+    def test_legacy_loading_v3(self, load_auth_mock, load_mock):
-        fake_client = FakeClient()
+        self.config(
-        mock_ks.return_value = fake_client
+            auth_version='v3.0',
-        self.assertEqual(fake_client, keystone._get_ksclient())
+            group=ironic_auth.LEGACY_SECTION)
-        self.assertEqual(fake_client, keystone._KS_CLIENT)
+        self.expected.update(dict(
-        self.assertEqual(1, mock_ks.call_count)
+            project_domain_id='default',
-
+            user_domain_id='default'))
-    @mock.patch.object(FakeAccessInfo, 'will_expire_soon', autospec=True)
+        keystone.get_session(self.test_group)
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
+        load_mock.assert_called_once_with(**self.expected)
-    def test_cache_client_cached(self, mock_ks, mock_expire):
-        mock_expire.return_value = False
-        fake_client = FakeClient()
-        keystone._KS_CLIENT = fake_client
-        self.assertEqual(fake_client, keystone._get_ksclient())
-        self.assertEqual(fake_client, keystone._KS_CLIENT)
-        self.assertFalse(mock_ks.called)
-
-    @mock.patch.object(FakeAccessInfo, 'will_expire_soon', autospec=True)
-    @mock.patch('keystoneclient.v2_0.client.Client', autospec=True)
-    def test_cache_client_expired(self, mock_ks, mock_expire):
-        mock_expire.return_value = True
-        fake_client = FakeClient()
-        keystone._KS_CLIENT = fake_client
-        new_client = FakeClient()
-        mock_ks.return_value = new_client
-        self.assertEqual(new_client, keystone._get_ksclient())
-        self.assertEqual(new_client, keystone._KS_CLIENT)
-        self.assertEqual(1, mock_ks.call_count)

ironic/tests/unit/common/test_neutron.py

@@ -19,86 +19,80 @@ from oslo_utils import uuidutils
 from ironic.common import exception
 from ironic.common import neutron
 from ironic.conductor import task_manager
+# from ironic.conf import auth as ironic_auth
 from ironic.tests import base
 from ironic.tests.unit.conductor import mgr_utils
 from ironic.tests.unit.db import base as db_base
 from ironic.tests.unit.objects import utils as object_utils
 
 
+@mock.patch.object(neutron, '_get_neutron_session')
+@mock.patch.object(client.Client, "__init__")
 class TestNeutronClient(base.TestCase):
 
     def setUp(self):
         super(TestNeutronClient, self).setUp()
-        self.config(url='test-url',
+        self.config(url_timeout=30,
-                    url_timeout=30,
                     retries=2,
                     group='neutron')
-        self.config(insecure=False,
+        self.config(admin_user='test-admin-user',
-                    certfile='test-file',
-                    admin_user='test-admin-user',
                     admin_tenant_name='test-admin-tenant',
                     admin_password='test-admin-password',
                     auth_uri='test-auth-uri',
                     group='keystone_authtoken')
+        # TODO(pas-ha) register session options to test legacy path
+        self.config(insecure=False,
+                    cafile='test-file',
+                    group='neutron')
 
-    @mock.patch.object(client.Client, "__init__")
+    def test_get_neutron_client_with_token(self, mock_client_init,
-    def test_get_neutron_client_with_token(self, mock_client_init):
+                                           mock_session):
         token = 'test-token-123'
+        sess = mock.Mock()
+        sess.get_endpoint.return_value = 'fake-url'
+        mock_session.return_value = sess
         expected = {'timeout': 30,
                     'retries': 2,
                     'insecure': False,
                     'ca_cert': 'test-file',
                     'token': token,
-                    'endpoint_url': 'test-url',
+                    'endpoint_url': 'fake-url'}
-                    'username': 'test-admin-user',
-                    'tenant_name': 'test-admin-tenant',
-                    'password': 'test-admin-password',
-                    'auth_url': 'test-auth-uri'}
 
         mock_client_init.return_value = None
         neutron.get_client(token=token)
         mock_client_init.assert_called_once_with(**expected)
 
-    @mock.patch.object(client.Client, "__init__")
+    def test_get_neutron_client_without_token(self, mock_client_init,
-    def test_get_neutron_client_without_token(self, mock_client_init):
+                                              mock_session):
-        expected = {'timeout': 30,
+        self.config(url='test-url',
-                    'retries': 2,
+                    group='neutron')
-                    'insecure': False,
+        sess = mock.Mock()
-                    'ca_cert': 'test-file',
+        mock_session.return_value = sess
-                    'token': None,
+        expected = {'retries': 2,
-                    'endpoint_url': 'test-url',
+                    'endpoint_override': 'test-url',
-                    'username': 'test-admin-user',
+                    'session': sess}
-                    'tenant_name': 'test-admin-tenant',
-                    'password': 'test-admin-password',
-                    'auth_url': 'test-auth-uri'}
-
         mock_client_init.return_value = None
         neutron.get_client(token=None)
         mock_client_init.assert_called_once_with(**expected)
 
-    @mock.patch.object(client.Client, "__init__")
+    def test_get_neutron_client_with_region(self, mock_client_init,
-    def test_get_neutron_client_with_region(self, mock_client_init):
+                                            mock_session):
-        expected = {'timeout': 30,
+        self.config(region_name='fake_region',
-                    'retries': 2,
-                    'insecure': False,
-                    'ca_cert': 'test-file',
-                    'token': None,
-                    'endpoint_url': 'test-url',
-                    'username': 'test-admin-user',
-                    'tenant_name': 'test-admin-tenant',
-                    'password': 'test-admin-password',
-                    'auth_url': 'test-auth-uri',
-                    'region_name': 'test-region'}
-
-        self.config(region_name='test-region',
                     group='keystone')
+        sess = mock.Mock()
+        mock_session.return_value = sess
+        expected = {'retries': 2,
+                    'region_name': 'fake_region',
+                    'session': sess}
+
         mock_client_init.return_value = None
         neutron.get_client(token=None)
         mock_client_init.assert_called_once_with(**expected)
 
-    @mock.patch.object(client.Client, "__init__")
+    def test_get_neutron_client_noauth(self, mock_client_init, mock_session):
-    def test_get_neutron_client_noauth(self, mock_client_init):
+        self.config(auth_strategy='noauth',
-        self.config(auth_strategy='noauth', group='neutron')
+                    url='test-url',
+                    group='neutron')
         expected = {'ca_cert': 'test-file',
                     'insecure': False,
                     'endpoint_url': 'test-url',
@@ -110,7 +104,7 @@ class TestNeutronClient(base.TestCase):
         neutron.get_client(token=None)
         mock_client_init.assert_called_once_with(**expected)
 
-    def test_out_range_auth_strategy(self):
+    def test_out_range_auth_strategy(self, mock_client_init, mock_session):
         self.assertRaises(ValueError, cfg.CONF.set_override,
                           'auth_strategy', 'fake', 'neutron',
                           enforce_type=True)
@@ -133,9 +127,13 @@ class TestNeutronNetworkActions(db_base.DbTestCase):
         self.neutron_port = {'id': '132f871f-eaec-4fed-9475-0d54465e0f00',
                              'mac_address': '52:54:00:cf:2d:32'}
         self.network_uuid = uuidutils.generate_uuid()
+        self.client_mock = mock.Mock()
+        patcher = mock.patch('ironic.common.neutron.get_client',
+                             return_value=self.client_mock)
+        patcher.start()
+        self.addCleanup(patcher.stop)
 
-    @mock.patch.object(client.Client, 'create_port')
+    def test_add_ports_to_vlan_network(self):
-    def test_add_ports_to_vlan_network(self, create_mock):
         # Ports will be created only if pxe_enabled is True
         object_utils.create_test_port(
             self.context, node_id=self.node.id,
@@ -159,15 +157,16 @@ class TestNeutronNetworkActions(db_base.DbTestCase):
             }
         }
         # Ensure we can create ports
-        create_mock.return_value = {'port': self.neutron_port}
+        self.client_mock.create_port.return_value = {
+            'port': self.neutron_port}
         expected = {port.uuid: self.neutron_port['id']}
         with task_manager.acquire(self.context, self.node.uuid) as task:
             ports = neutron.add_ports_to_network(task, self.network_uuid)
             self.assertEqual(expected, ports)
-            create_mock.assert_called_once_with(expected_body)
+            self.client_mock.create_port.assert_called_once_with(
+                expected_body)
 
-    @mock.patch.object(client.Client, 'create_port')
+    def test_add_ports_to_flat_network(self):
-    def test_add_ports_to_flat_network(self, create_mock):
         port = self.ports[0]
         expected_body = {
             'port': {
@@ -183,16 +182,17 @@ class TestNeutronNetworkActions(db_base.DbTestCase):
             }
         }
         # Ensure we can create ports
-        create_mock.return_value = {'port': self.neutron_port}
+        self.client_mock.create_port.return_value = {
+            'port': self.neutron_port}
         expected = {port.uuid: self.neutron_port['id']}
         with task_manager.acquire(self.context, self.node.uuid) as task:
             ports = neutron.add_ports_to_network(task, self.network_uuid,
                                                  is_flat=True)
             self.assertEqual(expected, ports)
-            create_mock.assert_called_once_with(expected_body)
+            self.client_mock.create_port.assert_called_once_with(
+                expected_body)
 
-    @mock.patch.object(client.Client, 'create_port')
+    def test_add_ports_to_flat_network_no_neutron_port_id(self):
-    def test_add_ports_to_flat_network_no_neutron_port_id(self, create_mock):
         port = self.ports[0]
         expected_body = {
             'port': {
@@ -208,15 +208,16 @@ class TestNeutronNetworkActions(db_base.DbTestCase):
             }
         }
         del self.neutron_port['id']
-        create_mock.return_value = {'port': self.neutron_port}
+        self.client_mock.create_port.return_value = {
+            'port': self.neutron_port}
         with task_manager.acquire(self.context, self.node.uuid) as task:
             self.assertRaises(exception.NetworkError,
                               neutron.add_ports_to_network,
                               task, self.network_uuid, is_flat=True)
-            create_mock.assert_called_once_with(expected_body)
+            self.client_mock.create_port.assert_called_once_with(
+                expected_body)
 
-    @mock.patch.object(client.Client, 'create_port')
+    def test_add_ports_to_vlan_network_instance_uuid(self):
-    def test_add_ports_to_vlan_network_instance_uuid(self, create_mock):
         self.node.instance_uuid = uuidutils.generate_uuid()
         self.node.save()
         port = self.ports[0]
@@ -235,18 +236,18 @@ class TestNeutronNetworkActions(db_base.DbTestCase):
             }
         }
         # Ensure we can create ports
-        create_mock.return_value = {'port': self.neutron_port}
+        self.client_mock.create_port.return_value = {'port': self.neutron_port}
         expected = {port.uuid: self.neutron_port['id']}
         with task_manager.acquire(self.context, self.node.uuid) as task:
             ports = neutron.add_ports_to_network(task, self.network_uuid)
             self.assertEqual(expected, ports)
-            create_mock.assert_called_once_with(expected_body)
+            self.client_mock.create_port.assert_called_once_with(expected_body)
 
     @mock.patch.object(neutron, 'rollback_ports')
-    @mock.patch.object(client.Client, 'create_port')
+    def test_add_network_fail(self, rollback_mock):
-    def test_add_network_fail(self, create_mock, rollback_mock):
         # Check that if creating a port fails, the ports are cleaned up
-        create_mock.side_effect = neutron_client_exc.ConnectionFailed
+        self.client_mock.create_port.side_effect = \
+            neutron_client_exc.ConnectionFailed
 
         with task_manager.acquire(self.context, self.node.uuid) as task:
             self.assertRaisesRegex(
@@ -255,9 +256,8 @@ class TestNeutronNetworkActions(db_base.DbTestCase):
             rollback_mock.assert_called_once_with(task, self.network_uuid)
 
     @mock.patch.object(neutron, 'rollback_ports')
-    @mock.patch.object(client.Client, 'create_port', return_value={})
+    def test_add_network_fail_create_any_port_empty(self, rollback_mock):
-    def test_add_network_fail_create_any_port_empty(self, create_mock,
+        self.client_mock.create_port.return_value = {}
-                                                    rollback_mock):
         with task_manager.acquire(self.context, self.node.uuid) as task:
             self.assertRaisesRegex(
                 exception.NetworkError, 'any PXE enabled port',
@@ -266,16 +266,16 @@ class TestNeutronNetworkActions(db_base.DbTestCase):
 
     @mock.patch.object(neutron, 'LOG')
     @mock.patch.object(neutron, 'rollback_ports')
-    @mock.patch.object(client.Client, 'create_port')
+    def test_add_network_fail_create_some_ports_empty(self, rollback_mock,
-    def test_add_network_fail_create_some_ports_empty(self, create_mock,
+                                                      log_mock):
-                                                      rollback_mock, log_mock):
         port2 = object_utils.create_test_port(
             self.context, node_id=self.node.id,
             uuid=uuidutils.generate_uuid(),
             address='52:54:55:cf:2d:32',
             extra={'vif_port_id': uuidutils.generate_uuid()}
         )
-        create_mock.side_effect = [{'port': self.neutron_port}, {}]
+        self.client_mock.create_port.side_effect = [
+            {'port': self.neutron_port}, {}]
         with task_manager.acquire(self.context, self.node.uuid) as task:
             neutron.add_ports_to_network(task, self.network_uuid)
             self.assertIn(str(port2.uuid),
@@ -309,35 +309,39 @@ class TestNeutronNetworkActions(db_base.DbTestCase):
                  'mac_address': [self.ports[0].address]}
             )
 
-    @mock.patch.object(client.Client, 'delete_port')
+    def test_remove_neutron_ports(self):
-    @mock.patch.object(client.Client, 'list_ports')
-    def test_remove_neutron_ports(self, list_mock, delete_mock):
         with task_manager.acquire(self.context, self.node.uuid) as task:
-            list_mock.return_value = {'ports': [self.neutron_port]}
+            self.client_mock.list_ports.return_value = {
+                'ports': [self.neutron_port]}
             neutron.remove_neutron_ports(task, {'param': 'value'})
-        list_mock.assert_called_once_with(**{'param': 'value'})
+        self.client_mock.list_ports.assert_called_once_with(
-        delete_mock.assert_called_once_with(self.neutron_port['id'])
+            **{'param': 'value'})
+        self.client_mock.delete_port.assert_called_once_with(
+            self.neutron_port['id'])
 
-    @mock.patch.object(client.Client, 'list_ports')
+    def test_remove_neutron_ports_list_fail(self):
-    def test_remove_neutron_ports_list_fail(self, list_mock):
         with task_manager.acquire(self.context, self.node.uuid) as task:
-            list_mock.side_effect = neutron_client_exc.ConnectionFailed
+            self.client_mock.list_ports.side_effect = \
+                neutron_client_exc.ConnectionFailed
             self.assertRaisesRegex(
                 exception.NetworkError, 'Could not get given network VIF',
                 neutron.remove_neutron_ports, task, {'param': 'value'})
-        list_mock.assert_called_once_with(**{'param': 'value'})
+        self.client_mock.list_ports.assert_called_once_with(
+            **{'param': 'value'})
 
-    @mock.patch.object(client.Client, 'delete_port')
+    def test_remove_neutron_ports_delete_fail(self):
-    @mock.patch.object(client.Client, 'list_ports')
-    def test_remove_neutron_ports_delete_fail(self, list_mock, delete_mock):
         with task_manager.acquire(self.context, self.node.uuid) as task:
-            delete_mock.side_effect = neutron_client_exc.ConnectionFailed
+            self.client_mock.delete_port.side_effect = \
-            list_mock.return_value = {'ports': [self.neutron_port]}
+                neutron_client_exc.ConnectionFailed
+            self.client_mock.list_ports.return_value = {
+                'ports': [self.neutron_port]}
             self.assertRaisesRegex(
                 exception.NetworkError, 'Could not remove VIF',
                 neutron.remove_neutron_ports, task, {'param': 'value'})
-        list_mock.assert_called_once_with(**{'param': 'value'})
+        self.client_mock.list_ports.assert_called_once_with(
-        delete_mock.assert_called_once_with(self.neutron_port['id'])
+            **{'param': 'value'})
+        self.client_mock.delete_port.assert_called_once_with(
+            self.neutron_port['id'])
 
     def test_get_node_portmap(self):
         with task_manager.acquire(self.context, self.node.uuid) as task:

ironic/tests/unit/common/test_swift.py

@@ -30,6 +30,7 @@ if six.PY3:
     file = io.BytesIO
 
 
+@mock.patch.object(swift, '_get_swift_session')
 @mock.patch.object(swift_client, 'Connection', autospec=True)
 class SwiftTestCase(base.TestCase):
 
@@ -37,42 +38,22 @@ class SwiftTestCase(base.TestCase):
         super(SwiftTestCase, self).setUp()
         self.swift_exception = swift_exception.ClientException('', '')
 
-        self.config(admin_user='admin', group='keystone_authtoken')
+    def test___init__(self, connection_mock, keystone_mock):
-        self.config(admin_tenant_name='tenant', group='keystone_authtoken')
+        sess = mock.Mock()
-        self.config(admin_password='password', group='keystone_authtoken')
+        sess.get_endpoint.return_value = 'http://swift:8080'
-        self.config(auth_uri='http://authurl', group='keystone_authtoken')
+        sess.get_token.return_value = 'fake_token'
-        self.config(auth_version='2', group='keystone_authtoken')
+        sess.verify = '/path/to/ca/file'
-        self.config(swift_max_retries=2, group='swift')
+        keystone_mock.return_value = sess
-        self.config(insecure=0, group='keystone_authtoken')
-        self.config(cafile='/path/to/ca/file', group='keystone_authtoken')
-        self.expected_params = {'retries': 2,
-                                'insecure': 0,
-                                'user': 'admin',
-                                'tenant_name': 'tenant',
-                                'key': 'password',
-                                'authurl': 'http://authurl/v2.0',
-                                'cacert': '/path/to/ca/file',
-                                'auth_version': '2'}
-
-    def test___init__(self, connection_mock):
         swift.SwiftAPI()
-        connection_mock.assert_called_once_with(**self.expected_params)
+        params = {'retries': 2,
-
+                  'preauthurl': 'http://swift:8080',
-    def test__init__with_region_from_config(self, connection_mock):
+                  'preauthtoken': 'fake_token',
-        self.config(region_name='region1', group='keystone_authtoken')
+                  'insecure': False,
-        swift.SwiftAPI()
+                  'cacert': '/path/to/ca/file'}
-        params = self.expected_params.copy()
-        params['os_options'] = {'region_name': 'region1'}
-        connection_mock.assert_called_once_with(**params)
-
-    def test__init__with_region_from_constructor(self, connection_mock):
-        swift.SwiftAPI(region_name='region1')
-        params = self.expected_params.copy()
-        params['os_options'] = {'region_name': 'region1'}
         connection_mock.assert_called_once_with(**params)
 
     @mock.patch.object(__builtin__, 'open', autospec=True)
-    def test_create_object(self, open_mock, connection_mock):
+    def test_create_object(self, open_mock, connection_mock, keystone_mock):
         swiftapi = swift.SwiftAPI()
         connection_obj_mock = connection_mock.return_value
         mock_file_handle = mock.MagicMock(spec=file)
@@ -91,7 +72,8 @@ class SwiftTestCase(base.TestCase):
 
     @mock.patch.object(__builtin__, 'open', autospec=True)
     def test_create_object_create_container_fails(self, open_mock,
-                                                  connection_mock):
+                                                  connection_mock,
+                                                  keystone_mock):
         swiftapi = swift.SwiftAPI()
         connection_obj_mock = connection_mock.return_value
         connection_obj_mock.put_container.side_effect = self.swift_exception
@@ -102,7 +84,8 @@ class SwiftTestCase(base.TestCase):
         self.assertFalse(connection_obj_mock.put_object.called)
 
     @mock.patch.object(__builtin__, 'open', autospec=True)
-    def test_create_object_put_object_fails(self, open_mock, connection_mock):
+    def test_create_object_put_object_fails(self, open_mock, connection_mock,
+                                            keystone_mock):
         swiftapi = swift.SwiftAPI()
         mock_file_handle = mock.MagicMock(spec=file)
         mock_file_handle.__enter__.return_value = 'file-object'
@@ -118,30 +101,30 @@ class SwiftTestCase(base.TestCase):
             'container', 'object', 'file-object', headers=None)
 
     @mock.patch.object(swift_utils, 'generate_temp_url', autospec=True)
-    def test_get_temp_url(self, gen_temp_url_mock, connection_mock):
+    def test_get_temp_url(self, gen_temp_url_mock, connection_mock,
+                          keystone_mock):
         swiftapi = swift.SwiftAPI()
         connection_obj_mock = connection_mock.return_value
-        auth = ['http://host/v1/AUTH_tenant_id', 'token']
+        connection_obj_mock.url = 'http://host/v1/AUTH_tenant_id'
-        connection_obj_mock.get_auth.return_value = auth
         head_ret_val = {'x-account-meta-temp-url-key': 'secretkey'}
         connection_obj_mock.head_account.return_value = head_ret_val
         gen_temp_url_mock.return_value = 'temp-url-path'
         temp_url_returned = swiftapi.get_temp_url('container', 'object', 10)
-        connection_obj_mock.get_auth.assert_called_once_with()
         connection_obj_mock.head_account.assert_called_once_with()
         object_path_expected = '/v1/AUTH_tenant_id/container/object'
         gen_temp_url_mock.assert_called_once_with(object_path_expected, 10,
                                                   'secretkey', 'GET')
         self.assertEqual('http://host/temp-url-path', temp_url_returned)
 
-    def test_delete_object(self, connection_mock):
+    def test_delete_object(self, connection_mock, keystone_mock):
         swiftapi = swift.SwiftAPI()
         connection_obj_mock = connection_mock.return_value
         swiftapi.delete_object('container', 'object')
         connection_obj_mock.delete_object.assert_called_once_with('container',
                                                                   'object')
 
-    def test_delete_object_exc_resource_not_found(self, connection_mock):
+    def test_delete_object_exc_resource_not_found(self, connection_mock,
+                                                  keystone_mock):
         swiftapi = swift.SwiftAPI()
         exc = swift_exception.ClientException(
             "Resource not found", http_status=http_client.NOT_FOUND)
@@ -152,7 +135,7 @@ class SwiftTestCase(base.TestCase):
         connection_obj_mock.delete_object.assert_called_once_with('container',
                                                                   'object')
 
-    def test_delete_object_exc(self, connection_mock):
+    def test_delete_object_exc(self, connection_mock, keystone_mock):
         swiftapi = swift.SwiftAPI()
         exc = swift_exception.ClientException("Operation error")
         connection_obj_mock = connection_mock.return_value
@@ -162,7 +145,7 @@ class SwiftTestCase(base.TestCase):
         connection_obj_mock.delete_object.assert_called_once_with('container',
                                                                   'object')
 
-    def test_head_object(self, connection_mock):
+    def test_head_object(self, connection_mock, keystone_mock):
         swiftapi = swift.SwiftAPI()
         connection_obj_mock = connection_mock.return_value
         expected_head_result = {'a': 'b'}
@@ -172,7 +155,7 @@ class SwiftTestCase(base.TestCase):
                                                                 'object')
         self.assertEqual(expected_head_result, actual_head_result)
 
-    def test_update_object_meta(self, connection_mock):
+    def test_update_object_meta(self, connection_mock, keystone_mock):
         swiftapi = swift.SwiftAPI()
         connection_obj_mock = connection_mock.return_value
         headers = {'a': 'b'}

ironic/tests/unit/conf/test_auth.py

@@ -0,0 +1,70 @@
+# Copyright 2016 Mirantis Inc
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from keystoneauth1 import identity as kaidentity
+from keystoneauth1 import loading as kaloading
+from oslo_config import cfg
+
+from ironic.conf import auth as ironic_auth
+from ironic.tests import base
+
+
+class AuthConfTestCase(base.TestCase):
+
+    def setUp(self):
+        super(AuthConfTestCase, self).setUp()
+        self.config(region_name='fake_region',
+                    group='keystone')
+        self.test_group = 'test_group'
+        self.cfg_fixture.conf.register_group(cfg.OptGroup(self.test_group))
+        ironic_auth.register_auth_opts(self.cfg_fixture.conf, self.test_group)
+        self.config(auth_type='password',
+                    group=self.test_group)
+        # NOTE(pas-ha) this is due to auth_plugin options
+        # being dynamically registered on first load,
+        # but we need to set the config before
+        plugin = kaloading.get_plugin_loader('password')
+        opts = kaloading.get_auth_plugin_conf_options(plugin)
+        self.cfg_fixture.register_opts(opts, group=self.test_group)
+        self.config(auth_url='http://127.0.0.1:9898',
+                    username='fake_user',
+                    password='fake_pass',
+                    project_name='fake_tenant',
+                    group=self.test_group)
+
+    def test_add_auth_opts(self):
+        opts = ironic_auth.add_auth_opts([])
+        # check that there is no duplicates
+        names = {o.dest for o in opts}
+        self.assertEqual(len(names), len(opts))
+        # NOTE(pas-ha) checking for most standard auth and session ones only
+        expected = {'timeout', 'insecure', 'cafile', 'certfile', 'keyfile',
+                    'auth_type', 'auth_url', 'username', 'password',
+                    'tenant_name', 'project_name', 'trust_id',
+                    'domain_id', 'user_domain_id', 'project_domain_id'}
+        self.assertTrue(expected.issubset(names))
+
+    def test_load_auth(self):
+        auth = ironic_auth.load_auth(self.cfg_fixture.conf, self.test_group)
+        # NOTE(pas-ha) 'password' auth_plugin is used
+        self.assertIsInstance(auth, kaidentity.generic.password.Password)
+        self.assertEqual('http://127.0.0.1:9898', auth.auth_url)
+
+    def test_load_auth_missing_options(self):
+        # NOTE(pas-ha) 'password' auth_plugin is used,
+        # so when we set the required auth_url to None,
+        # MissingOption is raised
+        self.config(auth_url=None, group=self.test_group)
+        self.assertIsNone(ironic_auth.load_auth(
+            self.cfg_fixture.conf, self.test_group))

ironic/tests/unit/drivers/modules/test_deploy_utils.py

@@ -31,7 +31,6 @@ from ironic.common import boot_devices
 from ironic.common import dhcp_factory
 from ironic.common import exception
 from ironic.common import image_service
-from ironic.common import keystone
 from ironic.common import states
 from ironic.common import utils as common_utils
 from ironic.conductor import task_manager
@@ -1381,6 +1380,42 @@ class OtherFunctionTestCase(db_base.DbTestCase):
         utils.warn_about_unsafe_shred_parameters()
         self.assertTrue(log_mock.warning.called)
 
+    @mock.patch.object(utils, '_get_ironic_session')
+    @mock.patch('ironic.common.keystone.get_service_url')
+    def test_get_ironic_api_url_from_config(self, mock_get_url, mock_ks):
+        mock_sess = mock.Mock()
+        mock_ks.return_value = mock_sess
+        fake_api_url = 'http://foo/'
+        mock_get_url.side_effect = exception.KeystoneFailure
+        self.config(api_url=fake_api_url, group='conductor')
+        url = utils.get_ironic_api_url()
+        # also checking for stripped trailing slash
+        self.assertEqual(fake_api_url[:-1], url)
+        self.assertFalse(mock_get_url.called)
+
+    @mock.patch.object(utils, '_get_ironic_session')
+    @mock.patch('ironic.common.keystone.get_service_url')
+    def test_get_ironic_api_url_from_keystone(self, mock_get_url, mock_ks):
+        mock_sess = mock.Mock()
+        mock_ks.return_value = mock_sess
+        fake_api_url = 'http://foo/'
+        mock_get_url.return_value = fake_api_url
+        self.config(api_url=None, group='conductor')
+        url = utils.get_ironic_api_url()
+        # also checking for stripped trailing slash
+        self.assertEqual(fake_api_url[:-1], url)
+        mock_get_url.assert_called_with(mock_sess)
+
+    @mock.patch.object(utils, '_get_ironic_session')
+    @mock.patch('ironic.common.keystone.get_service_url')
+    def test_get_ironic_api_url_fail(self, mock_get_url, mock_ks):
+        mock_sess = mock.Mock()
+        mock_ks.return_value = mock_sess
+        mock_get_url.side_effect = exception.KeystoneFailure()
+        self.config(api_url=None, group='conductor')
+        self.assertRaises(exception.InvalidParameterValue,
+                          utils.get_ironic_api_url)
+
 
 class VirtualMediaDeployUtilsTestCase(db_base.DbTestCase):
 
@@ -1923,11 +1958,12 @@ class AgentMethodsTestCase(db_base.DbTestCase):
         self.assertEqual('fake_agent', options['ipa-driver-name'])
         self.assertEqual(0, options['coreos.configdrive'])
 
-    @mock.patch.object(keystone, 'get_service_url', autospec=True)
+    @mock.patch.object(utils, '_get_ironic_session')
-    def test_build_agent_options_keystone(self, get_url_mock):
+    def test_build_agent_options_keystone(self, session_mock):
-
         self.config(api_url=None, group='conductor')
-        get_url_mock.return_value = 'api-url'
+        sess = mock.Mock()
+        sess.get_endpoint.return_value = 'api-url'
+        session_mock.return_value = sess
         options = utils.build_agent_options(self.node)
         self.assertEqual('api-url', options['ipa-api-url'])
         self.assertEqual('fake_agent', options['ipa-driver-name'])

ironic/tests/unit/drivers/modules/test_inspector.py

@@ -16,7 +16,6 @@ import mock
 
 from ironic.common import driver_factory
 from ironic.common import exception
-from ironic.common import keystone
 from ironic.common import states
 from ironic.conductor import task_manager
 from ironic.drivers.modules import inspector
@@ -128,12 +127,17 @@ class InspectHardwareTestCase(BaseTestCase):
         task.process_event.assert_called_once_with('fail')
 
 
-@mock.patch.object(keystone, 'get_admin_auth_token', lambda: 'the token')
 @mock.patch.object(client, 'get_status')
 class CheckStatusTestCase(BaseTestCase):
     def setUp(self):
         super(CheckStatusTestCase, self).setUp()
         self.node.provision_state = states.INSPECTING
+        mock_session = mock.Mock()
+        mock_session.get_token.return_value = 'the token'
+        sess_patch = mock.patch.object(inspector, '_get_inspector_session',
+                                       return_value=mock_session)
+        sess_patch.start()
+        self.addCleanup(sess_patch.stop)
 
     def test_not_inspecting(self, mock_get):
         self.node.provision_state = states.MANAGEABLE

ironic/tests/unit/drivers/modules/test_iscsi_deploy.py

@@ -27,7 +27,6 @@ from oslo_utils import fileutils
 from ironic.common import dhcp_factory
 from ironic.common import driver_factory
 from ironic.common import exception
-from ironic.common import keystone
 from ironic.common import pxe_utils
 from ironic.common import states
 from ironic.common import utils
@@ -446,38 +445,22 @@ class IscsiDeployMethodsTestCase(db_base.DbTestCase):
             self.assertEqual(states.ACTIVE, self.node.target_provision_state)
             self.assertIsNotNone(self.node.last_error)
 
-    @mock.patch.object(keystone, 'get_service_url', autospec=True)
+    @mock.patch('ironic.drivers.modules.deploy_utils.get_ironic_api_url')
-    def test_validate_good_api_url_from_config_file(self, mock_ks):
+    def test_validate_good_api_url(self, mock_get_url):
-        # not present in the keystone catalog
+        mock_get_url.return_value = 'http://127.0.0.1:1234'
-        mock_ks.side_effect = exception.KeystoneFailure
-        self.config(group='conductor', api_url='http://foo')
         with task_manager.acquire(self.context, self.node.uuid,
                                   shared=True) as task:
             iscsi_deploy.validate(task)
-            self.assertFalse(mock_ks.called)
+        mock_get_url.assert_called_once_with()
 
-    @mock.patch.object(keystone, 'get_service_url', autospec=True)
+    @mock.patch('ironic.drivers.modules.deploy_utils.get_ironic_api_url')
-    def test_validate_good_api_url_from_keystone(self, mock_ks):
+    def test_validate_fail_no_api_url(self, mock_get_url):
-        # present in the keystone catalog
+        mock_get_url.side_effect = exception.InvalidParameterValue('Ham!')
-        mock_ks.return_value = 'http://127.0.0.1:1234'
-        # not present in the config file
-        self.config(group='conductor', api_url=None)
-        with task_manager.acquire(self.context, self.node.uuid,
-                                  shared=True) as task:
-            iscsi_deploy.validate(task)
-            mock_ks.assert_called_once_with()
-
-    @mock.patch.object(keystone, 'get_service_url', autospec=True)
-    def test_validate_fail_no_api_url(self, mock_ks):
-        # not present in the keystone catalog
-        mock_ks.side_effect = exception.KeystoneFailure
-        # not present in the config file
-        self.config(group='conductor', api_url=None)
         with task_manager.acquire(self.context, self.node.uuid,
                                   shared=True) as task:
             self.assertRaises(exception.InvalidParameterValue,
                               iscsi_deploy.validate, task)
-            mock_ks.assert_called_once_with()
+        mock_get_url.assert_called_once_with()
 
     def test_validate_invalid_root_device_hints(self):
         with task_manager.acquire(self.context, self.node.uuid,

releasenotes/notes/keystone-auth-3155762c524e44df.yaml

@@ -0,0 +1,43 @@
+---
+upgrade:
+  - |
+    New way of configuring access credentials for OpenStack services clients.
+    For each service both Keystone session options
+    (timeout, SSL-related ones) and Keystone auth_plugin options
+    (auth_url, auth_type and correspondig auth_plugin options)
+    should be specified in the config section for this service.
+    Config section affected are
+
+    * ``[neutron]`` for Neutron service user
+    * ``[glance]`` for Glance service user
+    * ``[swift]`` for Swift service user
+    * ``[inspector]`` for Ironic Inspector service user
+    * ``[service_catalog]`` *new section* for Ironic service user,
+      used to discover Ironic endpoint from Keystone Catalog
+
+    This enables fine tuning of authentification for each service.
+
+    Backward-compatible options handling is provided
+    using values from ``[keystone_authtoken]`` config section,
+    but operators are advised to switch to the new config options.
+    For more information on sessions, auth plugins and their settings,
+    please refer to _http://docs.openstack.org/developer/keystoneauth/
+
+  - |
+    Small change in semantics of default for ``[neutron]url`` option
+
+    * default is changed to None.
+    * In case when [neutron]auth_strategy is ``noauth``,
+      default means use ``http://$my_ip:9696``.
+    * In case when [neutron]auth_strategy is ``keystone``,
+      default means to resolve the endpoint from Keystone Catalog.
+
+  - New config section ``[service_catalog]`` for access credentials used
+    to discover Ironic API URL from Keystone Catalog.
+    Previousely credentials from ``[keystone_authtoken]`` section were used,
+    which is now deprecated for such purpose.
+fixes:
+  - Do not rely on keystonemiddleware config options for instantiating
+    clients for other OpenStack services.
+    This allows changing keystonemiddleware options from legacy ones
+    and thus support Keystone V3 for token validation.

requirements.txt

@@ -12,7 +12,7 @@ netaddr!=0.7.16,>=0.7.12 # BSD
 paramiko>=2.0 # LGPLv2.1+
 python-neutronclient>=4.2.0 # Apache-2.0
 python-glanceclient>=2.0.0 # Apache-2.0
-python-keystoneclient!=1.8.0,!=2.1.0,>=1.7.0 # Apache-2.0
+keystoneauth1>=2.10.0  # Apache-2.0
 ironic-lib>=2.0.0 # Apache-2.0
 python-swiftclient>=2.2.0 # Apache-2.0
 pytz>=2013.6 # MIT