diff --git a/doc/source/admin/index.rst b/doc/source/admin/index.rst index 72fae7c9ea..c6b3e03c03 100644 --- a/doc/source/admin/index.rst +++ b/doc/source/admin/index.rst @@ -31,5 +31,5 @@ the major upgrades procedures. monitoring-systems.rst backup-restore.rst troubleshooting.rst - upgrades/minor-updates.rst + upgrades/minor-upgrades.rst upgrades/major-upgrades.rst diff --git a/doc/source/admin/upgrades/major-upgrades-manual-upgrade.rst b/doc/source/admin/upgrades/major-upgrades-manual-upgrade.rst deleted file mode 100644 index e5a2f9a73a..0000000000 --- a/doc/source/admin/upgrades/major-upgrades-manual-upgrade.rst +++ /dev/null @@ -1,263 +0,0 @@ -.. _upgrading-manually: - -Upgrading manually -================== - -Deployers can run the upgrade steps manually. Manual upgrades are useful for -scoping the changes in the upgrade process (for example, in very large -deployments with strict SLA requirements), or performing other upgrade -automations beyond what is provided by OpenStack-Ansible. - -The steps detailed here match those performed by the ``run-upgrade.sh`` -script. You can safely run these steps multiple times. - -Check out the |current_release_formal_name| release -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Ensure that your OpenStack-Ansible code is on the latest -|current_release_formal_name| tagged release. - -.. parsed-literal:: - - # git checkout |latest_tag| - -Prepare the shell variables -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Define these variables to reduce typing when running the remaining upgrade -tasks. Because these environments variables are shortcuts, this step is -optional. If you prefer, you can reference the files directly during the -upgrade. - -From the ``openstack-ansible`` root directory, run the following commands: - -.. code-block:: console - - # export MAIN_PATH="$(pwd)" - # export SCRIPTS_PATH="${MAIN_PATH}/scripts" - # export UPGRADE_PLAYBOOKS="${SCRIPTS_PATH}/upgrade-utilities/playbooks" - -Deal with existing OpenStack-Ansible artifacts -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The deployment node may have previous branch artifacts. - -Unset the following environment variables first: - -.. code-block:: console - - # unset ANSIBLE_INVENTORY - -Optionally, take a backup of your environment: - -.. code-block:: console - - # tar zcf /openstack/previous-ansible_`date +%F_%H%M`.tar.gz /etc/openstack_deploy /etc/ansible/ /usr/local/bin/openstack-ansible.rc - -Bootstrap Ansible again -~~~~~~~~~~~~~~~~~~~~~~~ - -Bootstrap Ansible again to ensure that all OpenStack-Ansible role -dependencies are in place before you run playbooks from the -|current_release_formal_name| release. - -.. code-block:: console - - # ${SCRIPTS_PATH}/bootstrap-ansible.sh - -Change to the playbooks directory -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Change to the playbooks directory so that the OpenStack-Ansible dynamic -inventory is found automatically. - -.. code-block:: console - - # cd playbooks - -Preflight checks -~~~~~~~~~~~~~~~~ - -Before starting with the upgraded version, perform preflight checks to ensure -your environment is stable. If any of those checks fail, the upgrade should -stop to let the deployer chose what to do. - -Clean up old facts -~~~~~~~~~~~~~~~~~~ - -Some configurations have changed, so purge old facts before -the upgrade. For more information, see :ref:`fact-cleanup-playbook`. - -.. code-block:: console - - # openstack-ansible "${UPGRADE_PLAYBOOKS}/ansible_fact_cleanup.yml" - -Update configuration and environment files -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The user configuration files in the ``/etc/openstack_deploy/`` directory and -the environment layout in the ``/etc/openstack_deploy/env.d`` directory have -new name values added in |current_release_formal_name|. Update the files as -follows. For more information, see :ref:`config-change-playbook`. - -.. code-block:: console - - # openstack-ansible "${UPGRADE_PLAYBOOKS}/deploy-config-changes.yml" - -Update user secrets file -~~~~~~~~~~~~~~~~~~~~~~~~ - -|current_release_formal_name| introduces new user secrets to the stack. -These secrets are populated automatically when you run the following playbook. -For more information, see :ref:`user-secrets-playbook`. - -.. code-block:: console - - # openstack-ansible "${UPGRADE_PLAYBOOKS}/user-secrets-adjustment.yml" - -Clean up the pip.conf file -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The presence of the ``pip.conf`` file can cause build failures during the -upgrade to |current_release_formal_name|. This playbook removes the -``pip.conf`` file on all the physical servers and on the repo containers. -For more information, see :ref:`pip-conf-removal`. - -.. code-block:: console - - # openstack-ansible "${UPGRADE_PLAYBOOKS}/pip-conf-removal.yml" - -Clean up the ceph-ansible galaxy namespaced roles -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The ceph-ansible common roles are no longer namespaced with a galaxy-style -'.' (ie. ``ceph.ceph-common`` is now cloned as ``ceph-common``), due to a -change in the way upstream meta dependencies are handled in the ceph roles. -The roles will be cloned according to the new naming, and an upgrade -playbook ``ceph-galaxy-removal.yml`` has been added to clean up the stale -galaxy-named roles. - -.. code-block:: console - - # openstack-ansible "${UPGRADE_PLAYBOOKS}/ceph-galaxy-removal.yml" - -Upgrade hosts -~~~~~~~~~~~~~ - -Before installing the infrastructure and OpenStack, update the host machines. - -.. code-block:: console - - # openstack-ansible setup-hosts.yml --limit '!galera_all:!rabbitmq_all' - -This command is the same setting up hosts on a new installation. The -``galera_all`` and ``rabbitmq_all`` host groups are excluded to prevent -reconfiguration and restarting of any of those containers. - -Update the other LXC container configuration -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Update the other container configuration independently. - -.. code-block:: console - - # openstack-ansible lxc-containers-create.yml -e \ - 'lxc_container_allow_restarts=false' --limit 'galera_all:rabbitmq_all' - -This command is a subset of the host setup playbook, limited to the -``galera_all`` and ``rabbitmq_all`` host groups. The configuration of those -containers is updated but a restart forany changes to take effect is deferred -to another playbook or later (see the next section). - -Perform a controlled rolling restart of the Galera containers -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Restart containers one at a time, ensuring that each is started, responding, -and synchronized with the other nodes in the cluster before moving on to the -next. This step allows the LXC container configuration that you applied earlier -to take effect, ensuring that the containers are restarted in a controlled -fashion. - -.. code-block:: console - - # openstack-ansible "${UPGRADE_PLAYBOOKS}/galera-cluster-rolling-restart.yml" - -Update repository servers -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Update the configuration of the repository servers and build new packages -required by the |current_release_formal_name| release. - -.. code-block:: console - - # openstack-ansible repo-install.yml - -Update HAProxy configuration -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Install and update any new or changed HAProxy service configurations. - -.. code-block:: console - - # openstack-ansible haproxy-install.yml - -Use the repository servers -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Now all containers can be pointed to the repo server's VIPs. - -.. code-block:: console - - # openstack-ansible repo-use.yml - -Upgrade the MariaDB version -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Upgrade MariaDB to the most recent 10.x minor release across the cluster. - -.. code-block:: console - - # openstack-ansible galera-install.yml -e 'galera_upgrade=true' - -Upgrade the infrastructure -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The following commands perform all of the steps from the setup-infrastructure -playbook, except for ``repo-install.yml``, ``haproxy-install.yml``, and -``galera-install.yml`` which you ran earlier. -Running these playbook applies the relevant |current_release_formal_name| -settings and packages. - -For certain versions of |previous_release_formal_name|, you must upgrade -the RabbitMQ service. - -For more information, see :ref:`setup-infra-playbook`. - -.. code-block:: console - - # openstack-ansible unbound-install.yml - # openstack-ansible memcached-install.yml - # openstack-ansible rabbitmq-install.yml -e 'rabbitmq_upgrade=true' - # openstack-ansible etcd-install.yml - # openstack-ansible utility-install.yml - # openstack-ansible rsyslog-install.yml - -Flush Memcached cache -~~~~~~~~~~~~~~~~~~~~~ - -Flush all of the caches in Memcached. For more information, -see :ref:`memcached-flush`. - -.. code-block:: console - - # openstack-ansible "${UPGRADE_PLAYBOOKS}/memcached-flush.yml" - -Upgrade OpenStack -~~~~~~~~~~~~~~~~~ - -Upgrade the OpenStack components with the same installation -playbook, without any additional options. - -.. code-block:: console - - # openstack-ansible setup-openstack.yml diff --git a/doc/source/admin/upgrades/major-upgrades-with-script.rst b/doc/source/admin/upgrades/major-upgrades-with-script.rst deleted file mode 100644 index 459b4be3c9..0000000000 --- a/doc/source/admin/upgrades/major-upgrades-with-script.rst +++ /dev/null @@ -1,29 +0,0 @@ -.. _upgrading-by-using-a-script: - -Upgrading by using a script -=========================== - -The |current_release_formal_name| release series of OpenStack-Ansible contains -the code for migrating from |previous_release_formal_name| to -|current_release_formal_name|. - -Running the upgrade script -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To upgrade from |previous_release_formal_name| to |current_release_formal_name| -by using the upgrade script, perform the following steps in the -``openstack-ansible`` directory: - -#. Change directory to the repository clone root directory: - - .. code-block:: console - - # cd /opt/openstack-ansible - -#. Run the following commands: - - .. parsed-literal:: - - # git checkout |latest_tag| - # ./scripts/run-upgrade.sh - diff --git a/doc/source/admin/upgrades/major-upgrades.rst b/doc/source/admin/upgrades/major-upgrades.rst index eb740cfcf0..c44c3c1894 100644 --- a/doc/source/admin/upgrades/major-upgrades.rst +++ b/doc/source/admin/upgrades/major-upgrades.rst @@ -16,16 +16,7 @@ Introduction For upgrades between major versions, the OpenStack-Ansible repository provides playbooks and scripts to upgrade an environment. The ``run-upgrade.sh`` script runs each upgrade playbook in the correct order, or playbooks can be run -individually if necessary. Alternatively, a deployer can upgrade manually. A -major upgrade process performs the following actions: - -- Modifies files residing in the ``/etc/openstack_deploy`` directory, to - reflect new configuration values. -- Places flag files that are created by the migration scripts in order to - achieve idempotency. These files are placed in the |upgrade_backup_dir| - directory. -- Upgrades the infrastructure servers. - See :ref:`setup-infra-playbook` for details. +individually if necessary. Alternatively, a deployer can upgrade manually. For more information about the major upgrade process, see :ref:`upgrading-by-using-a-script` and :ref:`upgrading-manually`. @@ -34,5 +25,178 @@ For more information about the major upgrade process, see |upgrade_warning| Test this on a development environment first. -.. include:: major-upgrades-with-script.rst -.. include:: major-upgrades-manual-upgrade.rst +.. _upgrading-by-using-a-script: + +Upgrading by using a script +=========================== + +The |current_release_formal_name| release series of OpenStack-Ansible contains +the code for migrating from |previous_release_formal_name| to +|current_release_formal_name|. + +Running the upgrade script +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To upgrade from |previous_release_formal_name| to |current_release_formal_name| +by using the upgrade script, perform the following steps in the +``openstack-ansible`` directory: + +#. Change directory to the repository clone root directory: + + .. code-block:: console + + # cd /opt/openstack-ansible + +#. Run the following commands: + + .. parsed-literal:: + + # git checkout |latest_tag| + # ./scripts/run-upgrade.sh + +For more information about the steps performed by the script, see +:ref:`upgrading-manually`. + +.. _upgrading-manually: + +Upgrading manually +================== + +Manual upgrades are useful for scoping the changes in the upgrade process +(for example, in very large deployments with strict SLA requirements), or +performing other upgrade automation beyond that provided by OpenStack-Ansible. + +The steps detailed here match those performed by the ``run-upgrade.sh`` +script. You can safely run these steps multiple times. + +Preflight checks +~~~~~~~~~~~~~~~~ + +Before starting with the upgrade, perform preflight health checks to ensure +your environment is stable. If any of those checks fail, ensure that the issue +is resolved before continuing. + +Check out the |current_release_formal_name| release +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Ensure that your OpenStack-Ansible code is on the latest +|current_release_formal_name| tagged release. + +.. parsed-literal:: + + # git checkout |latest_tag| + +Prepare the shell variables +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Define these variables to reduce typing when running the remaining upgrade +tasks. Because these environments variables are shortcuts, this step is +optional. If you prefer, you can reference the files directly during the +upgrade. + +.. code-block:: console + + # cd /opt/openstack-ansible + # export MAIN_PATH="$(pwd)" + # export SCRIPTS_PATH="${MAIN_PATH}/scripts" + +Backup the existing OpenStack-Ansible configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Make a backup of the configuration of the environment: + +.. parsed-literal:: + + # source_series_backup_file="/openstack/backup-openstack-ansible-|previous_series_name|.tar.gz" + # tar zcf ${source_series_backup_file} /etc/openstack_deploy /etc/ansible/ /usr/local/bin/openstack-ansible.rc + +Bootstrap the new Ansible and OSA roles +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To ensure that there is no currently set ANSIBLE_INVENTORY to override +the default inventory location, we unset the environment variable. + +.. code-block:: console + + # unset ANSIBLE_INVENTORY + +Bootstrap Ansible again to ensure that all OpenStack-Ansible role +dependencies are in place before you run playbooks from the +|current_release_formal_name| release. + +.. code-block:: console + + # ${SCRIPTS_PATH}/bootstrap-ansible.sh + +Change to the playbooks directory +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Change to the playbooks directory to simplify the CLI commands from here on +in the procedure, given that most playbooks executed are in this directory. + +.. code-block:: console + + # cd playbooks + +Implement changes to OSA configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If there have been any OSA variable name changes or environment/inventory +changes, there is a playbook to handle those changes to ensure service +continuity in the environment when the new playbooks run. The playbook is +tagged to ensure that any part of it can be executed on its own or skipped. +Please review the contents of the playbook for more information. + +.. code-block:: console + + # openstack-ansible "${SCRIPTS_PATH}/upgrade-utilities/deploy-config-changes.yml" + +Upgrade hosts +~~~~~~~~~~~~~ + +Before installing the infrastructure and OpenStack, update the host machines. + +.. code-block:: console + + # openstack-ansible setup-hosts.yml --limit '!galera_all:!rabbitmq_all' + +This command is the same setting up hosts on a new installation. The +``galera_all`` and ``rabbitmq_all`` host groups are excluded to prevent +reconfiguration and restarting of any of those containers as they need to +be updated, but not restarted. + +Once that is complete, upgrade the final host groups with the flag to prevent +container restarts. + +.. code-block:: console + + # openstack-ansible setup-hosts.yml -e 'lxc_container_allow_restarts=false' --limit 'galera_all:rabbitmq_all' + +Upgrade infrastructure +~~~~~~~~~~~~~~~~~~~~~~ + +We can now go ahead with the upgrade of all the infrastructure components. To +ensure that rabbitmq and mariadb are upgraded, we pass the appropriate flags. + +.. code-block:: console + + # openstack-ansible setup-infrastructure.yml -e 'galera_upgrade=true' -e 'rabbitmq_upgrade=true' + +With this complete, we can no restart the mariadb containers one at a time, +ensuring that each is started, responding, and synchronized with the other +nodes in the cluster before moving on to the next steps. This step allows +the LXC container configuration that you applied earlier to take effect, +ensuring that the containers are restarted in a controlled fashion. + +.. code-block:: console + + # openstack-ansible "${SCRIPTS_PATH}/upgrade-utilities/galera-cluster-rolling-restart.yml" + +Upgrade OpenStack +~~~~~~~~~~~~~~~~~ + +We can now go ahead with the upgrade of all the OpenStack components. + +.. code-block:: console + + # openstack-ansible setup-openstack.yml diff --git a/doc/source/admin/upgrades/minor-updates.rst b/doc/source/admin/upgrades/minor-upgrades.rst similarity index 100% rename from doc/source/admin/upgrades/minor-updates.rst rename to doc/source/admin/upgrades/minor-upgrades.rst diff --git a/doc/source/conf.py b/doc/source/conf.py index 343935cd0d..c5ae2ca114 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -55,7 +55,7 @@ if current_series == "latest": else: watermark = series_names = current_series.capitalize() git_cmd = ["git", "describe", "--abbrev=0", "--tags"] - latest_tag = subprocess.Popen(git_cmd,stdout=subprocess.PIPE).communicate()[0].strip('\n') + latest_tag = subprocess.Popen(git_cmd,stdout=subprocess.PIPE).communicate()[0].strip('\n') branch = "stable/{}".format(current_series) upgrade_warning = "The upgrade is always under active development." @@ -66,7 +66,6 @@ with open(os.path.join(CONF_PATH, GNOCCHI_DETAILS), 'r') as fdesc: gnocchi_branch = result.groups()[0] # References variable for substitutions -upgrade_backup_dir = "``/etc/openstack_deploy.{branchname}``".format(branchname=previous_series_name.upper()) deploy_guide_prefix = "http://docs.openstack.org/project-deploy-guide/openstack-ansible/{}/%s".format(current_series) dev_docs_prefix = "http://docs.openstack.org/openstack-ansible/{}/%s".format(current_series) @@ -74,17 +73,17 @@ dev_docs_prefix = "http://docs.openstack.org/openstack-ansible/{}/%s".format(cur rst_epilog = """ .. |current_release_git_branch_name| replace:: {current_release_git_branch_name} .. |current_release_gnocchi_git_branch_name| replace:: {current_release_gnocchi_git_branch_name} +.. |previous_series_name| replace:: {previous_series_name} .. |previous_release_formal_name| replace:: {previous_release_formal_name} .. |current_release_formal_name| replace:: {current_release_formal_name} -.. |upgrade_backup_dir| replace:: {upgrade_backup_dir} .. |latest_tag| replace:: {latest_tag} .. |upgrade_warning| replace:: {upgrade_warning} """.format( current_release_git_branch_name=branch, current_release_gnocchi_git_branch_name=gnocchi_branch, + previous_series_name=previous_series_name, previous_release_formal_name=previous_series_name.capitalize(), current_release_formal_name=current_series_name.capitalize(), - upgrade_backup_dir=upgrade_backup_dir, latest_tag=latest_tag, upgrade_warning=upgrade_warning, ) diff --git a/doc/source/reference/index.rst b/doc/source/reference/index.rst index 5fa45df90b..5e601cf24d 100644 --- a/doc/source/reference/index.rst +++ b/doc/source/reference/index.rst @@ -28,4 +28,3 @@ see the :dev_docs:`Contributors Guide `. configuration/advanced-config.rst architecture/index.rst commands/reference.rst - upgrades/reference.rst diff --git a/doc/source/reference/upgrades/reference-scripts.rst b/doc/source/reference/upgrades/reference-scripts.rst deleted file mode 100644 index 1c20b89920..0000000000 --- a/doc/source/reference/upgrades/reference-scripts.rst +++ /dev/null @@ -1,45 +0,0 @@ -Major Upgrade Scripts -===================== - -This section describes in detail the scripts that are used in the upgrade -process. - -Within the main :file:`scripts` directory there is an :file:`upgrade-utilities` -directory, which contains additional scripts that facilitate the initial -upgrade process. - -run-upgrade.sh -~~~~~~~~~~~~~~ - -The ``run-upgrade.sh`` script controls the overall upgrade process for -deployers who do not want to upgrade manually. It provides the following -environment variables: - -* ``SCRIPTS_PATH`` - The path to the top level scripts directory -* ``MAIN_PATH`` - The ``openstack_ansible`` root directory. -* ``UPGRADE_PLAYBOOKS`` - The path to the playbooks used in upgrading - -The upgrade script also bootstraps OpenStack-Ansible (using -``bootstrap-ansible.sh``) in order to provide the new role dependencies -introduced in the |previous_release_formal_name| series. - -.. _migrate-os-vars: - -migrate_openstack_vars.py -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Upstream decisions influenced the change of some variable names in -|current_release_formal_name|. This script replaces any instances of these -strings in the variable override files matching the pattern -``/etc/openstack_deploy/user_*.yml``. -Variable names within comments are updated. - -This script creates files in the form ``VARS_MIGRATED_`` and -places them in |upgrade_backup_dir| directory. -For example, after the script processes the -``/etc/openstack_deploy/user_variables.yml`` file, it creates the -``VARS_MIGRATED_user_variables.yml`` file in the |upgrade_backup_dir| -directory. This indicates to OpenStack-Ansible to skip this step on successive -runs. The script itself does not check for this file. - -This script is called by the :ref:`config-change-playbook`. diff --git a/doc/source/reference/upgrades/reference-upgrade-playbooks.rst b/doc/source/reference/upgrades/reference-upgrade-playbooks.rst deleted file mode 100644 index 576750f65f..0000000000 --- a/doc/source/reference/upgrades/reference-upgrade-playbooks.rst +++ /dev/null @@ -1,101 +0,0 @@ -Major Upgrade Playbooks -======================= - -This section provides details about the playbooks that are used in the -upgrade process. Within the main :file:`scripts` directory there is an -:file:`upgrade-utilities` directory, which contains an additional playbooks -directory. These playbooks facilitate the upgrade process. - -.. _fact-cleanup-playbook: - -ansible_fact_cleanup.yml -~~~~~~~~~~~~~~~~~~~~~~~~ - -This playbook calls a script to remove files in the -``/etc/openstack_deploy/ansible_facts/`` directory. - -.. _config-change-playbook: - -deploy-config-changes.yml -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -This playbook backs up the ``/etc/openstack_deploy`` directory before -changing the configuration. - -The``/etc/openstack_deploy`` directory is copied once to the -|upgrade_backup_dir| directory. - -.. _user-secrets-playbook: - -user-secrets-adjustment.yml -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -This playbook ensures that the user secrets file is updated based on the -example file in the main repository, making it possible to guarantee that all -secrets move into the upgraded environment and are generated appropriately. -This playbook adds only new secrets, such as those necessary for new services -or new settings added to existing services. Values that were set previously are -not changed. - -.. _pip-conf-removal: - -pip-conf-removal.yml -~~~~~~~~~~~~~~~~~~~~ - -The presence of the ``pip.conf`` file locks down all Python installations to -packages on the repo servers. If this file exists on a repo server or a -physical node, it causes a circular dependency issue and the upgrade fails. -This playbook removes the file on all the repo servers and physical nodes. - -.. _ceph-galaxy-removal: - -ceph-galaxy-removal.yml -~~~~~~~~~~~~~~~~~~~~~~~ - -The ceph-ansible common roles are no longer namespaced with a galaxy-style -'.' (ie. ``ceph.ceph-common`` is now cloned as ``ceph-common``), due to a -change in the way upstream meta dependencies are handled in the ceph roles. -The roles will be cloned according to the new naming, and an upgrade -playbook ``ceph-galaxy-removal.yml`` has been added to clean up the stale -galaxy-named roles. - -.. _setup-infra-playbook: - -setup-infrastructure.yml -~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``playbooks`` directory contains the ``setup-infrastructure.yml`` playbook. -The ``run-upgrade.sh`` script calls the ``setup-insfrastructure.yml`` playbook -with specific arguments to upgrade MariaDB and RabbitMQ. - -For example, to run an upgrade for both components at once, run the following -commands: - -.. code-block:: console - - # openstack-ansible setup-infrastructure.yml -e 'rabbitmq_upgrade=true' \ - -e 'galera_upgrade=true' - -The ``rabbitmq_upgrade`` variable tells the ``rabbitmq_server`` role to -upgrade RabbitMQ. - -.. note:: - The RabbitMQ server role installs patch releases automatically, - regardless of the value of ``rabbitmq_upgrade``. This variable - controls the upgrade of only the major or minor versions. - - Upgrading RabbitMQ in the |current_release_formal_name| release is optional. The - ``run-upgrade.sh`` script does not automatically upgrade it. To upgrade - RabbitMQ, insert the ``rabbitmq_upgrade: true`` - line into a file, such as ``/etc/openstack_deploy/user_variables.yml``. - -The ``galera_upgrade`` variable tells the ``galera_server`` role to remove the -current version of MariaDB and Galera and upgrade to the 10.*x* series. - -.. _memcached-flush: - -memcached-flush.yml -~~~~~~~~~~~~~~~~~~~ - -This playbook sends the ``flush_all`` command to Memcached with the help of -netcat. diff --git a/doc/source/reference/upgrades/reference.rst b/doc/source/reference/upgrades/reference.rst deleted file mode 100644 index ace55d8422..0000000000 --- a/doc/source/reference/upgrades/reference.rst +++ /dev/null @@ -1,7 +0,0 @@ -======================== -Major upgrades reference -======================== - - -.. include :: reference-scripts.rst -.. include :: reference-upgrade-playbooks.rst diff --git a/scripts/run-upgrade.sh b/scripts/run-upgrade.sh index 53b79bfa0e..8d31f5236f 100755 --- a/scripts/run-upgrade.sh +++ b/scripts/run-upgrade.sh @@ -30,9 +30,6 @@ export SCRIPTS_PATH="$(dirname "$(readlink -f "${0}")")" # The git checkout root path export MAIN_PATH="$(dirname "${SCRIPTS_PATH}")" -# The path to find all the upgrade playbooks -export UPGRADE_PLAYBOOKS="${SCRIPTS_PATH}/upgrade-utilities/playbooks" - # The expected source series name export SOURCE_SERIES="rocky" @@ -157,41 +154,29 @@ function main { pre_flight check_for_current - # ANSIBLE_INVENTORY location has changed between P and Q, so we ensure - # we don't point to previous inventory. - unset ANSIBLE_INVENTORY + # Backup source series artifacts + source_series_backup_file="/openstack/backup-openstack-ansible-${SOURCE_SERIES}.tar.gz" + if [[ ! -e ${source_series_backup_file} ]]; then + tar zcf ${source_series_backup_file} /etc/openstack_deploy /etc/ansible/ /usr/local/bin/openstack-ansible.rc + fi - # Archive previous version artifacts - tar zcf /openstack/previous-ansible_`date +%F_%H%M`.tar.gz /etc/openstack_deploy /etc/ansible/ /usr/local/bin/openstack-ansible.rc + # ANSIBLE_INVENTORY may be set to a previous/incorrect location. To + # ensure this is not the case, we unset the environment variable. + unset ANSIBLE_INVENTORY "${SCRIPTS_PATH}/bootstrap-ansible.sh" pushd ${MAIN_PATH}/playbooks - RUN_TASKS+=("${UPGRADE_PLAYBOOKS}/ansible_fact_cleanup.yml") - RUN_TASKS+=("${UPGRADE_PLAYBOOKS}/deploy-config-changes.yml") - RUN_TASKS+=("${UPGRADE_PLAYBOOKS}/user-secrets-adjustment.yml") - RUN_TASKS+=("${UPGRADE_PLAYBOOKS}/pip-conf-removal.yml") - RUN_TASKS+=("${UPGRADE_PLAYBOOKS}/ceph-galaxy-removal.yml") + RUN_TASKS+=("${SCRIPTS_PATH}/upgrade-utilities/deploy-config-changes.yml") # we don't want to trigger container restarts for these groups yet RUN_TASKS+=("setup-hosts.yml --limit '!galera_all:!rabbitmq_all'") # add new container config to containers but don't restart - RUN_TASKS+=("lxc-containers-create.yml -e 'lxc_container_allow_restarts=false' --limit 'galera_all:rabbitmq_all'") - # setup infra - RUN_TASKS+=("unbound-install.yml") - RUN_TASKS+=("repo-install.yml") - RUN_TASKS+=("haproxy-install.yml") - RUN_TASKS+=("repo-use.yml") - # explicitly perform mariadb upgrade - RUN_TASKS+=("galera-install.yml -e 'galera_upgrade=true'") - # explicitly perform controlled galera cluster restart - RUN_TASKS+=("${UPGRADE_PLAYBOOKS}/galera-cluster-rolling-restart.yml") - # individually run each of the remaining plays from setup-infrastructure - RUN_TASKS+=("memcached-install.yml") - RUN_TASKS+=("rabbitmq-install.yml -e 'rabbitmq_upgrade=true'") - RUN_TASKS+=("etcd-install.yml") - RUN_TASKS+=("utility-install.yml") - RUN_TASKS+=("rsyslog-install.yml") - RUN_TASKS+=("${UPGRADE_PLAYBOOKS}/memcached-flush.yml") + RUN_TASKS+=("setup-hosts.yml -e 'lxc_container_allow_restarts=false' --limit 'galera_all:rabbitmq_all'") + # upgrade infrastructure + RUN_TASKS+=("setup-infrastructure.yml -e 'galera_upgrade=true' -e 'rabbitmq_upgrade=true'") + # explicitly perform controlled galera cluster restart with new lxc config + RUN_TASKS+=("${SCRIPTS_PATH}/upgrade-utilities/galera-cluster-rolling-restart.yml") + # upgrade openstack RUN_TASKS+=("setup-openstack.yml") # Run the tasks in order for item in ${!RUN_TASKS[@]}; do diff --git a/scripts/upgrade-utilities/deploy-config-changes.yml b/scripts/upgrade-utilities/deploy-config-changes.yml new file mode 100644 index 0000000000..8f256cf3d4 --- /dev/null +++ b/scripts/upgrade-utilities/deploy-config-changes.yml @@ -0,0 +1,97 @@ +--- +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: Prepare environment and configuration for deploying the new release + hosts: localhost + connection: local + gather_facts: false + user: root + vars: + repo_root_dir: "{{ (playbook_dir ~ '/../../') | realpath }}" + ansible_python_interpreter: "/usr/bin/python" + tasks: + - name: Remove unnecessary env.d override files + shell: | + set -e + exit_code=0 + if [[ -e /etc/openstack_deploy/env.d ]]; then + for f in $(diff --brief --report-identical-files /etc/openstack_deploy/env.d /opt/openstack-ansible/inventory/env.d | awk '/identical/ {print $2}' 2>/dev/null); do + echo "Deleting ${f} because it is identical to the defaults." + rm -f ${f} + exit_code=2 + done + fi + exit ${exit_code} + args: + executable: /bin/bash + register: _envd_dir_cleanup + changed_when: _envd_dir_cleanup.rc == 2 + failed_when: _envd_dir_cleanup.rc not in [0,2] + tags: + - identical-envd-file-cleanup + + - name: Find any config files in the user-space env.d directory + find: + paths: + - "/etc/openstack_deploy/env.d" + patterns: '*.yml' + register: _envd_dir_contents + tags: + - custom-envd-file-check + + - name: Halt the upgrade and warn the user to inspect the env.d files for changes + fail: + msg: | + There are files in /etc/openstack_deploy/env.d which override the default inventory + layout in {{ repo_root_dir }}/inventory/env.d. The difference between these files + should be carefully reviewed to understand whether the changes are still necessary + and applicable to the environment. If all the user-space env.d files are necessary, + then please re-run this playbook with the CLI option '--skip-tags custom-envd-file-check'. + when: + - _envd_dir_contents.matched > 0 + tags: + - custom-envd-file-check + + - name: Read example user secrets file + shell: "grep '^[a-zA-Z]' {{ repo_root_dir }}/etc/openstack_deploy/user_secrets.yml" + register: new_secrets + tags: + - update-secrets + + - name: Read existing user secrets file + shell: "grep '^[a-zA-Z]' /etc/openstack_deploy/user_secrets.yml" + register: user_secrets + tags: + - update-secrets + + - name: Add missing secrets + lineinfile: + dest: "/etc/openstack_deploy/user_secrets.yml" + line: "{{ item }}" + with_items: "{{ new_secrets.stdout_lines }}" + when: + - "user_secrets.stdout.find(item) == -1" + tags: + - update-secrets + + - name: Generate new secrets + shell: "{{ repo_root_dir }}/scripts/pw-token-gen.py --file /etc/openstack_deploy/user_secrets.yml" + tags: + - update-secrets + + - name: Remove fact cache to ensure a fresh one is built during the upgrade + file: + path: "/etc/openstack_deploy/ansible_facts" + state: absent + tags: + - remove-fact-cache diff --git a/scripts/upgrade-utilities/playbooks/galera-cluster-rolling-restart.yml b/scripts/upgrade-utilities/galera-cluster-rolling-restart.yml similarity index 100% rename from scripts/upgrade-utilities/playbooks/galera-cluster-rolling-restart.yml rename to scripts/upgrade-utilities/galera-cluster-rolling-restart.yml diff --git a/scripts/upgrade-utilities/playbooks/ansible_fact_cleanup.yml b/scripts/upgrade-utilities/playbooks/ansible_fact_cleanup.yml deleted file mode 100644 index 61b158482e..0000000000 --- a/scripts/upgrade-utilities/playbooks/ansible_fact_cleanup.yml +++ /dev/null @@ -1,26 +0,0 @@ ---- -# Copyright 2015, Rackspace US, Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -- name: Ansible fact cleanup - hosts: localhost - connection: local - gather_facts: false - user: root - tasks: - - name: Remove any of the stored facts ansible may already have - command: "{{ upgrade_scripts }}/ansible_fact_cleanup.sh" - vars: - upgrade_scripts: "{{ playbook_dir }}/../scripts" - ansible_python_interpreter: "/usr/bin/python" diff --git a/scripts/upgrade-utilities/playbooks/ceph-galaxy-removal.yml b/scripts/upgrade-utilities/playbooks/ceph-galaxy-removal.yml deleted file mode 100644 index 92f93c032b..0000000000 --- a/scripts/upgrade-utilities/playbooks/ceph-galaxy-removal.yml +++ /dev/null @@ -1,31 +0,0 @@ ---- -# Copyright 2017, Logan Vig -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -- name: Remove the ceph galaxy named roles - hosts: localhost - gather_facts: true - user: root - tasks: - # These roles used to be named using galaxy format due to the naming of the - # meta dependencies in ceph-ansible. Now the meta dependencies no longer - # exist so we are free to name them more consistently with the rest of - # OSA's roles. - - name: Remove ceph galaxy named roles if found - file: - path: "/etc/ansible/roles/{{ item }}" - state: "absent" - with_items: - - ceph.ceph-docker-common - - ceph.ceph-common diff --git a/scripts/upgrade-utilities/playbooks/deploy-config-changes.yml b/scripts/upgrade-utilities/playbooks/deploy-config-changes.yml deleted file mode 100644 index d61adc7649..0000000000 --- a/scripts/upgrade-utilities/playbooks/deploy-config-changes.yml +++ /dev/null @@ -1,89 +0,0 @@ ---- -# Copyright 2016, Rackspace US, Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -- name: Upgrade environment/inventory configuration - hosts: localhost - connection: local - gather_facts: false - user: root - vars: - upgrade_scripts: "{{ playbook_dir }}/../scripts" - repo_root_dir: "{{ playbook_dir }}/../../../" - source_series: "rocky" - ansible_python_interpreter: "/usr/bin/python" - tasks: - - name: Create an old copy of openstack_deploy - copy: - src: "/etc/openstack_deploy/" - dest: "/etc/openstack_deploy.{{ source_series | upper }}/" - force: no - - - name: Check if there is a user-space env.d directory - stat: - path: "/etc/openstack_deploy/env.d" - register: _envd_dir - - - block: - - - name: Retrieve differences - shell: rsync -avun "{{ repo_root_dir }}/inventory/env.d/" "/etc/openstack_deploy/env.d/" | grep "yml$" - failed_when: false - register: diff_result - - - name: Copy new env.d files into place - copy: - src: "{{ repo_root_dir }}/inventory/env.d/{{ item }}" - dest: "/etc/openstack_deploy/env.d/{{ item }}" - force: no - with_items: - - "{{ diff_result.stdout_lines }}" - when: diff_result.stdout != "" - - - name: Check result for emptiness - debug: msg="All new env.d files are placed in the stock repo. No new changes" - when: diff_result.stdout == "" - - when: _envd_dir.stat.exists | bool - - - name: Update OpenStack variable names - command: "{{ upgrade_scripts }}/migrate_openstack_vars.py {{ item }} {{ (item | basename)[:-4] }}" - args: - creates: "/etc/openstack_deploy.{{ source_series | upper }}/VARS_MIGRATED_{{ (item | basename)[:-4] }}" - with_fileglob: - - "/etc/openstack_deploy/user_*.yml" - - - name: Write vars required for upgrade - lineinfile: - dest: /etc/openstack_deploy/user_variables.yml - regexp: "{{ item.regexp | default('^' + item.key) }}" - line: "{{ item.key }}: {{ item.value }}" - backrefs: "{{ item.backrefs | default('no') }}" - state: present - with_items: - # Replace "10.0" with "10.1" within galera_repo_url - - key: "galera_repo_url" - regexp: '^galera_repo_url: (https?://.*10)\.0(.*)$' - value: '\1.1\2' - backrefs: yes - # Replace "10.0" with "10.1" within galera_client_apt_repo_url - - key: "galera_client_apt_repo_url" - regexp: '^galera_client_apt_repo_url: (https?://.*10)\.0(.*)$' - value: '\1.1\2' - backrefs: yes - # Replace "10.0" with "10.1" within galera_client_repo_url - - key: "galera_client_repo_url" - regexp: '^galera_client_repo_url: (https?://.*10)\.0(.*)$' - value: '\1.1\2' - backrefs: yes diff --git a/scripts/upgrade-utilities/playbooks/memcached-flush.yml b/scripts/upgrade-utilities/playbooks/memcached-flush.yml deleted file mode 100644 index 791fc2bb90..0000000000 --- a/scripts/upgrade-utilities/playbooks/memcached-flush.yml +++ /dev/null @@ -1,34 +0,0 @@ ---- -# Copyright 2016, Rackspace US, Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -- name: Memcached cache flush - hosts: memcached - gather_facts: true - user: root - vars: - memcached_conf_dest: - debian: "/etc/memcached.conf" - redhat: "/etc/sysconfig/memcached" - netcat_package: - debian: "netcat-openbsd" - redhat: "nmap-ncat" - tasks: - - name: Ensuring netcat is installed - package: - name: "{{ netcat_package.get(ansible_os_family | lower) }}" - state: present - - name: Flush all of the cache in memcached - shell: | - echo 'flush_all' | nc $(awk '/^\-l/ {print $2}' {{ memcached_conf_dest.get(ansible_os_family | lower) }} | awk -F, '{ print $1 }') $(awk '/^\-p/ {print $2}' {{ memcached_conf_dest.get(ansible_os_family | lower) }} ) diff --git a/scripts/upgrade-utilities/playbooks/pip-conf-removal.yml b/scripts/upgrade-utilities/playbooks/pip-conf-removal.yml deleted file mode 100644 index bf29a6c6db..0000000000 --- a/scripts/upgrade-utilities/playbooks/pip-conf-removal.yml +++ /dev/null @@ -1,24 +0,0 @@ ---- -# Copyright 2015, Rackspace US, Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -- name: Remove pip.conf if found - hosts: hosts:repo_all - gather_facts: true - user: root - tasks: - - name: Remove pip.conf - file: - path: "{{ ansible_env.HOME }}/.pip/pip.conf" - state: "absent" diff --git a/scripts/upgrade-utilities/playbooks/user-secrets-adjustment.yml b/scripts/upgrade-utilities/playbooks/user-secrets-adjustment.yml deleted file mode 100644 index 30ddf095ee..0000000000 --- a/scripts/upgrade-utilities/playbooks/user-secrets-adjustment.yml +++ /dev/null @@ -1,84 +0,0 @@ ---- -# Copyright 2015, Rackspace US, Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -- name: User secrets adjustments - hosts: localhost - connection: local - gather_facts: false - user: root - tasks: - - name: Read example user secrets file - shell: "grep '^[a-zA-Z]' {{ playbook_dir }}/../../../etc/openstack_deploy/user_secrets.yml" - register: secrets - - - name: Rename changed secrets - lineinfile: - dest: "/etc/openstack_deploy/{{ _osa_secrets_file_name }}" - regexp: "^{{ item.old_name }}: (.*)$" - line: "{{ item.new_name }}: \\1" - backrefs: yes - with_items: - - old_name: "keystone_rabbitmq_password" - new_name: "keystone_oslomsg_rpc_password" - - old_name: "ceilometer_rabbitmq_password" - new_name: "ceilometer_oslomsg_rpc_password" - - old_name: "aodh_rabbitmq_password" - new_name: "aodh_oslomsg_rpc_password" - - old_name: "cinder_rabbitmq_password" - new_name: "cinder_oslomsg_rpc_password" - - old_name: "glance_rabbitmq_password" - new_name: "glance_oslomsg_rpc_password" - - old_name: "heat_rabbitmq_password" - new_name: "heat_oslomsg_rpc_password" - - old_name: "ironic_rabbitmq_password" - new_name: "ironic_oslomsg_rpc_password" - - old_name: "neutron_rabbitmq_password" - new_name: "neutron_oslomsg_rpc_password" - - old_name: "nova_rabbitmq_password" - new_name: "nova_oslomsg_rpc_password" - - old_name: "octavia_rabbitmq_password" - new_name: "octavia_oslomsg_rpc_password" - - old_name: "sahara_rabbitmq_password" - new_name: "sahara_oslomsg_rpc_password" - - old_name: "swift_rabbitmq_telemetry_password" - new_name: "swift_oslomsg_notify_password" - - old_name: "magnum_rabbitmq_password" - new_name: "magnum_oslomsg_rpc_password" - - old_name: "trove_rabbitmq_password" - new_name: "trove_oslomsg_rpc_password" - - old_name: "barbican_rabbitmq_password" - new_name: "barbican_oslomsg_rpc_password" - - old_name: "designate_rabbitmq_password" - new_name: "designate_oslomsg_rpc_password" - - old_name: "tacker_rabbitmq_password" - new_name: "tacker_oslomsg_rpc_password" - - - name: Read user secrets file - shell: "grep '^[a-zA-Z]' /etc/openstack_deploy/{{ _osa_secrets_file_name }}" - register: user_secrets - - - name: Add missing secrets - lineinfile: - dest: "/etc/openstack_deploy/{{ _osa_secrets_file_name }}" - line: "{{ item }}" - with_items: "{{ secrets.stdout_lines }}" - when: - - "user_secrets.stdout.find(item) == -1" - - - name: Generate new secrets - shell: "/opt/ansible-runtime/bin/python {{ playbook_dir }}/../../../scripts/pw-token-gen.py --file /etc/openstack_deploy/{{ _osa_secrets_file_name }}" - vars: - _osa_secrets_file_name: "{{ osa_secrets_file_name | default('user_secrets.yml') }}" - ansible_python_interpreter: "/usr/bin/python" diff --git a/scripts/upgrade-utilities/scripts/ansible_fact_cleanup.sh b/scripts/upgrade-utilities/scripts/ansible_fact_cleanup.sh deleted file mode 100755 index 9684d12256..0000000000 --- a/scripts/upgrade-utilities/scripts/ansible_fact_cleanup.sh +++ /dev/null @@ -1,18 +0,0 @@ -#!/usr/bin/env bash -# Copyright 2016, Rackspace US, Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# find all of the ansible_facts that may be saved and remove them -# prior to running the upgrade. -find /etc/openstack_deploy/ansible_facts/ -type f -exec rm {} \; diff --git a/scripts/upgrade-utilities/scripts/make_rst_table.py b/scripts/upgrade-utilities/scripts/make_rst_table.py deleted file mode 100755 index 594488dfdd..0000000000 --- a/scripts/upgrade-utilities/scripts/make_rst_table.py +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/env python -# Copyright 2016, Rackspace US, Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# This file is used to generate a ReStructured Text table suitable for -# documentating the variable name changes. Its contents are meant to be -# inserted into doc/source/upgrade-guide/scripts.rst. - -# As of right now, running this script and inserting the output into -# the file is manual. - -from migrate_openstack_vars import VAR_MAPPINGS - -# Print old/new values in each row, right aligned. -row_format = "| {:>40} | {:>40} |" - -# For the line separators, move the dividing '+' sign over so it's aligned -# with the '|' in the rows. -divider_format = "+-{:->42}---{:->40}" -header_divide_format = "+={:=>42}==={:=>40}" - - -# Header info -print(divider_format.format('+', '+')) -print(row_format.format('Old Value', 'New Value')) -print(header_divide_format.format('+', '+')) - -# If we just used the items method, we'd get an unsorted output. -keys = VAR_MAPPINGS.keys() -keys.sort() - -for key in keys: - print(row_format.format(key, VAR_MAPPINGS[key])) - print(divider_format.format('+', '+')) diff --git a/scripts/upgrade-utilities/scripts/migrate_openstack_vars.py b/scripts/upgrade-utilities/scripts/migrate_openstack_vars.py deleted file mode 100755 index 47f042503d..0000000000 --- a/scripts/upgrade-utilities/scripts/migrate_openstack_vars.py +++ /dev/null @@ -1,70 +0,0 @@ -#!/usr/bin/env python -# Copyright 2016, Rackspace US, Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# This could probably be made more generic, since the biggest change per -# service is the variable mappings -import sys - -VAR_MAPPINGS = { - # Add mapped items here - 'test-old': 'test-new' -} - - -def update_variables(old_contents): - """Replace all references to old variables. - - This includes comments and references within values for other variables. - """ - new_contents = [] - - for line in old_contents: - words = line.split() - - for word in words: - # Using the whitespace split above, the keys in the yaml file will - # have a : at the end, so we need to strip that off before - # replacing - if word.endswith(':'): - word = word[:-1] - - if word in VAR_MAPPINGS.keys(): - line = line.replace(word, VAR_MAPPINGS[word]) - - new_contents.append(line) - - return new_contents - - -def main(filename): - with open(filename, 'r') as f: - contents = f.readlines() - - new_contents = update_variables(contents) - - with open(filename, 'w') as f: - f.write(''.join(new_contents)) - -if __name__ == '__main__': - if len(sys.argv) < 3: - sys.exit("Filename and flag file reference required.") - - filename = sys.argv[1] - flag_ref = sys.argv[2] - main(filename) - - flag_file = '/etc/openstack_deploy.ROCKY/VARS_MIGRATED_%s' % flag_ref - with open(flag_file, 'w') as f: - f.write('OpenStack-Ansible variables migrated.') diff --git a/scripts/upgrade-utilities/scripts/test_migrate_openstack_vars.py b/scripts/upgrade-utilities/scripts/test_migrate_openstack_vars.py deleted file mode 100755 index 63edd134d5..0000000000 --- a/scripts/upgrade-utilities/scripts/test_migrate_openstack_vars.py +++ /dev/null @@ -1,86 +0,0 @@ -# Copyright 2016, Rackspace US, Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -from migrate_openstack_vars import main -from migrate_openstack_vars import VAR_MAPPINGS - -import os -import sys - - -FILE_NAME = 'test_user_variables.yml' - - -def set_up(): - # Create an example file with key/value pairs, as well as a comment - # The old to new value mappings are written to a file, then later the - # file is inspected to ensure no old values remain. - var_lines = ["{}: {}".format(key, val) for - key, val in VAR_MAPPINGS.items()] - var_lines.append('# A test comment') - sample = VAR_MAPPINGS.items()[0] - var_lines.append('# {} / {}'.format(*sample)) - with open(FILE_NAME, 'w') as f: - f.write('\n'.join(var_lines)) - - -def teardown(): - # Remove files so they don't pollute the directories. - os.remove(FILE_NAME) - -def test(): - main(FILE_NAME) - - with open(FILE_NAME, 'r') as f: - contents = f.readlines() - - for line in contents: - # only split lines that look like a key/value pair. - if ':' in line: - var, value = line.split(':', 1) - value = value.strip() - elif '/' in line: - # For the comment containing a variable, clean up the list - # contents before assigning the parts we want to test. - parts = line.split() - parts.remove('#') - parts.remove('/') - var, value = parts - else: - var = value = line - - - # Once run through the 'main' function, the keys and values should - # match - if not value == var: - import pdb; pdb.set_trace() # NOQA - print("Var and value don't match.") - print("Var: {}, Value: {}".format(var, value)) - sys.exit() - - invalid_variable = var not in VAR_MAPPINGS.values() - # Comments aren't in our test mapping, so make sure we ignore them - is_comment = line.startswith('#') - - if invalid_variable and not is_comment: - err = "Variable {} doesn't appear to be a valid new name." - sys.exit(err.format(var)) - - - print("Tests passed") - - -if __name__ == '__main__': - set_up() - test() - teardown()