DC Subcloud Deployment Phase and Abort Operations

Story: 2010756
Task: 49409

Change-Id: I303045db99258a593b88b9c6a919a317c6d309a2
Signed-off-by: Ngairangbam Mili <ngairangbam.mili@windriver.com>
This commit is contained in:
Ngairangbam Mili 2024-01-08 09:48:47 +00:00
parent 15db7b6a2d
commit 07ac17667f
12 changed files with 910 additions and 72 deletions

View File

@ -3,7 +3,7 @@ dcmanager
=========
.. include:: /shared/_includes/cli-ref-intro-dcmanager.rest
.. contents::
:local:
:depth: 2
@ -73,10 +73,10 @@ list, show, and update operations on a subcloud.
``subcloud delete``
Delete subcloud details from the database.
``subcloud-deploy show``
``subcloud deploy show``
Show the uploaded deployment files.
``subcloud-deploy upload``
``subcloud deploy upload``
Upload the subcloud deployment files.
``subcloud-group add``
@ -104,12 +104,12 @@ list, show, and update operations on a subcloud.
Manage a subcloud. Enables the active synchronization of data between the
central cloud and the subcloud.
``subcloud reconfig``
``subcloud deploy config``
Re-run the deployment playbook on a subcloud using an updated configuration
file.
``subcloud reinstall``
Reinstall a subcloud.
``subcloud redeploy``
Redeploy a subcloud.
``subcloud restore``
Restore a subcloud.

View File

@ -35,6 +35,7 @@ Installation
installing-and-provisioning-a-subcloud
installing-a-subcloud-using-redfish-platform-management-service
installing-a-subcloud-without-redfish-platform-management-service
subcloud-deployment-phases-0ce5f6fbf696
reinstalling-a-subcloud-with-redfish-platform-management-service
subcloud-deployment-with-local-installation-4982449058d5

View File

@ -27,15 +27,6 @@ subcloud, the subcloud installation has these phases:
- Uses Ansible to bootstrap |prod-long| on controller-0 in
the subcloud
.. note::
After a successful remote installation of a subcloud in a Distributed Cloud
system, a subsequent remote reinstallation fails because of an existing ssh
key entry in the ``/root/.ssh/known_hosts`` on the System Controller. In this
case, delete the host key entry, if present, from ``/root/.ssh/known_hosts``
on the System Controller before doing reinstallations.
.. note::
Remove all removable USB storage devices from subcloud servers before
@ -294,14 +285,14 @@ subcloud ansible log files: ``/var/log/dcmanager/ansible``, and named as
By default, 30GB is allocated for ``/opt/platform-backup``. If additional
persistent disk space is required, the partition can be increased in the next
subcloud reinstall using the following commands:
subcloud redeploy using the following commands:
- To increase ``/opt/platform-backup`` to 40GB, add the
**persistent_size: 40000** parameter to the
``subcloud-install-values.yaml`` file.
- Use the :command:`dcmanager subcloud update` command to save the
configuration change for the next subcloud reinstall.
configuration change for the next subcloud redeploy.
.. code-block:: none
@ -483,7 +474,7 @@ subcloud ansible log files: ``/var/log/dcmanager/ansible``, and named as
Subclouds Using the CLI <managing-subclouds-using-the-cli>`.
If there is a deployment failure, do not delete the subcloud, use the
:command:`subcloud reconfig` command, to reconfigure the subcloud. For
:command:`dcmanager subcloud deploy config` command, to reconfigure the subcloud. For
more information, see :ref:`Managing Subclouds Using the CLI
<managing-subclouds-using-the-cli>`.

View File

@ -1,5 +1,8 @@
.. Greg updates required for -High Security Vulnerability Document Updates
.. pja1558616715987
@ -34,15 +37,6 @@ subcloud, the subcloud installation process has two phases:
Cloud that uses Ansible to bootstrap |prod-long| on controller-0 in
the subcloud
.. note::
After a successful remote installation of a subcloud in a Distributed Cloud
system, a subsequent remote reinstallation fails because of an existing ssh
key entry in the ``/root/.ssh/known_hosts`` on the system controller. In this
case, delete the host key entry, if present, from ``/root/.ssh/known_hosts``
on the system controller before doing reinstallations.
.. rubric:: |prereq|
.. _installing-a-subcloud-without-redfish-platform-management-service-ul-g5j-3f3-qjb:

View File

@ -123,14 +123,14 @@ fails, delete subclouds, and monitor or change the managed status of subclouds.
~(keystone_admin)]$ dcmanager subcloud manage <subcloud-name>
- To reconfigure a subcloud, if deployment fails, use the :command:`subcloud reconfig` command.
- To reconfigure a subcloud, if deployment fails, use the :command:`subcloud deploy config` command.
.. note::
You can enter the ``sysadmin`` password to avoid being prompted for the password.
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud reconfig <subcloud-id/name> --deploy-config \
~(keystone_admin)]$ dcmanager subcloud deploy config <subcloud-id/name> --deploy-config \
<filepath> --sysadmin-password <password>

View File

@ -128,11 +128,11 @@ using the ansible playbook.
From the System Controller, reconfigure the subcloud, using dcmanager.
Specify the sysadmin password and the deployment configuration file, using
the :command:`dcmanager subcloud reconfig` command.
the :command:`dcmanager subcloud deploy config` command.
.. code-block:: none
~(keystone_admin)$ dcmanager subcloud reconfig --sysadmin-password <sysadmin_password> --deploy-config deployment-config-subcloud1-duplex.yaml <subcloud1>
~(keystone_admin)$ dcmanager subcloud deploy config --sysadmin-password <sysadmin_password> --deploy-config deployment-config-subcloud1-duplex.yaml <subcloud1>
where *<sysadmin_password>* is assumed to be the login password and
*<subcloud1>* is the name of the subcloud

View File

@ -149,7 +149,7 @@ image list should also contain:
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud-deploy upload --prestage-images nn.nn_images.lst
~(keystone_admin)]$ dcmanager subcloud deploy upload --prestage-images nn.nn_images.lst
+------------------+-----------------+
| Field | Value |
@ -168,7 +168,7 @@ image list should also contain:
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud-deploy show
~(keystone_admin)]$ dcmanager subcloud deploy show
+------------------+-------------------------+
| Field | Value |
@ -258,7 +258,7 @@ Verifying Usage of Prestaged Data
---------------------------------
To verify that the prestaged data is used over subcloud upgrade, subcloud
reinstall, or subcloud remote restore:
redeploy, or subcloud remote restore:
- Search for the the subcloud name in the log file, for example,
subcloud1 from ``/www/var/log/lighttpd-access.log``. There should not be

View File

@ -111,7 +111,7 @@ There are six phases for Rehoming a subcloud:
- If the subcloud can be remotely installed via Redfish Virtual Media service,
transfer the yaml file that contains the install data for this subcloud,
and use this install data in the new system controller, via the
``--install-values`` option, when running the remote subcloud reinstall,
``--install-values`` option, when running the remote subcloud redeploy,
upgrade or restore commands.

View File

@ -5,16 +5,16 @@
.. _reinstalling-a-subcloud-with-redfish-platform-management-service:
=============================================================
Reinstall a Subcloud with Redfish Platform Management Service
Redeploy a Subcloud with Redfish Platform Management Service
=============================================================
For subclouds with servers that support Redfish Virtual Media Service
\(version 1.2 or higher), you can use the Central cloud's CLI to reinstall
\(version 1.2 or higher), you can use the Central cloud's CLI to redeploy
the ISO and bootstrap subclouds from the Central cloud.
.. caution::
All application and data on the subcloud will be lost after reinstallation.
All application and data on the subcloud will be lost after redeployment.
Any records of |FPGA| device image updates on the subcloud will be lost.
You will need to reapply the |FPGA| device image update orchestration
@ -23,9 +23,9 @@ the ISO and bootstrap subclouds from the Central cloud.
.. rubric:: |context|
The subcloud reinstallation has these phases:
The subcloud redeployment has these phases:
Executing the dcmanager subcloud reinstall command in the Central Cloud:
Executing the dcmanager subcloud redeploy command in the Central Cloud:
- Uses Redfish Virtual Media Service to remote install the ISO on controller-0
in the subcloud.
@ -40,9 +40,9 @@ Executing the dcmanager subcloud reinstall command in the Central Cloud:
.. rubric:: |prereq|
- The install values are required for subcloud reinstallation. By default,
- The install values are required for subcloud redeployment. By default,
install values are stored in database after a subcloud installation or
upgrade, and the reinstallation will re-use these values. You can use the
upgrade, and the redeployment will re-use these values. You can use the
following CLI command in the Central cloud to update them if necessary:
.. code-block:: none
@ -53,9 +53,9 @@ Executing the dcmanager subcloud reinstall command in the Central Cloud:
Subcloud Using Redfish Platform Management Service
<installing-a-subcloud-using-redfish-platform-management-service>`.
You can only reinstall the same software version with the Central cloud on
You can only redeploy the same software version with the Central cloud on
the subcloud. If the software version of the subcloud is not same as the
System Controller, the reinstall command will update the software version of
System Controller, the redeploy command will update the software version of
the subcloud and install the correct version afterwards.
@ -73,8 +73,8 @@ Executing the dcmanager subcloud reinstall command in the Central Cloud:
| 1 | subcloud1| unmanaged | offline | complete | unknown |
+----+----------+------------+--------------+---------------+---------+
As the reinstall will cause data and application loss, it is not necessary
and not recommended to reinstall a healthy subcloud. Reinstallation request
As the redeploy will cause data and application loss, it is not necessary
and not recommended to redeploy a healthy subcloud. Redeployment request
of a managed or online subcloud will therefore be rejected.
.. rubric:: |proc|
@ -94,13 +94,13 @@ Executing the dcmanager subcloud reinstall command in the Central Cloud:
:start-after: begin-ref-2
:end-before: end-ref-2
#. Execute the reinstall CLI.
#. Execute the redeploy CLI.
For example:
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud reinstall subcloud1 --bootstrap-values /home/sysadmin/subcloud1-bootstrap-values.yml sysadmin-password <sysadmin_password>
~(keystone_admin)]$ dcmanager subcloud redeploy subcloud1 --bootstrap-values /home/sysadmin/subcloud1-bootstrap-values.yml sysadmin-password <sysadmin_password>
.. only:: partner
@ -108,13 +108,13 @@ Executing the dcmanager subcloud reinstall command in the Central Cloud:
:start-after: begin-ref-3
:end-before: end-ref-3
#. Confirm the reinstall of the subcloud.
#. Confirm the redeploy of the subcloud.
You are prompted to enter ``reinstall`` to confirm the reinstallation.
You are prompted to enter ``redeploy`` to confirm the redeployment.
.. warning::
This will reinstall the subcloud. All applications and data on the
This will redeploy the subcloud. All applications and data on the
subcloud will be lost.
Any records of |FPGA| device image updates on the subcloud will be lost.
@ -122,9 +122,9 @@ Executing the dcmanager subcloud reinstall command in the Central Cloud:
procedure. For more information, see :ref:`Device Image Update Orchestration
<device-image-update-orchestration>`.
Please type ``reinstall`` to confirm: reinstall
Please type ``redeploy`` to confirm: redeploy
Any other input will abort the reinstallation.
Any other input will abort the redeployment.
#. In the Central cloud, monitor the progress of the subcloud installation
and bootstrapping by viewing the deploy status field of the dcmanager
@ -151,8 +151,8 @@ Executing the dcmanager subcloud reinstall command in the Central Cloud:
- ``/var/log/dcmanager/ansible/subcloud1_playbook_output.log``
#. After the subcloud is successfully reinstalled and bootstrapped, run the
subcloud reconfig command to complete the process. The subcloud
#. After the subcloud is successfully redeployed and bootstrapped, run the
:command:`dcmanager subcloud deploy config` command to complete the process. The subcloud
availability status will change from offline to online when the
reconfiguration is complete. For more information, see :ref:`Manage
Subclouds Using the CLI <managing-subclouds-using-the-cli>`.

View File

@ -0,0 +1,754 @@
.. _subcloud-deployment-phases-0ce5f6fbf696:
==========================
Subcloud Deployment Phases
==========================
Subclouds can be deployed using individual phases.
When a subcloud is deployed using a single operation :command:`dcmanager subcloud add`
that comprises different deployment phases and if there is a failure on any of the
phases, you would need to wait for the entire operation to timeout.
Thus, instead of using a single operation, a subcloud can be deployed by
executing each phase individually. This gives a user more control of the
deployment process, allowing the execution of individual phases one at a time,
with the possibility of aborting and resuming the deployment.
.. rubric:: |context|
After physically installing the hardware and network connectivity of a
subcloud, the subcloud deployment process excutes the following phases in the
central cloud:
- The :command:`dcmanager subcloud deploy create` command.
Sets up the subcloud configuration in the central cloud.
- The :command:`dcmanager subcloud deploy install` command.
Uses Redfish Virtual Media Service to remotely install the ISO on
controller-0 in the subcloud.
- The :command:`dcmanager subcloud deploy bootstrap` command.
Uses Ansible to bootstrap |prod-long| on controller-0 in the subcloud.
- The :command:`dcmanager subcloud deploy config` command.
Uses Ansible to run deploy config on controller-0 in the subcloud.
- The :command:`dcmanager subcloud deploy complete` command.
If the subcloud is manually configured post bootstrap, it concludes the subcloud deployment.
.. note::
Remove all the removable USB storage devices from the subcloud servers before
installing a Redfish remote subcloud.
.. rubric:: |prereq|
- A new system CLI option ``--active`` is added to the :command:`load-import`
command to allow the import into the system controller
``/opt/dc-vault/loads``. The purpose of this is to allow Redfish install of
subclouds referencing a single full copy of the ``bootimage.iso`` at
``/opt/dc-vault/loads``. (Previously, the full ``bootimage.iso`` was
duplicated for each :command:`subcloud add` command).
.. note::
This is required only once and does not have to be done for every subcloud install.
The :command:`dcmanager` command recognizes bootimage names that end with
``.iso`` and ``.sig``. For example:
.. code-block:: none
~(keystone_admin)]$ system --os-region-name SystemController load-import --active |installer-image-name|.iso |installer-image-name|.sig
The ISO imported via ``load-import --active`` must be at the same patch
level as the system controller. This ensures that the subcloud boot image
aligns with the patch level of the load to be installed on the subcloud.
.. warning::
If the patch level of load-imported ISO does not match the system
controller patch level, the subcloud patch state may not align with the
system controller patch state.
- Run the :command:`load-import` command on controller-0 to import the new
release. You can specify either the full file path or relative paths to the
``*.iso`` bootimage file and to the ``*.sig`` bootimage signature file.
.. code-block:: none
$ source /etc/platform/openrc
~(keystone_admin)]$ system load-import [--local] /home/sysadmin/<bootimage>.iso <bootimage>.sig
+--------------------+-----------+
| Property | Value |
+--------------------+-----------+
| id | 2 |
| state | importing |
| software_version | nn.nn |
| compatible_version | nn.nn |
| required_patches | |
+--------------------+-----------+
The :command:`load-import` must be done on controller-0.
(Optional) If ``--local`` is specified, the ISO and sig files are uploaded
directly from the active controller, where `<local_iso_file_path>` and
`<local_sig_file_path>` are paths on the active controller to load ISO files
and sig files, respectively.
.. note::
- If ``--local`` is specified, the ISO and sig files are transferred
directly from the active controller filesystem to the load directory,
otherwise the files are transferred via the API.
- This may take a few minutes to complete.
In order to deploy subclouds from either controller, all local files that are
referenced in the ``subcloud-bootstrap-values.yaml`` file must exist on both
controllers (for example, ``/home/sysadmin/docker-registry-ca-cert.pem``).
.. rubric:: |proc|
#. At the subcloud location, physically install the servers and network
connectivity required for the subcloud.
.. note::
Do not power off the servers. The host portion of the server can be
powered off, but the |BMC| portion of the server must be powered and
accessible from the system controller.
There is no need to wipe the disks.
.. note::
The servers require connectivity to a gateway router that provides IP
routing between the subcloud management or admin subnet and the system controller
management subnet, and between the subcloud |OAM| subnet and the
system controller subnet.
.. include:: /_includes/installing-a-subcloud-using-redfish-platform-management-service.rest
:start-after: begin-ref-1
:end-before: end-ref-1
#. Create the ``subcloud-install-values.yaml`` file and use the content to pass the file
into the :command:`dcmanager subcloud deploy create` command, using the
``--install-values`` command option.
.. note::
If the subcloud will be manually installed, skip this step and go to deploy bootstrap.
.. note::
If your controller is on a ZTSystems Triton server that requires a
longer timeout value, you can now use the ``rd.net.timeout.ipv6dad``
dracut parameter to specify an increased timeout value for dracut to
wait for the interface to have carrier, and complete IPv6 duplicate
address detection |DAD|. For the ZTSystems server, this can take more
than four minutes. It is recommended that you set this value to 300 seconds,
by specifying the following in the ``subcloud-install-values.yaml``
file:
.. code-block:: none
rd.net.timeout.ipv6dad: 300
.. note::
The ``wait_for_timeout`` value must be chosen based on your network
performance (bandwidth, latency, and quality) and should be increased
if the network does not meet the minimum or timeout requirements.
The default value of 3600 seconds is based on a network bandwidth
of 100 Mbps with a 50 ms delay.
.. only:: partner
.. include:: /_includes/installing-a-subcloud-using-redfish-platform-management-service.rest
:start-after: begin-syslimit
:end-before: end-syslimit
For example, :command:`--install-values /home/sysadmin/subcloud-install-values.yaml`.
.. parsed-literal::
bootstrap_interface: <bootstrap_interface_name> # e.g. eno1
bootstrap_address: <bootstrap_interface_ip_address> # e.g.128.224.151.183
bootstrap_address_prefix: <bootstrap_netmask> # e.g. 23
# Board Management Controller
bmc_address: <BMCs_IPv4_or_IPv6_address> # e.g. 128.224.64.180
bmc_username: <bmc_username> # e.g. root
# If the subcloud's bootstrap IP interface and the system controller are not on the
# same network, then the customer must configure a default route or static route
# so that the Central Cloud can login bootstrap the newly installed subcloud.
# If nexthop_gateway is specified and the network_address is not specified then a
# default route will be configured. Otherwise, if a network_address is specified then
# a static route will be configured.
nexthop_gateway: <default_route_address> for # e.g. 128.224.150.1 (required)
network_address: <static_route_address> # e.g. 128.224.144.0
network_mask: <static_route_mask> # e.g. 255.255.254.0
# Installation type codes
#0 - Standard Controller, Serial Console
#1 - Standard Controller, Graphical Console
#2 - AIO, Serial Console
#3 - AIO, Graphical Console
#4 - AIO Low-latency, Serial Console
#5 - AIO Low-latency, Graphical Console
install_type: 3
# Optional parameters defaults can be modified by uncommenting the option with a modified value.
# This option can be set to extend the installing stage timeout value
# wait_for_timeout: 3600
# Set this options for https
no_check_certificate: True
# If the bootstrap interface is a vlan interface then configure the vlan ID.
# bootstrap_vlan: <vlan_id>
# Override default filesystem device.
# rootfs_device: "/dev/disk/by-path/pci-0000:00:1f.2-ata-1.0"
# boot_device: "/dev/disk/by-path/pci-0000:00:1f.2-ata-1.0"
# Set the value for persistent file system (/opt/platform-backup).
# The value must be whole number (in MB) that is greater than or equal
# to 30000.
persistent_size: 30000
# Configure custom arguments applied at boot within the installed subcloud.
# Multiple boot arguments can be provided by separating each argument by a
# single comma. Spaces are not allowed.
# Example: extra_boot_params: multi-drivers-switch=cvl-2.54
# extra_boot_params:
.. _increase-subcloud-platform-backup-size:
.. note::
By default, 30GB is allocated for ``/opt/platform-backup``. If additional
persistent disk space is required, the partition can be increased in the next
subcloud redeploy using the following commands:
- To increase ``/opt/platform-backup`` to 40GB, add the **persistent_size: 40000**
parameter to the ``subcloud-install-values.yaml`` file.
- Use the :command:`dcmanager subcloud update` command to save the
configuration change for the next subcloud redeployment.
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud update --install-values <subcloud-install-values.yaml> <subcloud-name>
For a new subcloud deployment, use the :command:`dcmanager subcloud deploy create`
command with the ``subcloud-install-values.yaml`` file containing the desired
``persistent_size`` value.
#. At the system controller, create a
``/home/sysadmin/subcloud-bootstrap-values.yaml`` overrides file for the
subcloud.
For example:
.. code-block:: none
system_mode: simplex
name: "subcloud"
description: "test"
location: "loc"
management_subnet: 192.168.101.0/24
management_start_address: 192.168.101.2
management_end_address: 192.168.101.50
management_gateway_address: 192.168.101.1
external_oam_subnet: 10.10.10.0/24
external_oam_gateway_address: 10.10.10.1
external_oam_floating_address: 10.10.10.12
systemcontroller_gateway_address: 192.168.204.101
docker_registries:
k8s.gcr.io:
url: registry.central:9001/k8s.gcr.io
gcr.io:
url: registry.central:9001/gcr.io
ghcr.io:
url: registry.central:9001/ghcr.io
quay.io:
url: registry.central:9001/quay.io
docker.io:
url: registry.central:9001/docker.io
docker.elastic.co:
url: registry.central:9001/docker.elastic.co
registry.k8s.io:
url: registry.central:9001/registry.k8s.io
icr.io:
url: registry.central:9001/icr.io
defaults:
username: sysinv
password: <sysinv_password>
type: docker
Where <sysinv_password> can be found by running the following command as
'sysadmin' on the central cloud:
.. code-block:: none
$ keyring get sysinv services
In the above example, if the admin network is used for communication
between the subcloud and system controller, then the
``management_gateway_address`` parameter should be replaced with admin
subnet information.
For example:
.. code-block:: none
management_subnet: 192.168.101.0/24
management_start_address: 192.168.101.2
management_end_address: 192.168.101.50
admin_subnet: 192.168.102.0/24
admin_start_address: 192.168.102.2
admin_end_address: 192.168.102.50
admin_gateway_address: 192.168.102.1
This configuration will install container images from the local registry on
your central cloud. The central cloud's local registry's HTTPS Certificate
must have the central cloud's |OAM| IP, **registry.local** and
**registry.central** in the certificate's |SAN| list. For example, a valid
certificate contains a |SAN| list:
.. code-block:: none
"DNS.1: registry.local DNS.2: registry.central IP.1: floating_management IP.2: floating_OAM"
If required, run the following command on the central cloud prior to
bootstrapping the subcloud to install the new certificate for the central
cloud with the updated |SAN| list:
.. code-block:: none
~(keystone_admin)]$ system certificate-install -m docker_registry path_to_cert
If you prefer to install container images from the default external
registries, make the following substitutions for the **docker_registries**
sections of the file.
.. code-block:: none
docker_registries:
defaults:
username: <your_default_registry_username>
password: <your_default_registry_password>
.. include:: /_includes/installing-a-subcloud-using-redfish-platform-management-service.rest
:start-after: begin-subcloud-1
:end-before: end-subcloud-1
#. Create the subcloud using :command:`dcmanager`.
When calling the :command:`subcloud deploy create` command, specify the install
values, bootstrap values, deploy config values, and the subcloud's sysadmin password.
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud deploy create \
--bootstrap-address <oam_ip_address_of_subclouds_controller-0> \
--bootstrap-values /home/sysadmin/subcloud1-bootstrap-values.yaml \
--deploy-config /home/sysadmin/subcloud1-deploy-config.yaml \
--install-values /home/sysadmin/install-values.yaml \
--bmc-password <bmc_password>
--release <software-release>
If ``--sysadmin-password`` is not specified, you are prompted to
enter it once the full command is invoked. The password is masked
when it is entered.
.. code-block:: none
Enter the sysadmin password for the subcloud:
(Optional) The ``--deploy-config`` option must reference the deployment configuration
file mentioned above. In the deployment configurations, static routes from
the management or admin interface of a subcloud to the system controller's
management subnet must be explicitly listed. This ensures that the subcloud
comes online after deployment. If the admin network is used for
communication between the system controller and subcloud, the deployment
configuration file must include both an admin network type and a management
network type interface.
(Optional) The ``--bmc-password <password>`` option is used for subcloud
installation and is required only if the ``--install-values`` option is
specified.
If the ``--bmc-password <password>`` option is omitted and the
``--install-values`` option is specified, the system administrator will be
prompted to enter it, following the :command:`dcmanager subcloud deploy create`
command. This option is ignored if the ``--install-values`` option is not
specified. The password is masked when it is entered.
.. code-block:: none
Enter the bmc password for the subcloud:
The :command:`dcmanager subcloud show` or :command:`dcmanager subcloud list`
command can be used to view subcloud deploy create status.
The **deploy status** field has the following values for this phase:
.. include:: /shared/_includes/subcloud-deploy-status.rest
:start-after: begin-deploy-status-create
:end-before: end-deploy-status-create
#. Install the subcloud using :command:`dcmanager`.
To install the subcloud using Redfish Virtual Media Service, use the
:command:`subcloud deploy install` command. Both ``--install-values`` and
``--release`` parameters are optional if they were provided previouslly,
and will replace them if present on request.
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud deploy install <subcloud-name> \
--install-values /home/sysadmin/install-values.yaml \
--sysadmin-password <sysadmin_password> \
--bmc-password <bmc_password> \
--release <software-release>
If ``--sysadmin-password`` is not specified, you are prompted to enter
it once the full command is invoked. The password is masked when it is
entered.
.. code-block:: none
Enter the sysadmin password for the subcloud:
(Optional) The ``--bmc-password <password>`` option is used for subcloud
installation and is required only if the ``--install- values`` option is
specified.
If the ``--bmc-password <password>`` option is omitted and the
``--install-values`` option is specified, the system administrator will be
prompted to enter it, following the :command:`dcmanager subcloud add`
command. This option is ignored if the ``--install-values`` option is not
specified. The password is masked when it is entered.
.. code-block:: none
Enter the bmc password for the subcloud:
The :command:`dcmanager subcloud show` or :command:`dcmanager subcloud list` command
can be used to view subcloud deploy install progress.
The **deploy status** field has the following values for this phase:
.. include:: /shared/_includes/subcloud-deploy-status.rest
:start-after: begin-deploy-status-install
:end-before: end-deploy-status-install
#. Bootstrap the subcloud using :command:`dcmanager`.
To bootstrap the subcloud, use the :command:`subcloud deploy bootstrap`
command. Both ``--bootstrap-address`` and ``--bootstrap-values`` parameters
are optional, and will replace the previous values if provided.
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud deploy bootstrap <subcloud-name> \
--bootstrap-address <oam_ip_address_of_subclouds_controller-0> \
--bootstrap-values /home/sysadmin/subcloud1-bootstrap-values.yaml \
--sysadmin-password <sysadmin_password>
If ``--sysadmin-password`` is not specified, you are prompted to enter
it once the full command is invoked. The password is masked when it is
entered.
.. code-block:: none
Enter the sysadmin password for the subcloud:
The :command:`dcmanager subcloud show` or :command:`dcmanager subcloud list` command can
be used to view subcloud deploy bootstrap progress.
The **deploy status** field has the following values for this phase:
.. include:: /shared/_includes/subcloud-deploy-status.rest
:start-after: begin-deploy-status-bootstrap
:end-before: end-deploy-status-bootstrap
#. Configure the subcloud using :command:`dcmanager`.
To configure the subcloud, use the :command:`subcloud deploy config`
command. The ``--deploy-config`` parameter is optional if it was
provided previouslly, and will replace it if present on request.
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud deploy config <subcloud-name> \
--deploy-config /home/sysadmin/subcloud1-deploy-config.yaml \
--sysadmin-password <sysadmin_password>
If ``--sysadmin-password`` is not specified, you are prompted to enter
it once the full command is invoked. The password is masked when it is
entered.
.. code-block:: none
Enter the sysadmin password for the subcloud:
(Optional) The ``--deploy-config`` option must reference the deployment
configuration file mentioned above. In the deployment configurations,
static routes from the management or admin interface of a subcloud to the
system controller's management subnet must be explicitly listed. This
ensures that the subcloud comes online after deployment. If the admin
network is used for communication between the system controller and
subcloud, the deployment configuration file must include both an admin
network type and a management network type interface.
The :command:`dcmanager subcloud show` or :command:`dcmanager subcloud list`
command can be used to view the subcloud deploy configuration status.
The **deploy status** field has the following values for this phase:
.. include:: /shared/_includes/subcloud-deploy-status.rest
:start-after: begin-deploy-status-config
:end-before: end-deploy-status-config
#. Complete the subcloud deployment using :command:`dcmanager`.
When manually configuring the subcloud, the deployment must be concluded by
running the :command:`subcloud deploy complete` command.
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud deploy complete <subcloud-name>
The **deploy status** field will transition to ``Complete``.
#. At the Central Cloud / System Controller, monitor the progress of the
subcloud install, bootstrapping, and deployment by using the **deploy status**
field of the :command:`dcmanager subcloud list` command.
.. caution::
If there is a failure during installation or bootstraping, you can resume
the deployment using the `dcmanager subcloud deploy resume` command, or
you can run individual phases with `dcmanager subcloud deploy
install`, `dcmanager subcloud deploy bootstrap` or `dcmanager subcloud
deploy config`.
#. If ``deploy_status`` shows an installation, bootstrap, or deployment failure
state, you can use the :command:`dcmanager subcloud errors` command in order to get
more detailed information about the failure.
For example:
.. code-block:: none
[sysadmin@controller-0 ~(keystone_admin)]$ dcmanager subcloud errors 1
FAILED bootstrapping playbook of (subcloud).
detail: fatal: [subcloud]: FAILED! => changed=true
failed_when_result: true
msg: non-zero return code
500 Server Error: Internal Server Error ("manifest unknown: manifest unknown")
Image download failed: admin-2.cumulus.mss.com: 30093/wind-river/cloud-platform-deployment-manager: WRCP_22.06 500 Server Error: Internal Server Error ("Get https://admin-2.cumulus .mss.com: 30093/v2/: dial tcp: lookup admin-2.cumulus.mss.com on 10.41.0.1:53: read udp 10.41.1.3:40251->10.41.0.1:53: i/o timeout")
Image download failed: gcd.io/kubebuilder/kube-rdac-proxy:v0.11.0 500 Server Error: Internal Server Error ("Get https://gcd.io/v2/: dial tcp: lookup gcd.io on 10.41.0.1:53: read udp 10.41.1.3:52485->10.41.0.1:53: i/o timeout")
raise Exception("Failed to download images %s" % failed_downloads)
Exception: Failed to download images ["admin-2.cumulus.mss.com: 30093/wind-river/cloud-platform-deployment-manager: WRCP_22.06", "gcd.io kubebuilder/kube-rdac-proxy:v0.11.0"]
FAILED TASK: TASK [common/push-docker-images Download images and push to local registry] Wednesday 12 October 2022 12:27:31 +0000 (0:00:00.042)
0:16:34.495
#. You can also monitor detailed logging of the subcloud installation,
bootstrapping, and deployment by monitoring the log file
``/var/log/dcmanager/ansible/<subcloud_name>_playbook_output.log`` on the
active controller in the central cloud.
For example:
.. code-block:: none
controller-0:/home/sysadmin# tail /var/log/dcmanager/ansible/subcloud_playbook_output.log
k8s.gcr.io: {password: secret, url: null}
quay.io: {password: secret, url: null}
)
TASK [bootstrap/bringup-essential-services : Mark the bootstrap as completed] ***
changed: [subcloud]
PLAY RECAP *********************************************************************
subcloud : ok=230 changed=137 unreachable=0 failed=0
.. note::
The subcloud_playbook_output.log can rotate, the previous log file will
be subcloud_playbook_output.log.1.
If the install, bootstrap, or config phase fails, it can be re-executed
using the same command that is used to trigger the deploy phase.
If more debugging is required, set ``rvmc_debug_level`` in the
``install-values.yaml`` file. For more information, see :ref:`installing-a-subcloud-using-redfish-platform-management-service`.
----------------------------------------
Abort and Resume the Subcloud Deployment
----------------------------------------
The subcloud deployment can be aborted and resumed for the install, bootstrap
and config phases.
To abort the deployment, use the :command:`subcloud deploy abort` command.
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud deploy abort <subcloud-name>
View the subcloud deploy abort status by running the :command:`dcmanager subcloud show` or :command:`dcmanager subcloud list`
command.
The **deploy status** field has the following values for this phase:
.. include:: /shared/_includes/subcloud-deploy-status.rest
:start-after: begin-deploy-status-abort
:end-before: end-deploy-status-abort
.. note::
If the :command:`dcmanager subcloud deploy abort <subcloud-name>` command
is called during the installation phase, the subcloud will be shut down via
Redfish Virtual Media Service.
To resume the deployment, use the :command:`subcloud deploy resume` command.
The parameter values will be reused from the previous phases if new ones are not
provided in the request.
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud deploy resume <subcloud-name> \
--bootstrap-address <oam_ip_address_of_subclouds_controller-0> \
--bootstrap-values /home/sysadmin/subcloud1-bootstrap-values.yaml \
--sysadmin-password <sysadmin_password> \
--deploy-config /home/sysadmin/subcloud1-deploy-config.yaml \
--install-values /home/sysadmin/install-values.yaml \
--bmc-password <bmc_password> \
--release <software-release>
.. rubric:: |postreq|
- For detailed |prod| procedures on manually configuring the subcloud for the desired
deployment configuration, see the post-bootstrap steps of |inst-doc|.
- Check and update docker registry credentials on the subcloud:
.. code-block:: none
REGISTRY="docker-registry"
SECRET_UUID='system service-parameter-list | fgrep
$REGISTRY | fgrep auth-secret | awk '{print $10}''
SECRET_REF='openstack secret list | fgrep $
{SECRET_UUID} | awk '{print $2}''
openstack secret get ${SECRET_REF} --payload -f value
The secret payload should be ``username: sysinv password:<password>``. If
the secret payload is ``username: admin password:<password>``, see
:ref:`updating-docker-registry-credentials-on-a-subcloud` for more
information.
- For more information on bootstrapping and deploying, see the procedures
listed under :ref:`install-a-subcloud`.
- Add static route for nodes in subclouds to access openldap service.
In |DC| system, openldap service is running on central cloud. In order for the nodes
in the subclouds to access openldap service, such as :command:`ssh` to the nodes as openldap
users, a static route to the System Controller is required to be added in these
nodes. This applies to controller nodes, worker nodes, and storage nodes (nodes
that have sssd running).
The static route can be added on each of the nodes in the subcloud using the system
CLI.
The following examples show how to add the static route in controller node and
worker node:
.. code-block:: none
system host-route-add controller-0 mgmt0 <Central Cloud mgmt subnet> 64 <Gateway IP address>
system host-route-add compute-0 mgmt0 <Central Cloud mgmt subnet> 64 <Gateway IP address>
The static route can also be added using deploy config by adding the route
in its configuration file.
The following examples show how to add the route configuration in a controller and
worker host profiles of the deploy config's configuration file:
.. code-block:: none
Controller node:
---
apiVersion: starlingx.windriver.com/v1
kind: HostProfile
metadata:
labels:
controller-tools.k8s.io: "1.0"
name: controller-0-profile
namespace: deployment
spec:
administrativeState: unlocked
bootDevice: /dev/disk/by-path/pci-0000:c3:00.0-nvme-1
console: ttyS0,115200n8
installOutput: text
......
routes:
- gateway: <Gateway IP address>
activeinterface: mgmt0
metric: 1
prefix: 64
subnet: <Central Cloud mgmt subnet>
Worker node:
---
apiVersion: starlingx.windriver.com/v1
kind: HostProfile
metadata:
labels:
controller-tools.k8s.io: "1.0"
name: compute-0-profile
namespace: deployment
spec:
administrativeState: unlocked
boardManagement:
credentials:
password:
secret: bmc-secret
type: dynamic
bootDevice: /dev/disk/by-path/pci-0000:00:1f.2-ata-1.0
clockSynchronization: ntp
console: ttyS0,115200n8
installOutput: text
......
routes:
- gateway: <Gateway IP address>
interface: mgmt0
metric: 1
prefix: 64
subnet: <Central Cloud mgmt subnet>

View File

@ -36,26 +36,23 @@ For example:
The **deploy status** field has the following values:
.. container::
.. include:: /shared/_includes/subcloud-deploy-status.rest
:start-after: begin-deploy-status-create
:end-before: end-deploy-status-create
.. container:: hideable
``Pre-Install``
This status indicates that the ISO for the subcloud is being updated by
the Central Cloud with the boot menu parameters, and kickstart
configuration as specified in the ``install-values.yaml`` file.
``Installing``
This status indicates that the subcloud's ISO is being installed from
the Central Cloud to the subcloud using the Redfish Virtual Media
service on the subcloud's |BMC|.
.. include:: /shared/_includes/subcloud-deploy-status.rest
:start-after: begin-deploy-status-install
:end-before: end-deploy-status-install
.. container::
``Bootstrapping``
This status indicates that the Ansible bootstrap of |prod-long|
software on the subcloud's controller-0 is in progress.
``Complete``
This status indicates that subcloud deployment is complete.
.. include:: /shared/_includes/subcloud-deploy-status.rest
:start-after: begin-deploy-status-bootstrap
:end-before: end-deploy-status-bootstrap
.. include:: /_includes/installing-a-subcloud-using-redfish-platform-management-service.rest
:start-after: begin-deploying-state
@ -63,6 +60,4 @@ The **deploy status** field has the following values:
The subcloud bootstrapping and deployment can take up to 30 minutes.
.. end-monitor-progress

View File

@ -0,0 +1,103 @@
.. begin-deploy-status-create
``Creating``
This status indicates that the subcloud configuration is being created
based on provided files and parameters.
``Create-failed``
This status indicates that the subcloud creation stage failed. Check
``/var/log/dcmanager/dcmanager.log`` for more details on the failure.
``Create-complete``
This status indicates that the subcloud was successfully created.
.. end-deploy-status-create
.. begin-deploy-status-install
``Pre-Install``
This status indicates that the ISO for the subcloud is being updated by
the central cloud with the boot menu parameters and kickstart
configuration as specified in the ``install-values.yaml`` file.
``Pre-Install-failed``
This status indicates that the ISO preparation stage failed. Check
``/var/log/dcmanager/dcmanager.log`` for more details on the failure.
``Installing``
This status indicates that the subcloud's ISO is being installed from
the central cloud to the subcloud using the Redfish Virtual Media
service on the subcloud's |BMC|.
``Install-failed``
This status indicates that the subcloud's ISO installation failed. Check
``/var/log/dcmanager/ansible/<subcloud_name>_playbook_output.log`` or
:command:`dcmanager subcloud errors` for more details on the failure.
``Install-complete``
This status indicates that the subcloud's ISO installation completed
successfully.
.. end-deploy-status-install
.. begin-deploy-status-bootstrap
``Pre-bootstrap``
This status indicates that the necessary configurations for subcloud Ansible
bootstrap are being generated based on the provided ``bootstrap-values.yaml`` file.
``Pre-bootstrap-failed``
This status indicates that the Ansible bootstrap preparation stage failed.
Check ``/var/log/dcmanager/dcmanager.log`` for more details on the failure.
``Bootstrapping``
This status indicates that the Ansible bootstrap of |prod-long|
software on the subcloud's controller-0 is in progress.
``Bootstrap-failed``
This status indicates that the subcloud Ansible bootstrap stage failed. Check
``/var/log/dcmanager/ansible/<subcloud_name>_playbook_output.log`` or
:command:`dcmanager subcloud errors` for more details on the failure.
``Bootstrap-complete``
This status indicates that the subcloud Ansible bootstrap operation completed
successfully.
.. end-deploy-status-bootstrap
.. begin-deploy-status-config
``Pre-config``
This status indicates that the necessary configurations for deploy config
are being generated based on the provided ``deploy-values.yaml`` file.
``Pre-config-failed``
This status indicates that the deploy config preparation stage failed.
Check ``/var/log/dcmanager/dcmanager.log`` for more details on the failure.
``Configuring``
This status indicates that the deploy config playbook execution
on the subcloud's controller-0 is in progress.
``Config-failed``
This status indicates that the deploy config playbook failed. Check
``/var/log/dcmanager/ansible/<subcloud_name>_playbook_output.log`` or
:command:`dcmanager subcloud errors` for more details on the failure.
``Complete``
This status indicates that subcloud deployment is complete.
.. end-deploy-status-config
.. begin-deploy-status-abort
``Aborting-<phase>``
This status indicates that the current subcloud deployment phase
is currently being aborted.
``<phase>-aborted``
This status indicates that the specified subcloud deployment phase
is successfully aborted.
.. end-deploy-status-abort