From 36cefa0c527329e6617c0bcd8b93a8e58d5bd5b7 Mon Sep 17 00:00:00 2001 From: Elisamara Aoki Goncalves Date: Thu, 11 Jul 2024 13:37:23 +0000 Subject: [PATCH] Update STARLINGX Developer/Contributor documentation on building ISOs and Patches Remove patch requirements install step. Update command. Story: 2010676 Task: 50548 Change-Id: I0a36369a47f6fea6b855ac941700f5024fe7a8f8 Signed-off-by: Elisamara Aoki Goncalves --- .../starlingx_patching.rst | 259 ++++++++++++------ doc/source/shared/abbrevs.txt | 1 + 2 files changed, 172 insertions(+), 88 deletions(-) diff --git a/doc/source/developer_resources/starlingx_patching.rst b/doc/source/developer_resources/starlingx_patching.rst index 3f7d92bf4..e06c3fb3c 100644 --- a/doc/source/developer_resources/starlingx_patching.rst +++ b/doc/source/developer_resources/starlingx_patching.rst @@ -21,7 +21,7 @@ vulnerabilities, feature enhancements, etc. StarlingX patching supports two kinds of patches: -* *In-service* patches, which don’t require a reboot for those hosts to be +* *In-service* patches, which do not require a reboot for those hosts to be patched. Only affected processes need to be restarted. * *Reboot-required* patches, which require a host reboot operation to make the @@ -42,8 +42,13 @@ A StarlingX patch includes one or more rpms that contain some updates to the system. Before starting to generate a patch, you must identify rpms to be installed against the deployed system. +Starting at StarlingX 10.0, the patches will now be built and applied using +|USM|, which means the patch process will be simplified as there is no need to +deliver ostree deltas anymore and just debian packages (.deb's). Legacy +patching will no longer be supported. + ***************************** -Identify rpms to be installed +Identify debs to be Installed ***************************** First, figure out the software version of the deployed system, using either @@ -58,7 +63,7 @@ Use Horizon GUI Discover your OAM floating IP address with the :command:`system oam-show` command. -#. Log in to Horizon with an admin/. +#. Log in to Horizon with an ``admin/``. #. Identify the software version of the deployed system by clicking :menuselection:`Admin > Platform > System Configuration` and clicking the @@ -72,7 +77,7 @@ Use StarlingX CLI node to show the software version. In the example below, the software version is "19.09". - :: + .. code-block:: controller-0:~$ . /etc/platform/openrc [sysadmin@controller-0 ~(keystone_admin)]$ system show @@ -99,121 +104,199 @@ Use StarlingX CLI +----------------------+--------------------------------------+ -After you figure out the software version, check the latest StarlingX -build to identify updated rpms against that version. Select one or -more rpms for patch generation and continue to the next section. +After you figure out the software version, check the latest StarlingX build to +identify updated debs against that version. Select one or more debs for patch +generation and continue to the next section. ************* -Build patches +Build Patches ************* -After you determine the rpms to be installed, set up an environment for +After you determine the debs to be installed, set up an environment for patch build. As a StarlingX developer, the easiest way to set up this type of environment is to use a StarlingX build container, with a few changes. -############# -Prerequisites -############# +.. rubric:: |prereq| -* Set up a StarlingX build container following `the - build guide `_. +#. Set up a StarlingX build container following `the + build guide `_. -* Download StarlingX source code. +#. Download StarlingX source code. -* Be sure that the rpms to be installed are already available in the build - container filesystem. +#. Populate your build using ``build-pkgs --reuse`` or ``build-pkgs --all``. + Using the reuse flag should be faster if you already have the running env. -Set up the patch build environment using the steps below. Note that -this environment is intended for developers working on this feature -and is not intended to be used for a StarlingX-based product. +#. Make your changes, usually by cherry picking some commit. -#. Install two python packages ``crypto`` and ``pycrypto``, which are the - python dependencies of the ``cgcs-patch`` package. +#. Apply the changes to the necessary files and than build the packages + related to these files with: - :: + .. code-block:: none - $ sudo pip install crypto pycrypto + $ build-pkgs -p -#. Use the script ``$MY_REPO/stx/stx-update/extras/scripts/patch_build.sh`` - to build patches. This script sources the **PLATFORM_RELEASE** - variable from the **release-info.inc** file, and sets up the PYTHONPATH to - point to cgcs-patch package in the repo. This avoids the need to install - cgcs-patch or manually set the PLATFORM_RELEASE variable. +#. Build all necessary packages. To find out the package name, look for a + debian ``meta_data.yaml`` file. - View the details of the patch build command using the command: - ``$MY_REPO/stx/stx-update/extras/scripts/patch_build.sh -h`` +.. rubric:: |proc| - :: +Set up the patch build environment using the procedure below. - $ $MY_REPO/stx/stx-update/cgcs-patch/bin/patch_build --help - Usage: patch_build [ ] ... - Options: - --id Patch ID - --release Platform release version - --status Patch Status Code (ie. O, R, V) - --unremovable Marks patch as unremovable - --reboot-required Marks patch as reboot-required (default=Y) - --summary Patch Summary - --desc Patch Description - --warn Patch Warnings - --inst Patch Install Instructions - --req Required Patch - --controller New package for controller - --worker New package for worker node - --worker-lowlatency New package for worker-lowlatency node - --storage New package for storage node - --controller-worker New package for combined node - --controller-worker-lowlatency New package for lowlatency - combined node - --all-nodes New package for all node types +.. note:: - You must specify the patch id, whether a reboot is required or not, any - dependent patches, the file path of rpms, etc. For new rpm packages, you - must specify which hosts require them using options, for example - ``--controller``. + This environment is intended for developers working on this feature + and is not intended to be used for a StarlingX-based product. - When the command is finished, you get a patch file with the name - ``.patch``. +.. note:: -Let’s dive a little deeper into the generated patch. + Patch requirements are automatically installed when containers are created. -#. The patch file is a gzip compressed package. Check it - using the :command:`file` command. Below is an example, including command output. - :: +#. Create your patch recipe. The recipe file is used as input and it has all + the patch metadata. Packages that will be included in the patch need to be + added to the `packages` object ("stx" are packages we build, "binary" are + packages we download). - $ file 001.patch - 001.patch: gzip compressed data, was "001.patch", last modified: - Fri Aug 16 05:56:59 2019, max compression + .. code-block:: none -#. Unzip the patch file to discover four files, ``software.tar``, - ``metadata.tar``, ``signature``, and ``signature.v2``. + + + 1.0.0 + + starlingx + + sample patch test + + Sample description + + Sample instructions + + Sample warning + + Y + + N + + DEV + + + + + + + scripts/pre-install.sh + scripts/post-install.sh + + + + sysvinv + linux + linux-rt + + + + + curl + + - :: +**Patch build** - $ tar -xf 001.patch - $ tree - ├── 001.patch - ├── metadata.tar - ├── signature - ├── signature.v2 - └── software.tar +#. Create the patch. - * Unzip ``software.tar`` to find that it contains all rpms to be - installed. Note that all rpms have been signed during patch - build using the key - ``$MY_REPO/build-tools/signing/ima_signing_key.priv``. + .. code-block:: none - * ``metadata.tar`` contains only one file ``metadata.xml``, which - includes all the information provided by the patch build command. It is - used by the StarlingX patching runtime system. + $ patch-builder --recipe EXAMPLES/patch-recipe-sample.xml + ... + 2024-06-12 13:12:49,566 - patch_builder - DEBUG: Copying pre-install script: /localdisk/designer/USER/master-env/cgcs-root/build-tools/stx/patch/scripts/pre-install.sh + 2024-06-12 13:12:49,567 - patch_builder - INFO: Renaming /localdisk/designer/USER/master-env/cgcs-root/build-tools/stx/patch/scripts/pre-install.sh to pre-install.sh + 2024-06-12 13:12:49,567 - patch_builder - DEBUG: Copying post-install script: /localdisk/designer/USER/master-env/cgcs-root/build-tools/stx/patch/scripts/post-install.sh + 2024-06-12 13:12:49,567 - patch_builder - INFO: Renaming /localdisk/designer/USER/master-env/cgcs-root/build-tools/stx/patch/scripts/post-install.sh to post-install.sh + 2024-06-12 13:12:49,567 - patch_builder - DEBUG: Generating metadata file + 2024-06-12 13:12:49,569 - patch_builder - DEBUG: Generating signature for patch files ['metadata.tar', 'software.tar', 'pre-install.sh', 'post-install.sh'] + 2024-06-12 13:12:49,888 - patch_builder - INFO: Formal signing status: False + 2024-06-12 13:12:49,888 - patch_builder - INFO: Saving file software.tar + 2024-06-12 13:12:51,441 - patch_builder - INFO: Saving file post-install.sh + 2024-06-12 13:12:51,441 - patch_builder - INFO: Saving file metadata.tar + 2024-06-12 13:12:51,442 - patch_builder - INFO: Saving file signature + 2024-06-12 13:12:51,442 - patch_builder - INFO: Saving file signature.v2 + 2024-06-12 13:12:51,442 - patch_builder - INFO: Saving file pre-install.sh + 2024-06-12 13:12:51,442 - patch_builder - INFO: Patch file created /localdisk/loadbuild/USER/master-env/patch_output/starlingx-1.0.0.patch - * ``signature`` is a combination of the md5 values of ``software.tar`` and - ``metadata.tar``. +#. Verify patch content. + + .. code-block:: none + + $ tar -tvf /localdisk/loadbuild/USER/master-env/patch_output/starlingx-1.0.0.patch + USER/cgts 74772480 2024-06-12 13:12 software.tar + USER/cgts 440 2024-06-12 13:12 post-install.sh + USER/cgts 10240 2024-06-12 13:12 metadata.tar + USER/cgts 32 2024-06-12 13:12 signature + USER/cgts 256 2024-06-12 13:12 signature.v2 + USER/cgts 433 2024-06-12 13:12 pre-install.sh + +#. Verify patch metadata. + + .. code-block:: none + + $ tar xf /localdisk/loadbuild/USER/stx-master/patch_output/sample_patch.patch -O metadata.tar | tar x -O + + + starlingx-1.0.0 + 1.0.0 + starlingx + sample patch test + Sample instructions + Sample warning + DEV + N + Y + + + pre-install.sh + post-install.sh + + curl_7.74.0-1.3+deb11u11_amd64.deb + linux-compiler-gcc-10-x86_5.10.205-1.stx.79_amd64.deb + linux-headers-5.10.0-6-amd64_5.10.205-1.stx.84_amd64.deb + linux-headers-5.10.0-6-common_5.10.205-1.stx.84_all.deb + linux-image-5.10.0-6-amd64-unsigned_5.10.205-1.stx.84_amd64.deb + linux-kbuild-5.10_5.10.205-1.stx.84_amd64.deb + linux-libc-dev_5.10.205-1.stx.84_amd64.deb + linux-perf-5.10_5.10.205-1.stx.84_amd64.deb + linux-rt-headers-5.10.0-6-rt-amd64_5.10.205-1.stx.79_amd64.deb + linux-rt-headers-5.10.0-6-rt-common_5.10.205-1.stx.79_all.deb + linux-rt-image-5.10.0-6-rt-amd64-unsigned_5.10.205-1.stx.79_amd64.deb + linux-rt-kbuild-5.10_5.10.205-1.stx.79_amd64.deb + + + + +#. Verify patch content (``software.tar``). + + .. code-block:: none + + $ tar xf /localdisk/loadbuild/USER/stx-master/patch_output/sample_patch.patch -O software.tar | tar t + + linux-kbuild-5.10_5.10.205-1.stx.84_amd64.deb + linux-compiler-gcc-10-x86_5.10.205-1.stx.79_amd64.deb + linux-rt-image-5.10.0-6-rt-amd64-unsigned_5.10.205-1.stx.79_amd64.deb + linux-rt-kbuild-5.10_5.10.205-1.stx.79_amd64.deb + linux-perf-5.10_5.10.205-1.stx.84_amd64.deb + linux-rt-headers-5.10.0-6-rt-common_5.10.205-1.stx.79_all.deb + linux-rt-headers-5.10.0-6-rt-amd64_5.10.205-1.stx.79_amd64.deb + linux-headers-5.10.0-6-amd64_5.10.205-1.stx.84_amd64.deb + linux-image-5.10.0-6-amd64-unsigned_5.10.205-1.stx.84_amd64.deb + linux-libc-dev_5.10.205-1.stx.84_amd64.deb + curl_7.74.0-1.3+deb11u11_amd64.deb + linux-headers-5.10.0-6-common_5.10.205-1.stx.84_all.deb - * ``signature.v2`` is a signature file for ``software.tar`` and - ``metadata.tar``. In this case, it is generated using the key file - ``$MY_REPO/build-tools/signing/dev-private-key.pem``. ------------- Apply a patch diff --git a/doc/source/shared/abbrevs.txt b/doc/source/shared/abbrevs.txt index 68307a082..7355472c3 100755 --- a/doc/source/shared/abbrevs.txt +++ b/doc/source/shared/abbrevs.txt @@ -208,6 +208,7 @@ .. |UDS| replace:: :abbr:`UDS (Unix Domain Socket)` .. |UEFI| replace:: :abbr:`UEFI (Unified Extensible Firmware Interface)` .. |UFT| replace:: :abbr:`UFT (Unified Flow Tool)` +.. |USM| replace:: :abbr:`USM (Unified Software Management)` .. |UUID| replace:: :abbr:`UUID (Universally Unique Identifier)` .. |UUIDs| replace:: :abbr:`UUIDs (Universally Unique Identifiers)` .. |VF| replace:: :abbr:`VF (Virtual Function)`