diff --git a/doc/source/developer_guide/2018_10/index.rst b/doc/source/developer_guide/2018_10/index.rst new file mode 100644 index 000000000..16628792a --- /dev/null +++ b/doc/source/developer_guide/2018_10/index.rst @@ -0,0 +1,839 @@ +=========================== +Developer guide stx.2018.10 +=========================== + +This section contains the steps for building a StarlingX ISO from +the stx.2018.10 branch. + +If a developer guide is needed for a previous release, review the +:doc:`developer guides for all previous releases `. + +------------ +Requirements +------------ + +The recommended minimum requirements include: + +********************* +Hardware requirements +********************* + +A workstation computer with: + +- Processor: x86_64 is the only supported architecture +- Memory: At least 32GB RAM +- Hard Disk: 500GB HDD +- Network: Network adapter with active Internet connection + +********************* +Software requirements +********************* + +A workstation computer with: + +- Operating System: Ubuntu 16.04 LTS 64-bit +- Docker +- Android Repo Tool +- Proxy settings configured (if required) + + - See + http://lists.starlingx.io/pipermail/starlingx-discuss/2018-July/000136.html + for more details + +- Public SSH key + +----------------------------- +Development environment setup +----------------------------- + +This section describes how to set up a StarlingX development system on a +workstation computer. After completing these steps, you can +build a StarlingX ISO image on the following Linux distribution: + +- Ubuntu 16.04 LTS 64-bit + +**************************** +Update your operating system +**************************** + +Before proceeding with the build, ensure your Ubuntu distribution is up to date. +You first need to update the local database list of available packages: + +.. code:: sh + + $ sudo apt-get update + +****************************************** +Installation requirements and dependencies +****************************************** + +^^^^ +User +^^^^ + +1. Make sure you are a non-root user with sudo enabled when you build the + StarlingX ISO. You also need to either use your existing user or create a + separate **: + + .. code:: sh + + $ sudo useradd -m -d /home/ + +2. Your ** should have sudo privileges: + + .. code:: sh + + $ sudo sh -c "echo ' ALL=(ALL:ALL) ALL' >> /etc/sudoers" + $ sudo su -c + +^^^ +Git +^^^ + +3. Install the required packages on the Ubuntu host system: + + .. code:: sh + + $ sudo apt-get install make git curl + +4. Make sure to set up your identity using the following two commands. + Be sure to provide your actual name and email address: + + .. code:: sh + + $ git config --global user.name "Name LastName" + $ git config --global user.email "Email Address" + +^^^^^^^^^ +Docker CE +^^^^^^^^^ + +5. Install the required Docker CE packages in the Ubuntu host system. See + `Get Docker CE for + Ubuntu `__ + for more information. + +6. Log out and log in to add your ** to the Docker group: + + .. code:: sh + + $ sudo usermod -aG docker + +^^^^^^^^^^^^^^^^^ +Android Repo Tool +^^^^^^^^^^^^^^^^^ + +7. Install the required Android Repo Tool in the Ubuntu host system. Follow + the steps in the `Installing + Repo `__ + section. + +********************** +Install public SSH key +********************** + +#. Follow these instructions on GitHub to `Generate a Public SSH + Key `__. + Then upload your public key to your GitHub and Gerrit account + profiles: + + - `Upload to + Github `__ + - `Upload to + Gerrit `__ + +**************************** +Create a workspace directory +**************************** + +#. Create a *starlingx* workspace directory on your system. + Best practices dictate creating the workspace directory + in your $HOME directory: + + .. code:: sh + + $ mkdir -p $HOME/starlingx/ + +************************* +Install stx-tools project +************************* + +#. Under your $HOME directory, clone the project: + + .. code:: sh + + $ cd $HOME + $ git clone https://git.starlingx.io/stx-tools + +#. Navigate to the *<$HOME/stx-tools>* project + directory: + + .. code:: sh + + $ cd $HOME/stx-tools/ + +----------------------------- +Prepare the base Docker image +----------------------------- + +StarlingX base Docker image handles all steps related to StarlingX ISO +creation. This section describes how to customize the base Docker image +building process. + +******************** +Configuration values +******************** + +You can customize values for the StarlingX base Docker image using a +text-based configuration file named ``localrc``: + +- ``HOST_PREFIX`` points to the directory that hosts the 'designer' + subdirectory for source code, the 'loadbuild' subdirectory for + the build environment, generated RPMs, and the ISO image. +- ``HOST_MIRROR_DIR`` points to the directory that hosts the CentOS mirror + repository. + +^^^^^^^^^^^^^^^^^^^^^^^^^^ +localrc configuration file +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Create your ``localrc`` configuration file. For example: + + .. code:: sh + + # tbuilder localrc + MYUNAME= + PROJECT=starlingx + HOST_PREFIX=$HOME/starlingx/workspace + HOST_MIRROR_DIR=$HOME/starlingx/mirror + +*************************** +Build the base Docker image +*************************** + +Once the ``localrc`` configuration file has been customized, it is time +to build the base Docker image. + +#. If necessary, you might have to set http/https proxy in your + Dockerfile before building the docker image: + + .. code:: sh + + ENV http_proxy " http://your.actual_http_proxy.com:your_port " + ENV https_proxy " https://your.actual_https_proxy.com:your_port " + ENV ftp_proxy " http://your.actual_ftp_proxy.com:your_port " + RUN echo " proxy=http://your-proxy.com:port " >> /etc/yum.conf + +#. The ``tb.sh`` script automates the Base Docker image build: + + .. code:: sh + + ./tb.sh create + +---------------------------------- +Build the CentOS mirror repository +---------------------------------- + +The creation of the StarlingX ISO relies on a repository of RPM Binaries, +RPM Sources, and Tar Compressed files. This section describes how to build +this CentOS mirror repository. + +******************************* +Run repository Docker container +******************************* + +| Run the following commands under a terminal identified as "**One**": + +#. Navigate to the *$HOME/stx-tools/centos-mirror-tool* project + directory: + + .. code:: sh + + $ cd $HOME/stx-tools/centos-mirror-tools/ + +#. Launch the Docker container using the previously created base Docker image + *:*. As /localdisk is defined as the workdir of the + container, you should use the same folder name to define the volume. + The container starts to run and populate 'logs' and 'output' folders in + this directory. The container runs from the same directory in which the + scripts are stored. + + .. code:: sh + + $ docker run -it --volume $(pwd):/localdisk local/$USER-stx-builder:7.4 bash + +***************** +Download packages +***************** + +#. Inside the Docker container, enter the following commands to download + the required packages to populate the CentOS mirror repository: + + :: + + # cd localdisk && bash download_mirror.sh + +#. Monitor the download of packages until it is complete. When the download + is complete, the following message appears: + + :: + + totally 17 files are downloaded! + step #3: done successfully + IMPORTANT: The following 3 files are just bootstrap versions. Based on them, the workable images + for StarlingX could be generated by running "update-pxe-network-installer" command after "build-iso" + - out/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img + - out/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img + - out/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz + +*************** +Verify packages +*************** + +#. Verify no missing or failed packages exist: + + :: + + # cat logs/*_missing_*.log + # cat logs/*_failmove_*.log + +#. In case missing or failed packages do exist, which is usually caused by + network instability (or timeout), you need to download the packages + manually. + Doing so assures you get all RPMs listed in + *rpms_3rdparties.lst*/*rpms_centos.lst*/*rpms_centos3rdparties.lst*. + +****************** +Packages structure +****************** + +The following is a general overview of the packages structure resulting +from downloading the packages: + +:: + + /home//stx-tools/centos-mirror-tools/output + └── stx-r1 + └── CentOS + └── pike + ├── Binary + │   ├── EFI + │   ├── images + │   ├── isolinux + │   ├── LiveOS + │   ├── noarch + │   └── x86_64 + ├── downloads + │   ├── integrity + │   └── puppet + └── Source + +******************************* +Create CentOS mirror repository +******************************* + +Outside your Repository Docker container, in another terminal identified +as "**Two**", run the following commands: + +#. From terminal identified as "**Two**", create a *mirror/CentOS* + directory under your *starlingx* workspace directory: + + .. code:: sh + + $ mkdir -p $HOME/starlingx/mirror/CentOS/ + +#. Copy the built CentOS Mirror Repository built under + *$HOME/stx-tools/centos-mirror-tool* to the *$HOME/starlingx/mirror/* + workspace directory: + + .. code:: sh + + $ cp -r $HOME/stx-tools/centos-mirror-tools/output/stx-r1/ $HOME/starlingx/mirror/CentOS/ + + +------------------------- +Create StarlingX packages +------------------------- + +***************************** +Run building Docker container +***************************** + +#. From the terminal identified as "**Two**", create the workspace folder: + + .. code:: sh + + $ mkdir -p $HOME/starlingx/workspace + +#. Navigate to the *$HOME/stx-tools* project directory: + + .. code:: sh + + $ cd $HOME/stx-tools + +#. Verify environment variables: + + .. code:: sh + + $ bash tb.sh env + +#. Run the building Docker container: + + .. code:: sh + + $ bash tb.sh run + +#. Execute the buiding Docker container: + + .. code:: sh + + $ bash tb.sh exec + +********************************* +Download source code repositories +********************************* + +#. From the terminal identified as "**Two**", which is now inside the + Building Docker container, start the internal environment: + + .. code:: sh + + $ eval $(ssh-agent) + $ ssh-add + +#. Use the repo tool to create a local clone of the stx-manifest + Git repository based on the "r/2018.10" branch: + + .. code:: sh + + $ cd $MY_REPO_ROOT_DIR + $ repo init -u https://git.starlingx.io/stx-manifest -m default.xml -b r/2018.10 + + **NOTE:** To use the "repo" command to clone the stx-manifest repository and + check out the "master" branch, omit the "-b r/2018.10" option. + Following is an example: + + .. code:: sh + + $ repo init -u https://git.starlingx.io/stx-manifest -m default.xml + +#. Synchronize the repository: + + .. code:: sh + + $ repo sync -j`nproc` + +#. Create a tarballs repository: + + .. code:: sh + + $ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/ + + Alternatively, you can run the "populate_downloads.sh" script to copy + the tarballs instead of using a symlink: + + .. code:: sh + + $ populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/ + + Outside the container + +#. From another terminal identified as "**Three**", create mirror binaries: + + .. code:: sh + + $ mkdir -p $HOME/starlingx/mirror/CentOS/stx-installer + $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img $HOME/starlingx/mirror/CentOS/stx-installer/initrd.img + $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz $HOME/starlingx/mirror/CentOS/stx-installer/vmlinuz + $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img $HOME/starlingx/mirror/CentOS/stx-installer/squashfs.img + +************** +Build packages +************** + +#. Go back to the terminal identified as "**Two**", which is the Building Docker container. + +#. **Temporal!** Build-Pkgs Errors. Be prepared to have some missing / + corrupted rpm and tarball packages generated during + `Build the CentOS Mirror Repository`_, which will cause the next step + to fail. If that step does fail, manually download those missing / + corrupted packages. + +#. Update the symbolic links: + + .. code:: sh + + $ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/ + +#. Build the packages: + + .. code:: sh + + $ build-pkgs + +#. **Optional!** Generate-Cgcs-Tis-Repo: + + While this step is optional, it improves performance on subsequent + builds. The cgcs-tis-repo has the dependency information that + sequences the build order. To generate or update the information, you + need to execute the following command after building modified or new + packages. + + .. code:: sh + + $ generate-cgcs-tis-repo + +------------------- +Build StarlingX ISO +------------------- + +#. Build the image: + + .. code:: sh + + $ build-iso + +--------------- +Build installer +--------------- + +To get your StarlingX ISO ready to use, you must create the initialization +files used to boot the ISO, additional controllers, and compute nodes. + +**NOTE:** You only need this procedure during your first build and +every time you upgrade the kernel. + +After running "build-iso", run: + +.. code:: sh + + $ build-pkgs --installer + +This builds *rpm* and *anaconda* packages. Then run: + +.. code:: sh + + $ update-pxe-network-installer + +The *update-pxe-network-installer* covers the steps detailed in +*$MY_REPO/stx/stx-metal/installer/initrd/README*. This script +creates three files on +*/localdisk/loadbuild/pxe-network-installer/output*. + +:: + + new-initrd.img + new-squashfs.img + new-vmlinuz + +Rename the files as follows: + +:: + + initrd.img + squashfs.img + vmlinuz + +Two ways exist for using these files: + +#. Store the files in the */import/mirror/CentOS/stx-installer/* folder + for future use. +#. Store the files in an arbitrary location and modify the + *$MY_REPO/stx/stx-metal/installer/pxe-network-installer/centos/build_srpm.data* + file to point to these files. + +Recreate the *pxe-network-installer* package and rebuild the image: + +.. code:: sh + + $ build-pkgs --clean pxe-network-installer + $ build-pkgs pxe-network-installer + $ build-iso + +Your ISO image should be able to boot. + +**************** +Additional notes +**************** + +- In order to get the first boot working, this complete procedure needs + to be done. However, once the init files are created, these can be + stored in a shared location where different developers can make use + of them. Updating these files is not a frequent task and should be + done whenever the kernel is upgraded. +- StarlingX is in active development. Consequently, it is possible that in the + future the **0.2** version will change to a more generic solution. + +--------------- +Build avoidance +--------------- + +******* +Purpose +******* + +Greatly reduce build times after using "repo" to syncronized a local +repository with an upstream source (i.e. "repo sync"). +Build avoidance works well for designers working +within a regional office. Starting from a new workspace, "build-pkgs" +typically requires three or more hours to complete. Build avoidance +reduces this step to approximately 20 minutes. + +*********** +Limitations +*********** + +- Little or no benefit for designers who refresh a pre-existing + workspace at least daily (e.g. download_mirror.sh, repo sync, + generate-cgcs-centos-repo.sh, build-pkgs, build-iso). In these cases, + an incremental build (i.e. reuse of same workspace without a "build-pkgs + --clean") is often just as efficient. +- Not likely to be useful to solo designers, or teleworkers that wish + to compile on using their home computers. Build avoidance downloads build + artifacts from a reference build, and WAN speeds are generally too + slow. + +***************** +Method (in brief) +***************** + +#. Reference Builds + + - A server in the regional office performs regular (e.g. daily) + automated builds using existing methods. These builds are called + "reference builds". + - The builds are timestamped and preserved for some time (i.e. a + number of weeks). + - A build CONTEXT, which is a file produced by "build-pkgs" + at location *$MY_WORKSPACE/CONTEXT*, is captured. It is a bash script that can + cd to each and every Git and checkout the SHA that contributed to + the build. + - For each package built, a file captures the md5sums of all the + source code inputs required to build that package. These files are + also produced by "build-pkgs" at location + *$MY_WORKSPACE//rpmbuild/SOURCES//srpm_reference.md5*. + - All these build products are accessible locally (e.g. a regional + office) using "rsync". + + **NOTE:** Other protocols can be added later. + +#. Designers + + - Request a build avoidance build. Recommended after you have + done synchronized the repository (i.e. "repo sync"). + + :: + + repo sync + generate-cgcs-centos-repo.sh + populate_downloads.sh + build-pkgs --build-avoidance + + - Use combinations of additional arguments, environment variables, and a + configuration file unique to the regional office to specify an URL + to the reference builds. + + - Using a configuration file to specify the location of your reference build: + + :: + + mkdir -p $MY_REPO/local-build-data + + cat <<- EOF > $MY_REPO/local-build-data/build_avoidance_source + # Optional, these are already the default values. + BUILD_AVOIDANCE_DATE_FORMAT="%Y%m%d" + BUILD_AVOIDANCE_TIME_FORMAT="%H%M%S" + BUILD_AVOIDANCE_DATE_TIME_DELIM="T" + BUILD_AVOIDANCE_DATE_TIME_POSTFIX="Z" + BUILD_AVOIDANCE_DATE_UTC=1 + BUILD_AVOIDANCE_FILE_TRANSFER="rsync" + + # Required, unique values for each regional office + BUILD_AVOIDANCE_USR="jenkins" + BUILD_AVOIDANCE_HOST="stx-builder.mycompany.com" + BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build" + EOF + + - Using command-line arguments to specify the location of your reference + build: + + :: + + build-pkgs --build-avoidance --build-avoidance-dir /localdisk/loadbuild/jenkins/StarlingX_Reference_Build --build-avoidance-host stx-builder.mycompany.com --build-avoidance-user jenkins + + - Prior to your build attempt, you need to accept the host key. + Doing so prevents "rsync" failures on a "yes/no" prompt. + You only have to do this once. + + :: + + grep -q $BUILD_AVOIDANCE_HOST $HOME/.ssh/known_hosts + if [ $? != 0 ]; then + ssh-keyscan $BUILD_AVOIDANCE_HOST >> $HOME/.ssh/known_hosts + fi + + + - "build-pkgs" does the following: + + - From newest to oldest, scans the CONTEXTs of the various + reference builds. Selects the first (i.e. most recent) context that + satisfies the following requirement: every Git the SHA + specifies in the CONTEXT is present. + - The selected context might be slightly out of date, but not by + more than a day. This assumes daily reference builds are run. + - If the context has not been previously downloaded, then + download it now. This means you need to download select portions of the + reference build workspace into the designer's workspace. This + includes all the SRPMS, RPMS, MD5SUMS, and miscellaneous supporting + files. Downloading these files usually takes about 10 minutes + over an office LAN. + - The designer could have additional commits or uncommitted changes + not present in the reference builds. Affected packages are + identified by the differing md5sum's. In these cases, the packages + are re-built. Re-builds usually take five or more minutes, + depending on the packages that have changed. + + - What if no valid reference build is found? Then build-pkgs will fall + back to a regular build. + +**************** +Reference builds +**************** + +- The regional office implements an automated build that pulls the + latest StarlingX software and builds it on a regular basis (e.g. + daily builds). Jenkins, cron, or similar tools can trigger these builds. +- Each build is saved to a unique directory, and preserved for a time + that is reflective of how long a designer might be expected to work + on a private branch without syncronizing with the master branch. + This takes about two weeks. + +- The *MY_WORKSPACE* directory for the build shall have a common root + directory, and a leaf directory that is a sortable time stamp. The + suggested format is *YYYYMMDDThhmmss*. + + .. code:: sh + + $ sudo apt-get update + BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build" + BUILD_TIMESTAMP=$(date -u '+%Y%m%dT%H%M%SZ') + MY_WORKSPACE=${BUILD_AVOIDANCE_DIR}/${BUILD_TIMESTAMP} + +- Designers can access all build products over the internal network of + the regional office. The current prototype employs "rsync". Other + protocols that can efficiently share, copy, or transfer large directories + of content can be added as needed. + +^^^^^^^^^^^^^^ +Advanced usage +^^^^^^^^^^^^^^ + +Can the reference build itself use build avoidance? Yes it can. +Can it reference itself? Yes it can. +In both these cases, caution is advised. To protect against any possible +'divergence from reality', you should limit how many steps you remove +a build avoidance build from a full build. + +Suppose we want to implement a self-referencing daily build in an +environment where a full build already occurs every Saturday. +To protect ourselves from a +build failure on Saturday we also want a limit of seven days since +the last full build. Your build script might look like this ... + +:: + + ... + BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build" + BUILD_AVOIDANCE_HOST="stx-builder.mycompany.com" + FULL_BUILD_DAY="Saturday" + MAX_AGE_DAYS=7 + + LAST_FULL_BUILD_LINK="$BUILD_AVOIDANCE_DIR/latest_full_build" + LAST_FULL_BUILD_DAY="" + NOW_DAY=$(date -u "+%A") + BUILD_TIMESTAMP=$(date -u '+%Y%m%dT%H%M%SZ') + MY_WORKSPACE=${BUILD_AVOIDANCE_DIR}/${BUILD_TIMESTAMP} + + # update software + repo init -u ${BUILD_REPO_URL} -b ${BUILD_BRANCH} + repo sync --force-sync + $MY_REPO_ROOT_DIR/stx-tools/toCOPY/generate-cgcs-centos-repo.sh + $MY_REPO_ROOT_DIR/stx-tools/toCOPY/populate_downloads.sh + + # User can optionally define BUILD_METHOD equal to one of 'FULL', 'AVOIDANCE', or 'AUTO' + # Sanitize BUILD_METHOD + if [ "$BUILD_METHOD" != "FULL" ] && [ "$BUILD_METHOD" != "AVOIDANCE" ]; then + BUILD_METHOD="AUTO" + fi + + # First build test + if [ "$BUILD_METHOD" != "FULL" ] && [ ! -L $LAST_FULL_BUILD_LINK ]; then + echo "latest_full_build symlink missing, forcing full build" + BUILD_METHOD="FULL" + fi + + # Build day test + if [ "$BUILD_METHOD" == "AUTO" ] && [ "$NOW_DAY" == "$FULL_BUILD_DAY" ]; then + echo "Today is $FULL_BUILD_DAY, forcing full build" + BUILD_METHOD="FULL" + fi + + # Build age test + if [ "$BUILD_METHOD" != "FULL" ]; then + LAST_FULL_BUILD_DATE=$(basename $(readlink $LAST_FULL_BUILD_LINK) | cut -d '_' -f 1) + LAST_FULL_BUILD_DAY=$(date -d $LAST_FULL_BUILD_DATE "+%A") + AGE_SECS=$(( $(date "+%s") - $(date -d $LAST_FULL_BUILD_DATE "+%s") )) + AGE_DAYS=$(( $AGE_SECS/60/60/24 )) + if [ $AGE_DAYS -ge $MAX_AGE_DAYS ]; then + echo "Haven't had a full build in $AGE_DAYS days, forcing full build" + BUILD_METHOD="FULL" + fi + BUILD_METHOD="AVOIDANCE" + fi + + #Build it + if [ "$BUILD_METHOD" == "FULL" ]; then + build-pkgs --no-build-avoidance + else + build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER + fi + if [ $? -ne 0 ]; then + echo "Build failed in build-pkgs" + exit 1 + fi + + build-iso + if [ $? -ne 0 ]; then + echo "Build failed in build-iso" + exit 1 + fi + + if [ "$BUILD_METHOD" == "FULL" ]; then + # A successful full build. Set last full build symlink. + if [ -L $LAST_FULL_BUILD_LINK ]; then + rm -rf $LAST_FULL_BUILD_LINK + fi + ln -sf $MY_WORKSPACE $LAST_FULL_BUILD_LINK + fi + ... + +A final note.... + +To use the full build day as your avoidance build reference point, +modify the "build-pkgs" commands above to use "--build-avoidance-day ", +as shown in the following two examples: + +:: + + build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $FULL_BUILD_DAY + + # Here is another example with a bit more shuffling of the above script. + + build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $LAST_FULL_BUILD_DAY + +The advantage is that our build is never more than one step removed +from a full build. This assumes the full build was successful. + +The disadvantage is that by the end of the week, the reference build is getting +rather old. During active weeks, build times could approach build times for +full builds. diff --git a/doc/source/developer_guide/index.rst b/doc/source/developer_guide/index.rst index 250a1f1ad..22eefd12c 100644 --- a/doc/source/developer_guide/index.rst +++ b/doc/source/developer_guide/index.rst @@ -1,838 +1,16 @@ -.. _developer-guide: +================ +Developer guides +================ -=============== -Developer Guide -=============== +Developer guides for StarlingX are release specific. To build a +StarlingX ISO from the latest release, use the +:doc:`/developer_guide/2018_10/index`. -This section contains the steps for building a StarlingX ISO from -the "r/2018.10" branch. +To build an ISO from a previous release of StarlingX, use the +developer guide for your specific release: ------------- -Requirements ------------- +.. toctree:: + :maxdepth: 1 -The recommended minimum requirements include: - -********************* -Hardware Requirements -********************* - -A workstation computer with: - -- Processor: x86_64 is the only supported architecture -- Memory: At least 32GB RAM -- Hard Disk: 500GB HDD -- Network: Network adapter with active Internet connection - -********************* -Software Requirements -********************* - -A workstation computer with: - -- Operating System: Ubuntu 16.04 LTS 64-bit -- Docker -- Android Repo Tool -- Proxy Settings Configured (If Required) - - - See - http://lists.starlingx.io/pipermail/starlingx-discuss/2018-July/000136.html - for more details - -- Public SSH Key - ------------------------------ -Development Environment Setup ------------------------------ - -This section describes how to set up a StarlingX development system on a -workstation computer. After completing these steps, you can -build a StarlingX ISO image on the following Linux distribution: - -- Ubuntu 16.04 LTS 64-bit - -**************************** -Update Your Operating System -**************************** - -Before proceeding with the build, ensure your Ubuntu distribution is up to date. -You first need to update the local database list of available packages: - -.. code:: sh - - $ sudo apt-get update - -****************************************** -Installation Requirements and Dependencies -****************************************** - -^^^^ -User -^^^^ - -1. Make sure you are a non-root user with sudo enabled when you build the - StarlingX ISO. You also need to either use your existing user or create a - separate **: - - .. code:: sh - - $ sudo useradd -m -d /home/ - -2. Your ** should have sudo privileges: - - .. code:: sh - - $ sudo sh -c "echo ' ALL=(ALL:ALL) ALL' >> /etc/sudoers" - $ sudo su -c - -^^^ -Git -^^^ - -3. Install the required packages on the Ubuntu host system: - - .. code:: sh - - $ sudo apt-get install make git curl - -4. Make sure to set up your identity using the following two commands. - Be sure to provide your actual name and email address: - - .. code:: sh - - $ git config --global user.name "Name LastName" - $ git config --global user.email "Email Address" - -^^^^^^^^^ -Docker CE -^^^^^^^^^ - -5. Install the required Docker CE packages in the Ubuntu host system. See - `Get Docker CE for - Ubuntu `__ - for more information. - -6. Log out and log in to add your ** to the Docker group: - - .. code:: sh - - $ sudo usermod -aG docker - -^^^^^^^^^^^^^^^^^ -Android Repo Tool -^^^^^^^^^^^^^^^^^ - -7. Install the required Android Repo Tool in the Ubuntu host system. Follow - the steps in the `Installing - Repo `__ - section. - -********************** -Install Public SSH Key -********************** - -#. Follow these instructions on GitHub to `Generate a Public SSH - Key `__. - Then upload your public key to your GitHub and Gerrit account - profiles: - - - `Upload to - Github `__ - - `Upload to - Gerrit `__ - -**************************** -Create a Workspace Directory -**************************** - -#. Create a *starlingx* workspace directory on your system. - Best practices dictate creating the workspace directory - in your $HOME directory: - - .. code:: sh - - $ mkdir -p $HOME/starlingx/ - -************************* -Install stx-tools Project -************************* - -#. Under your $HOME directory, clone the project: - - .. code:: sh - - $ cd $HOME - $ git clone https://git.starlingx.io/stx-tools - -#. Navigate to the *<$HOME/stx-tools>* project - directory: - - .. code:: sh - - $ cd $HOME/stx-tools/ - ------------------------------ -Prepare the Base Docker Image ------------------------------ - -StarlingX base docker image handles all steps related to StarlingX ISO -creation. This section describes how to customize the base Docker image -building process. - -******************** -Configuration Values -******************** - -You can customize values for the StarlingX base Docker image using a -text-based configuration file named ``localrc``: - -- ``HOST_PREFIX`` points to the directory that hosts the 'designer' - subdirectory for source code, the 'loadbuild' subdirectory for - the build environment, generated RPMs, and the ISO image. -- ``HOST_MIRROR_DIR`` points to the directory that hosts the CentOS mirror - repository. - -^^^^^^^^^^^^^^^^^^^^^^^^^^ -localrc configuration file -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Create your ``localrc`` configuration file. Following is an example: - - .. code:: sh - - # tbuilder localrc - MYUNAME= - PROJECT=starlingx - HOST_PREFIX=$HOME/starlingx/workspace - HOST_MIRROR_DIR=$HOME/starlingx/mirror - -*************************** -Build the base Docker image -*************************** - -Once the ``localrc`` configuration file has been customized, it is time -to build the base Docker image. - -#. If necessary, you might have to set http/https proxy in your - Dockerfile before building the docker image: - - .. code:: sh - - ENV http_proxy " http://your.actual_http_proxy.com:your_port " - ENV https_proxy " https://your.actual_https_proxy.com:your_port " - ENV ftp_proxy " http://your.actual_ftp_proxy.com:your_port " - RUN echo " proxy=http://your-proxy.com:port " >> /etc/yum.conf - -#. The ``tb.sh`` script automates the Base Docker image build: - - .. code:: sh - - ./tb.sh create - ----------------------------------- -Build the CentOS Mirror Repository ----------------------------------- - -The creation of the StarlingX ISO relies on a repository of RPM Binaries, -RPM Sources, and Tar Compressed files. This section describes how to build -this CentOS mirror repository. - -******************************* -Run Repository Docker Container -******************************* - -| Run the following commands under a terminal identified as "**One**": - -#. Navigate to the *$HOME/stx-tools/centos-mirror-tool* project - directory: - - .. code:: sh - - $ cd $HOME/stx-tools/centos-mirror-tools/ - -#. Launch the Docker container using the previously created base Docker image - *:*. As /localdisk is defined as the workdir of the - container, you should use the same folder name to define the volume. - The container starts to run and populate 'logs' and 'output' folders in - this directory. The container runs from the same directory in which the - scripts are stored. - - .. code:: sh - - $ docker run -it --volume $(pwd):/localdisk local/$USER-stx-builder:7.4 bash - -***************** -Download Packages -***************** - -#. Inside the Docker container, enter the following commands to download - the required packages to populate the CentOS mirror repository: - - :: - - # cd localdisk && bash download_mirror.sh - -#. Monitor the download of packages until it is complete. When the download - is complete, the following message appears: - - :: - - totally 17 files are downloaded! - step #3: done successfully - IMPORTANT: The following 3 files are just bootstrap versions. Based on them, the workable images - for StarlingX could be generated by running "update-pxe-network-installer" command after "build-iso" - - out/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img - - out/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img - - out/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz - -*************** -Verify Packages -*************** - -#. Verify no missing or failed packages exist: - - :: - - # cat logs/*_missing_*.log - # cat logs/*_failmove_*.log - -#. In case missing or failed packages do exist, which is usually caused by - network instability (or timeout), you need to download the packages - manually. - Doing so assures you get all RPMs listed in - *rpms_3rdparties.lst*/*rpms_centos.lst*/*rpms_centos3rdparties.lst*. - -****************** -Packages Structure -****************** - -The following is a general overview of the packages structure resulting -from downloading the packages: - -:: - - /home//stx-tools/centos-mirror-tools/output - └── stx-r1 - └── CentOS - └── pike - ├── Binary - │   ├── EFI - │   ├── images - │   ├── isolinux - │   ├── LiveOS - │   ├── noarch - │   └── x86_64 - ├── downloads - │   ├── integrity - │   └── puppet - └── Source - -******************************* -Create CentOS Mirror Repository -******************************* - -Outside your Repository Docker container, in another terminal identified -as "**Two**", run the following commands: - -#. From terminal identified as "**Two**", create a *mirror/CentOS* - directory under your *starlingx* workspace directory: - - .. code:: sh - - $ mkdir -p $HOME/starlingx/mirror/CentOS/ - -#. Copy the built CentOS Mirror Repository built under - *$HOME/stx-tools/centos-mirror-tool* to the *$HOME/starlingx/mirror/* - workspace directory: - - .. code:: sh - - $ cp -r $HOME/stx-tools/centos-mirror-tools/output/stx-r1/ $HOME/starlingx/mirror/CentOS/ - - -------------------------- -Create StarlingX Packages -------------------------- - -***************************** -Run Building Docker Container -***************************** - -#. From the terminal identified as "**Two**", create the workspace folder: - - .. code:: sh - - $ mkdir -p $HOME/starlingx/workspace - -#. Navigate to the *$HOME/stx-tools* project directory: - - .. code:: sh - - $ cd $HOME/stx-tools - -#. Verify environment variables: - - .. code:: sh - - $ bash tb.sh env - -#. Run the building Docker container: - - .. code:: sh - - $ bash tb.sh run - -#. Execute the buiding Docker container: - - .. code:: sh - - $ bash tb.sh exec - -********************************* -Download Source Code Repositories -********************************* - -#. From the terminal identified as "**Two**", which is now inside the - Building Docker container, start the internal environment: - - .. code:: sh - - $ eval $(ssh-agent) - $ ssh-add - -#. Use the repo tool to create a local clone of the stx-manifest - Git repository based on the "r/2018.10" branch: - - .. code:: sh - - $ cd $MY_REPO_ROOT_DIR - $ repo init -u https://git.starlingx.io/stx-manifest -m default.xml -b r/2018.10 - - **NOTE:** To use the "repo" command to clone the stx-manifest repository and - check out the "master" branch, omit the "-b r/2018.10" option. - Following is an example: - - .. code:: sh - - $ repo init -u https://git.starlingx.io/stx-manifest -m default.xml - -#. Synchronize the repository: - - .. code:: sh - - $ repo sync -j`nproc` - -#. Create a tarballs repository: - - .. code:: sh - - $ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/ - - Alternatively, you can run the "populate_downloads.sh" script to copy - the tarballs instead of using a symlink: - - .. code:: sh - - $ populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/ - - Outside the container - -#. From another terminal identified as "**Three**", create mirror binaries: - - .. code:: sh - - $ mkdir -p $HOME/starlingx/mirror/CentOS/stx-installer - $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img $HOME/starlingx/mirror/CentOS/stx-installer/initrd.img - $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz $HOME/starlingx/mirror/CentOS/stx-installer/vmlinuz - $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img $HOME/starlingx/mirror/CentOS/stx-installer/squashfs.img - -************** -Build Packages -************** - -#. Go back to the terminal identified as "**Two**", which is the Building Docker container. - -#. **Temporal!** Build-Pkgs Errors. Be prepared to have some missing / - corrupted rpm and tarball packages generated during - `Build the CentOS Mirror Repository`_, which will cause the next step - to fail. If that step does fail, manually download those missing / - corrupted packages. - -#. Update the symbolic links: - - .. code:: sh - - $ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/ - -#. Build the packages: - - .. code:: sh - - $ build-pkgs - -#. **Optional!** Generate-Cgcs-Tis-Repo: - - While this step is optional, it improves performance on subsequent - builds. The cgcs-tis-repo has the dependency information that - sequences the build order. To generate or update the information, you - need to execute the following command after building modified or new - packages. - - .. code:: sh - - $ generate-cgcs-tis-repo - -------------------- -Build StarlingX ISO -------------------- - -#. Build the image: - - .. code:: sh - - $ build-iso - ---------------- -Build Installer ---------------- - -To get your StarlingX ISO ready to use, you must create the initialization -files used to boot the ISO, additional controllers, and compute nodes. - -**NOTE:** You only need this procedure during your first build and -every time you upgrade the kernel. - -After running "build-iso", run: - -.. code:: sh - - $ build-pkgs --installer - -This builds *rpm* and *anaconda* packages. Then run: - -.. code:: sh - - $ update-pxe-network-installer - -The *update-pxe-network-installer* covers the steps detailed in -*$MY_REPO/stx/stx-metal/installer/initrd/README*. This script -creates three files on -*/localdisk/loadbuild/pxe-network-installer/output*. - -:: - - new-initrd.img - new-squashfs.img - new-vmlinuz - -Rename the files as follows: - -:: - - initrd.img - squashfs.img - vmlinuz - -Two ways exist for using these files: - -#. Store the files in the */import/mirror/CentOS/stx-installer/* folder - for future use. -#. Store the files in an arbitrary location and modify the - *$MY_REPO/stx/stx-metal/installer/pxe-network-installer/centos/build_srpm.data* - file to point to these files. - -Recreate the *pxe-network-installer* package and rebuild the image: - -.. code:: sh - - $ build-pkgs --clean pxe-network-installer - $ build-pkgs pxe-network-installer - $ build-iso - -Your ISO image should be able to boot. - -**************** -Additional Notes -**************** - -- In order to get the first boot working, this complete procedure needs - to be done. However, once the init files are created, these can be - stored in a shared location where different developers can make use - of them. Updating these files is not a frequent task and should be - done whenever the kernel is upgraded. -- StarlingX is in active development. Consequently, it is possible that in the - future the **0.2** version will change to a more generic solution. - ---------------- -Build Avoidance ---------------- - -******* -Purpose -******* - -Greatly reduce build times after using "repo" to syncronized a local -repository with an upstream source (i.e. "repo sync"). -Build avoidance works well for designers working -within a regional office. Starting from a new workspace, "build-pkgs" -typically requires three or more hours to complete. Build avoidance -reduces this step to approximately 20 minutes. - -*********** -Limitations -*********** - -- Little or no benefit for designers who refresh a pre-existing - workspace at least daily (e.g. download_mirror.sh, repo sync, - generate-cgcs-centos-repo.sh, build-pkgs, build-iso). In these cases, - an incremental build (i.e. reuse of same workspace without a "build-pkgs - --clean") is often just as efficient. -- Not likely to be useful to solo designers, or teleworkers that wish - to compile on using their home computers. Build avoidance downloads build - artifacts from a reference build, and WAN speeds are generally too - slow. - -***************** -Method (in brief) -***************** - -#. Reference Builds - - - A server in the regional office performs regular (e.g. daily) - automated builds using existing methods. These builds are called - "reference builds". - - The builds are timestamped and preserved for some time (i.e. a - number of weeks). - - A build CONTEXT, which is a file produced by "build-pkgs" - at location *$MY_WORKSPACE/CONTEXT*, is captured. It is a bash script that can - cd to each and every Git and checkout the SHA that contributed to - the build. - - For each package built, a file captures the md5sums of all the - source code inputs required to build that package. These files are - also produced by "build-pkgs" at location - *$MY_WORKSPACE//rpmbuild/SOURCES//srpm_reference.md5*. - - All these build products are accessible locally (e.g. a regional - office) using "rsync". - - **NOTE:** Other protocols can be added later. - -#. Designers - - - Request a build avoidance build. Recommended after you have - done synchronized the repository (i.e. "repo sync"). - - :: - - repo sync - generate-cgcs-centos-repo.sh - populate_downloads.sh - build-pkgs --build-avoidance - - - Use combinations of additional arguments, environment variables, and a - configuration file unique to the regional office to specify an URL - to the reference builds. - - - Using a configuration file to specify the location of your reference build: - - :: - - mkdir -p $MY_REPO/local-build-data - - cat <<- EOF > $MY_REPO/local-build-data/build_avoidance_source - # Optional, these are already the default values. - BUILD_AVOIDANCE_DATE_FORMAT="%Y%m%d" - BUILD_AVOIDANCE_TIME_FORMAT="%H%M%S" - BUILD_AVOIDANCE_DATE_TIME_DELIM="T" - BUILD_AVOIDANCE_DATE_TIME_POSTFIX="Z" - BUILD_AVOIDANCE_DATE_UTC=1 - BUILD_AVOIDANCE_FILE_TRANSFER="rsync" - - # Required, unique values for each regional office - BUILD_AVOIDANCE_USR="jenkins" - BUILD_AVOIDANCE_HOST="stx-builder.mycompany.com" - BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build" - EOF - - - Using command-line arguments to specify the location of your reference - build: - - :: - - build-pkgs --build-avoidance --build-avoidance-dir /localdisk/loadbuild/jenkins/StarlingX_Reference_Build --build-avoidance-host stx-builder.mycompany.com --build-avoidance-user jenkins - - - Prior to your build attempt, you need to accept the host key. - Doing so prevents "rsync" failures on a "yes/no" prompt. - You only have to do this once. - - :: - - grep -q $BUILD_AVOIDANCE_HOST $HOME/.ssh/known_hosts - if [ $? != 0 ]; then - ssh-keyscan $BUILD_AVOIDANCE_HOST >> $HOME/.ssh/known_hosts - fi - - - - "build-pkgs" does the following: - - - From newest to oldest, scans the CONTEXTs of the various - reference builds. Selects the first (i.e. most recent) context that - satisfies the following requirement: every Git the SHA - specifies in the CONTEXT is present. - - The selected context might be slightly out of date, but not by - more than a day. This assumes daily reference builds are run. - - If the context has not been previously downloaded, then - download it now. This means you need to download select portions of the - reference build workspace into the designer's workspace. This - includes all the SRPMS, RPMS, MD5SUMS, and miscellaneous supporting - files. Downloading these files usually takes about 10 minutes - over an office LAN. - - The designer could have additional commits or uncommitted changes - not present in the reference builds. Affected packages are - identified by the differing md5sum's. In these cases, the packages - are re-built. Re-builds usually take five or more minutes, - depending on the packages that have changed. - - - What if no valid reference build is found? Then build-pkgs will fall - back to a regular build. - -**************** -Reference Builds -**************** - -- The regional office implements an automated build that pulls the - latest StarlingX software and builds it on a regular basis (e.g. - daily builds). Jenkins, cron, or similar tools can trigger these builds. -- Each build is saved to a unique directory, and preserved for a time - that is reflective of how long a designer might be expected to work - on a private branch without syncronizing with the master branch. - This takes about two weeks. - -- The *MY_WORKSPACE* directory for the build shall have a common root - directory, and a leaf directory that is a sortable time stamp. The - suggested format is *YYYYMMDDThhmmss*. - - .. code:: sh - - $ sudo apt-get update - BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build" - BUILD_TIMESTAMP=$(date -u '+%Y%m%dT%H%M%SZ') - MY_WORKSPACE=${BUILD_AVOIDANCE_DIR}/${BUILD_TIMESTAMP} - -- Designers can access all build products over the internal network of - the regional office. The current prototype employs "rsync". Other - protocols that can efficiently share, copy, or transfer large directories - of content can be added as needed. - -^^^^^^^^^^^^^^ -Advanced Usage -^^^^^^^^^^^^^^ - -Can the reference build itself use build avoidance? Yes it can. -Can it reference itself? Yes it can. -In both these cases, caution is advised. To protect against any possible -'divergence from reality', you should limit how many steps you remove -a build avoidance build from a full build. - -Suppose we want to implement a self-referencing daily build in an -environment where a full build already occurs every Saturday. -To protect ourselves from a -build failure on Saturday we also want a limit of seven days since -the last full build. Your build script might look like this ... - -:: - - ... - BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build" - BUILD_AVOIDANCE_HOST="stx-builder.mycompany.com" - FULL_BUILD_DAY="Saturday" - MAX_AGE_DAYS=7 - - LAST_FULL_BUILD_LINK="$BUILD_AVOIDANCE_DIR/latest_full_build" - LAST_FULL_BUILD_DAY="" - NOW_DAY=$(date -u "+%A") - BUILD_TIMESTAMP=$(date -u '+%Y%m%dT%H%M%SZ') - MY_WORKSPACE=${BUILD_AVOIDANCE_DIR}/${BUILD_TIMESTAMP} - - # update software - repo init -u ${BUILD_REPO_URL} -b ${BUILD_BRANCH} - repo sync --force-sync - $MY_REPO_ROOT_DIR/stx-tools/toCOPY/generate-cgcs-centos-repo.sh - $MY_REPO_ROOT_DIR/stx-tools/toCOPY/populate_downloads.sh - - # User can optionally define BUILD_METHOD equal to one of 'FULL', 'AVOIDANCE', or 'AUTO' - # Sanitize BUILD_METHOD - if [ "$BUILD_METHOD" != "FULL" ] && [ "$BUILD_METHOD" != "AVOIDANCE" ]; then - BUILD_METHOD="AUTO" - fi - - # First build test - if [ "$BUILD_METHOD" != "FULL" ] && [ ! -L $LAST_FULL_BUILD_LINK ]; then - echo "latest_full_build symlink missing, forcing full build" - BUILD_METHOD="FULL" - fi - - # Build day test - if [ "$BUILD_METHOD" == "AUTO" ] && [ "$NOW_DAY" == "$FULL_BUILD_DAY" ]; then - echo "Today is $FULL_BUILD_DAY, forcing full build" - BUILD_METHOD="FULL" - fi - - # Build age test - if [ "$BUILD_METHOD" != "FULL" ]; then - LAST_FULL_BUILD_DATE=$(basename $(readlink $LAST_FULL_BUILD_LINK) | cut -d '_' -f 1) - LAST_FULL_BUILD_DAY=$(date -d $LAST_FULL_BUILD_DATE "+%A") - AGE_SECS=$(( $(date "+%s") - $(date -d $LAST_FULL_BUILD_DATE "+%s") )) - AGE_DAYS=$(( $AGE_SECS/60/60/24 )) - if [ $AGE_DAYS -ge $MAX_AGE_DAYS ]; then - echo "Haven't had a full build in $AGE_DAYS days, forcing full build" - BUILD_METHOD="FULL" - fi - BUILD_METHOD="AVOIDANCE" - fi - - #Build it - if [ "$BUILD_METHOD" == "FULL" ]; then - build-pkgs --no-build-avoidance - else - build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER - fi - if [ $? -ne 0 ]; then - echo "Build failed in build-pkgs" - exit 1 - fi - - build-iso - if [ $? -ne 0 ]; then - echo "Build failed in build-iso" - exit 1 - fi - - if [ "$BUILD_METHOD" == "FULL" ]; then - # A successful full build. Set last full build symlink. - if [ -L $LAST_FULL_BUILD_LINK ]; then - rm -rf $LAST_FULL_BUILD_LINK - fi - ln -sf $MY_WORKSPACE $LAST_FULL_BUILD_LINK - fi - ... - -A final note.... - -To use the full build day as your avoidance build reference point, -modify the "build-pkgs" commands above to use "--build-avoidance-day ", -as shown in the following two examples: - -:: - - build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $FULL_BUILD_DAY - - # Here is another example with a bit more shuffling of the above script. - - build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $LAST_FULL_BUILD_DAY - -The advantage is that our build is never more than one step removed -from a full build. This assumes the full build was successful. - -The disadvantage is that by the end of the week, the reference build is getting -rather old. During active weeks, build times could approach build times for -full builds. + /developer_guide/latest/index + /developer_guide/2018_10/index diff --git a/doc/source/developer_guide/latest/index.rst b/doc/source/developer_guide/latest/index.rst new file mode 100644 index 000000000..4c4f458e9 --- /dev/null +++ b/doc/source/developer_guide/latest/index.rst @@ -0,0 +1,839 @@ +=========================== +Developer guide stx.2019.05 +=========================== + +This section contains the steps for building a StarlingX ISO from +the stx.2019.05 branch. + +If a developer guide is needed for a previous release, review the +:doc:`developer guides for all previous releases `. + +------------ +Requirements +------------ + +The recommended minimum requirements include: + +********************* +Hardware requirements +********************* + +A workstation computer with: + +- Processor: x86_64 is the only supported architecture +- Memory: At least 32GB RAM +- Hard Disk: 500GB HDD +- Network: Network adapter with active Internet connection + +********************* +Software requirements +********************* + +A workstation computer with: + +- Operating System: Ubuntu 16.04 LTS 64-bit +- Docker +- Android Repo Tool +- Proxy settings configured (if required) + + - See + http://lists.starlingx.io/pipermail/starlingx-discuss/2018-July/000136.html + for more details + +- Public SSH key + +----------------------------- +Development environment setup +----------------------------- + +This section describes how to set up a StarlingX development system on a +workstation computer. After completing these steps, you can +build a StarlingX ISO image on the following Linux distribution: + +- Ubuntu 16.04 LTS 64-bit + +**************************** +Update your operating system +**************************** + +Before proceeding with the build, ensure your Ubuntu distribution is up to date. +You first need to update the local database list of available packages: + +.. code:: sh + + $ sudo apt-get update + +****************************************** +Installation requirements and dependencies +****************************************** + +^^^^ +User +^^^^ + +1. Make sure you are a non-root user with sudo enabled when you build the + StarlingX ISO. You also need to either use your existing user or create a + separate **: + + .. code:: sh + + $ sudo useradd -m -d /home/ + +2. Your ** should have sudo privileges: + + .. code:: sh + + $ sudo sh -c "echo ' ALL=(ALL:ALL) ALL' >> /etc/sudoers" + $ sudo su -c + +^^^ +Git +^^^ + +3. Install the required packages on the Ubuntu host system: + + .. code:: sh + + $ sudo apt-get install make git curl + +4. Make sure to set up your identity using the following two commands. + Be sure to provide your actual name and email address: + + .. code:: sh + + $ git config --global user.name "Name LastName" + $ git config --global user.email "Email Address" + +^^^^^^^^^ +Docker CE +^^^^^^^^^ + +5. Install the required Docker CE packages in the Ubuntu host system. See + `Get Docker CE for + Ubuntu `__ + for more information. + +6. Log out and log in to add your ** to the Docker group: + + .. code:: sh + + $ sudo usermod -aG docker + +^^^^^^^^^^^^^^^^^ +Android Repo Tool +^^^^^^^^^^^^^^^^^ + +7. Install the required Android Repo Tool in the Ubuntu host system. Follow + the steps in the `Installing + Repo `__ + section. + +********************** +Install public SSH key +********************** + +#. Follow these instructions on GitHub to `Generate a Public SSH + Key `__. + Then upload your public key to your GitHub and Gerrit account + profiles: + + - `Upload to + Github `__ + - `Upload to + Gerrit `__ + +**************************** +Create a workspace directory +**************************** + +#. Create a *starlingx* workspace directory on your system. + Best practices dictate creating the workspace directory + in your $HOME directory: + + .. code:: sh + + $ mkdir -p $HOME/starlingx/ + +************************* +Install stx-tools project +************************* + +#. Under your $HOME directory, clone the project: + + .. code:: sh + + $ cd $HOME + $ git clone https://git.starlingx.io/stx-tools + +#. Navigate to the *<$HOME/stx-tools>* project + directory: + + .. code:: sh + + $ cd $HOME/stx-tools/ + +----------------------------- +Prepare the base Docker image +----------------------------- + +StarlingX base Docker image handles all steps related to StarlingX ISO +creation. This section describes how to customize the base Docker image +building process. + +******************** +Configuration values +******************** + +You can customize values for the StarlingX base Docker image using a +text-based configuration file named ``localrc``: + +- ``HOST_PREFIX`` points to the directory that hosts the 'designer' + subdirectory for source code, the 'loadbuild' subdirectory for + the build environment, generated RPMs, and the ISO image. +- ``HOST_MIRROR_DIR`` points to the directory that hosts the CentOS mirror + repository. + +^^^^^^^^^^^^^^^^^^^^^^^^^^ +localrc configuration file +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Create your ``localrc`` configuration file. For example: + + .. code:: sh + + # tbuilder localrc + MYUNAME= + PROJECT=starlingx + HOST_PREFIX=$HOME/starlingx/workspace + HOST_MIRROR_DIR=$HOME/starlingx/mirror + +*************************** +Build the base Docker image +*************************** + +Once the ``localrc`` configuration file has been customized, it is time +to build the base Docker image. + +#. If necessary, you might have to set http/https proxy in your + Dockerfile before building the docker image: + + .. code:: sh + + ENV http_proxy " http://your.actual_http_proxy.com:your_port " + ENV https_proxy " https://your.actual_https_proxy.com:your_port " + ENV ftp_proxy " http://your.actual_ftp_proxy.com:your_port " + RUN echo " proxy=http://your-proxy.com:port " >> /etc/yum.conf + +#. The ``tb.sh`` script automates the Base Docker image build: + + .. code:: sh + + ./tb.sh create + +---------------------------------- +Build the CentOS mirror repository +---------------------------------- + +The creation of the StarlingX ISO relies on a repository of RPM Binaries, +RPM Sources, and Tar Compressed files. This section describes how to build +this CentOS mirror repository. + +******************************* +Run repository Docker container +******************************* + +| Run the following commands under a terminal identified as "**One**": + +#. Navigate to the *$HOME/stx-tools/centos-mirror-tool* project + directory: + + .. code:: sh + + $ cd $HOME/stx-tools/centos-mirror-tools/ + +#. Launch the Docker container using the previously created base Docker image + *:*. As /localdisk is defined as the workdir of the + container, you should use the same folder name to define the volume. + The container starts to run and populate 'logs' and 'output' folders in + this directory. The container runs from the same directory in which the + scripts are stored. + + .. code:: sh + + $ docker run -it --volume $(pwd):/localdisk local/$USER-stx-builder:7.4 bash + +***************** +Download packages +***************** + +#. Inside the Docker container, enter the following commands to download + the required packages to populate the CentOS mirror repository: + + :: + + # cd localdisk && bash download_mirror.sh + +#. Monitor the download of packages until it is complete. When the download + is complete, the following message appears: + + :: + + totally 17 files are downloaded! + step #3: done successfully + IMPORTANT: The following 3 files are just bootstrap versions. Based on them, the workable images + for StarlingX could be generated by running "update-pxe-network-installer" command after "build-iso" + - out/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img + - out/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img + - out/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz + +*************** +Verify packages +*************** + +#. Verify no missing or failed packages exist: + + :: + + # cat logs/*_missing_*.log + # cat logs/*_failmove_*.log + +#. In case missing or failed packages do exist, which is usually caused by + network instability (or timeout), you need to download the packages + manually. + Doing so assures you get all RPMs listed in + *rpms_3rdparties.lst*/*rpms_centos.lst*/*rpms_centos3rdparties.lst*. + +****************** +Packages structure +****************** + +The following is a general overview of the packages structure resulting +from downloading the packages: + +:: + + /home//stx-tools/centos-mirror-tools/output + └── stx-r1 + └── CentOS + └── pike + ├── Binary + │   ├── EFI + │   ├── images + │   ├── isolinux + │   ├── LiveOS + │   ├── noarch + │   └── x86_64 + ├── downloads + │   ├── integrity + │   └── puppet + └── Source + +******************************* +Create CentOS mirror repository +******************************* + +Outside your Repository Docker container, in another terminal identified +as "**Two**", run the following commands: + +#. From terminal identified as "**Two**", create a *mirror/CentOS* + directory under your *starlingx* workspace directory: + + .. code:: sh + + $ mkdir -p $HOME/starlingx/mirror/CentOS/ + +#. Copy the built CentOS Mirror Repository built under + *$HOME/stx-tools/centos-mirror-tool* to the *$HOME/starlingx/mirror/* + workspace directory: + + .. code:: sh + + $ cp -r $HOME/stx-tools/centos-mirror-tools/output/stx-r1/ $HOME/starlingx/mirror/CentOS/ + + +------------------------- +Create StarlingX packages +------------------------- + +***************************** +Run building Docker container +***************************** + +#. From the terminal identified as "**Two**", create the workspace folder: + + .. code:: sh + + $ mkdir -p $HOME/starlingx/workspace + +#. Navigate to the *$HOME/stx-tools* project directory: + + .. code:: sh + + $ cd $HOME/stx-tools + +#. Verify environment variables: + + .. code:: sh + + $ bash tb.sh env + +#. Run the building Docker container: + + .. code:: sh + + $ bash tb.sh run + +#. Execute the buiding Docker container: + + .. code:: sh + + $ bash tb.sh exec + +********************************* +Download source code repositories +********************************* + +#. From the terminal identified as "**Two**", which is now inside the + Building Docker container, start the internal environment: + + .. code:: sh + + $ eval $(ssh-agent) + $ ssh-add + +#. Use the repo tool to create a local clone of the stx-manifest + Git repository based on the "r/2018.10" branch: + + .. code:: sh + + $ cd $MY_REPO_ROOT_DIR + $ repo init -u https://git.starlingx.io/stx-manifest -m default.xml -b r/2018.10 + + **NOTE:** To use the "repo" command to clone the stx-manifest repository and + check out the "master" branch, omit the "-b r/2018.10" option. + Following is an example: + + .. code:: sh + + $ repo init -u https://git.starlingx.io/stx-manifest -m default.xml + +#. Synchronize the repository: + + .. code:: sh + + $ repo sync -j`nproc` + +#. Create a tarballs repository: + + .. code:: sh + + $ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/ + + Alternatively, you can run the "populate_downloads.sh" script to copy + the tarballs instead of using a symlink: + + .. code:: sh + + $ populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/ + + Outside the container + +#. From another terminal identified as "**Three**", create mirror binaries: + + .. code:: sh + + $ mkdir -p $HOME/starlingx/mirror/CentOS/stx-installer + $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img $HOME/starlingx/mirror/CentOS/stx-installer/initrd.img + $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz $HOME/starlingx/mirror/CentOS/stx-installer/vmlinuz + $ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img $HOME/starlingx/mirror/CentOS/stx-installer/squashfs.img + +************** +Build packages +************** + +#. Go back to the terminal identified as "**Two**", which is the Building Docker container. + +#. **Temporal!** Build-Pkgs Errors. Be prepared to have some missing / + corrupted rpm and tarball packages generated during + `Build the CentOS Mirror Repository`_, which will cause the next step + to fail. If that step does fail, manually download those missing / + corrupted packages. + +#. Update the symbolic links: + + .. code:: sh + + $ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/ + +#. Build the packages: + + .. code:: sh + + $ build-pkgs + +#. **Optional!** Generate-Cgcs-Tis-Repo: + + While this step is optional, it improves performance on subsequent + builds. The cgcs-tis-repo has the dependency information that + sequences the build order. To generate or update the information, you + need to execute the following command after building modified or new + packages. + + .. code:: sh + + $ generate-cgcs-tis-repo + +------------------- +Build StarlingX ISO +------------------- + +#. Build the image: + + .. code:: sh + + $ build-iso + +--------------- +Build installer +--------------- + +To get your StarlingX ISO ready to use, you must create the initialization +files used to boot the ISO, additional controllers, and compute nodes. + +**NOTE:** You only need this procedure during your first build and +every time you upgrade the kernel. + +After running "build-iso", run: + +.. code:: sh + + $ build-pkgs --installer + +This builds *rpm* and *anaconda* packages. Then run: + +.. code:: sh + + $ update-pxe-network-installer + +The *update-pxe-network-installer* covers the steps detailed in +*$MY_REPO/stx/stx-metal/installer/initrd/README*. This script +creates three files on +*/localdisk/loadbuild/pxe-network-installer/output*. + +:: + + new-initrd.img + new-squashfs.img + new-vmlinuz + +Rename the files as follows: + +:: + + initrd.img + squashfs.img + vmlinuz + +Two ways exist for using these files: + +#. Store the files in the */import/mirror/CentOS/stx-installer/* folder + for future use. +#. Store the files in an arbitrary location and modify the + *$MY_REPO/stx/stx-metal/installer/pxe-network-installer/centos/build_srpm.data* + file to point to these files. + +Recreate the *pxe-network-installer* package and rebuild the image: + +.. code:: sh + + $ build-pkgs --clean pxe-network-installer + $ build-pkgs pxe-network-installer + $ build-iso + +Your ISO image should be able to boot. + +**************** +Additional notes +**************** + +- In order to get the first boot working, this complete procedure needs + to be done. However, once the init files are created, these can be + stored in a shared location where different developers can make use + of them. Updating these files is not a frequent task and should be + done whenever the kernel is upgraded. +- StarlingX is in active development. Consequently, it is possible that in the + future the **0.2** version will change to a more generic solution. + +--------------- +Build avoidance +--------------- + +******* +Purpose +******* + +Greatly reduce build times after using "repo" to syncronized a local +repository with an upstream source (i.e. "repo sync"). +Build avoidance works well for designers working +within a regional office. Starting from a new workspace, "build-pkgs" +typically requires three or more hours to complete. Build avoidance +reduces this step to approximately 20 minutes. + +*********** +Limitations +*********** + +- Little or no benefit for designers who refresh a pre-existing + workspace at least daily (e.g. download_mirror.sh, repo sync, + generate-cgcs-centos-repo.sh, build-pkgs, build-iso). In these cases, + an incremental build (i.e. reuse of same workspace without a "build-pkgs + --clean") is often just as efficient. +- Not likely to be useful to solo designers, or teleworkers that wish + to compile on using their home computers. Build avoidance downloads build + artifacts from a reference build, and WAN speeds are generally too + slow. + +***************** +Method (in brief) +***************** + +#. Reference Builds + + - A server in the regional office performs regular (e.g. daily) + automated builds using existing methods. These builds are called + "reference builds". + - The builds are timestamped and preserved for some time (i.e. a + number of weeks). + - A build CONTEXT, which is a file produced by "build-pkgs" + at location *$MY_WORKSPACE/CONTEXT*, is captured. It is a bash script that can + cd to each and every Git and checkout the SHA that contributed to + the build. + - For each package built, a file captures the md5sums of all the + source code inputs required to build that package. These files are + also produced by "build-pkgs" at location + *$MY_WORKSPACE//rpmbuild/SOURCES//srpm_reference.md5*. + - All these build products are accessible locally (e.g. a regional + office) using "rsync". + + **NOTE:** Other protocols can be added later. + +#. Designers + + - Request a build avoidance build. Recommended after you have + done synchronized the repository (i.e. "repo sync"). + + :: + + repo sync + generate-cgcs-centos-repo.sh + populate_downloads.sh + build-pkgs --build-avoidance + + - Use combinations of additional arguments, environment variables, and a + configuration file unique to the regional office to specify an URL + to the reference builds. + + - Using a configuration file to specify the location of your reference build: + + :: + + mkdir -p $MY_REPO/local-build-data + + cat <<- EOF > $MY_REPO/local-build-data/build_avoidance_source + # Optional, these are already the default values. + BUILD_AVOIDANCE_DATE_FORMAT="%Y%m%d" + BUILD_AVOIDANCE_TIME_FORMAT="%H%M%S" + BUILD_AVOIDANCE_DATE_TIME_DELIM="T" + BUILD_AVOIDANCE_DATE_TIME_POSTFIX="Z" + BUILD_AVOIDANCE_DATE_UTC=1 + BUILD_AVOIDANCE_FILE_TRANSFER="rsync" + + # Required, unique values for each regional office + BUILD_AVOIDANCE_USR="jenkins" + BUILD_AVOIDANCE_HOST="stx-builder.mycompany.com" + BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build" + EOF + + - Using command-line arguments to specify the location of your reference + build: + + :: + + build-pkgs --build-avoidance --build-avoidance-dir /localdisk/loadbuild/jenkins/StarlingX_Reference_Build --build-avoidance-host stx-builder.mycompany.com --build-avoidance-user jenkins + + - Prior to your build attempt, you need to accept the host key. + Doing so prevents "rsync" failures on a "yes/no" prompt. + You only have to do this once. + + :: + + grep -q $BUILD_AVOIDANCE_HOST $HOME/.ssh/known_hosts + if [ $? != 0 ]; then + ssh-keyscan $BUILD_AVOIDANCE_HOST >> $HOME/.ssh/known_hosts + fi + + + - "build-pkgs" does the following: + + - From newest to oldest, scans the CONTEXTs of the various + reference builds. Selects the first (i.e. most recent) context that + satisfies the following requirement: every Git the SHA + specifies in the CONTEXT is present. + - The selected context might be slightly out of date, but not by + more than a day. This assumes daily reference builds are run. + - If the context has not been previously downloaded, then + download it now. This means you need to download select portions of the + reference build workspace into the designer's workspace. This + includes all the SRPMS, RPMS, MD5SUMS, and miscellaneous supporting + files. Downloading these files usually takes about 10 minutes + over an office LAN. + - The designer could have additional commits or uncommitted changes + not present in the reference builds. Affected packages are + identified by the differing md5sum's. In these cases, the packages + are re-built. Re-builds usually take five or more minutes, + depending on the packages that have changed. + + - What if no valid reference build is found? Then build-pkgs will fall + back to a regular build. + +**************** +Reference builds +**************** + +- The regional office implements an automated build that pulls the + latest StarlingX software and builds it on a regular basis (e.g. + daily builds). Jenkins, cron, or similar tools can trigger these builds. +- Each build is saved to a unique directory, and preserved for a time + that is reflective of how long a designer might be expected to work + on a private branch without syncronizing with the master branch. + This takes about two weeks. + +- The *MY_WORKSPACE* directory for the build shall have a common root + directory, and a leaf directory that is a sortable time stamp. The + suggested format is *YYYYMMDDThhmmss*. + + .. code:: sh + + $ sudo apt-get update + BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build" + BUILD_TIMESTAMP=$(date -u '+%Y%m%dT%H%M%SZ') + MY_WORKSPACE=${BUILD_AVOIDANCE_DIR}/${BUILD_TIMESTAMP} + +- Designers can access all build products over the internal network of + the regional office. The current prototype employs "rsync". Other + protocols that can efficiently share, copy, or transfer large directories + of content can be added as needed. + +^^^^^^^^^^^^^^ +Advanced usage +^^^^^^^^^^^^^^ + +Can the reference build itself use build avoidance? Yes it can. +Can it reference itself? Yes it can. +In both these cases, caution is advised. To protect against any possible +'divergence from reality', you should limit how many steps you remove +a build avoidance build from a full build. + +Suppose we want to implement a self-referencing daily build in an +environment where a full build already occurs every Saturday. +To protect ourselves from a +build failure on Saturday we also want a limit of seven days since +the last full build. Your build script might look like this ... + +:: + + ... + BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build" + BUILD_AVOIDANCE_HOST="stx-builder.mycompany.com" + FULL_BUILD_DAY="Saturday" + MAX_AGE_DAYS=7 + + LAST_FULL_BUILD_LINK="$BUILD_AVOIDANCE_DIR/latest_full_build" + LAST_FULL_BUILD_DAY="" + NOW_DAY=$(date -u "+%A") + BUILD_TIMESTAMP=$(date -u '+%Y%m%dT%H%M%SZ') + MY_WORKSPACE=${BUILD_AVOIDANCE_DIR}/${BUILD_TIMESTAMP} + + # update software + repo init -u ${BUILD_REPO_URL} -b ${BUILD_BRANCH} + repo sync --force-sync + $MY_REPO_ROOT_DIR/stx-tools/toCOPY/generate-cgcs-centos-repo.sh + $MY_REPO_ROOT_DIR/stx-tools/toCOPY/populate_downloads.sh + + # User can optionally define BUILD_METHOD equal to one of 'FULL', 'AVOIDANCE', or 'AUTO' + # Sanitize BUILD_METHOD + if [ "$BUILD_METHOD" != "FULL" ] && [ "$BUILD_METHOD" != "AVOIDANCE" ]; then + BUILD_METHOD="AUTO" + fi + + # First build test + if [ "$BUILD_METHOD" != "FULL" ] && [ ! -L $LAST_FULL_BUILD_LINK ]; then + echo "latest_full_build symlink missing, forcing full build" + BUILD_METHOD="FULL" + fi + + # Build day test + if [ "$BUILD_METHOD" == "AUTO" ] && [ "$NOW_DAY" == "$FULL_BUILD_DAY" ]; then + echo "Today is $FULL_BUILD_DAY, forcing full build" + BUILD_METHOD="FULL" + fi + + # Build age test + if [ "$BUILD_METHOD" != "FULL" ]; then + LAST_FULL_BUILD_DATE=$(basename $(readlink $LAST_FULL_BUILD_LINK) | cut -d '_' -f 1) + LAST_FULL_BUILD_DAY=$(date -d $LAST_FULL_BUILD_DATE "+%A") + AGE_SECS=$(( $(date "+%s") - $(date -d $LAST_FULL_BUILD_DATE "+%s") )) + AGE_DAYS=$(( $AGE_SECS/60/60/24 )) + if [ $AGE_DAYS -ge $MAX_AGE_DAYS ]; then + echo "Haven't had a full build in $AGE_DAYS days, forcing full build" + BUILD_METHOD="FULL" + fi + BUILD_METHOD="AVOIDANCE" + fi + + #Build it + if [ "$BUILD_METHOD" == "FULL" ]; then + build-pkgs --no-build-avoidance + else + build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER + fi + if [ $? -ne 0 ]; then + echo "Build failed in build-pkgs" + exit 1 + fi + + build-iso + if [ $? -ne 0 ]; then + echo "Build failed in build-iso" + exit 1 + fi + + if [ "$BUILD_METHOD" == "FULL" ]; then + # A successful full build. Set last full build symlink. + if [ -L $LAST_FULL_BUILD_LINK ]; then + rm -rf $LAST_FULL_BUILD_LINK + fi + ln -sf $MY_WORKSPACE $LAST_FULL_BUILD_LINK + fi + ... + +A final note.... + +To use the full build day as your avoidance build reference point, +modify the "build-pkgs" commands above to use "--build-avoidance-day ", +as shown in the following two examples: + +:: + + build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $FULL_BUILD_DAY + + # Here is another example with a bit more shuffling of the above script. + + build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $LAST_FULL_BUILD_DAY + +The advantage is that our build is never more than one step removed +from a full build. This assumes the full build was successful. + +The disadvantage is that by the end of the week, the reference build is getting +rather old. During active weeks, build times could approach build times for +full builds. diff --git a/doc/source/index.rst b/doc/source/index.rst index 93d2445cc..2ff5cdb34 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -2,16 +2,10 @@ StarlingX Documentation ======================= -Welcome to the StarlingX documentation. This is the documentation -for release stx.2018.10. - -Additional information about this release is available in the +Welcome to the StarlingX documentation. This is the documentation for release +stx.2018.10. Additional information about this release is available in the :ref:`release-notes`. -.. Add the additional version info here e.g. - The following documentation versions are available: - StarlingX stx.2019.09 | StarlingX stx.2019.04 - For more information about the project, consult the `Project Specifications `__. diff --git a/doc/source/installation_guide/controller_storage.rst b/doc/source/installation_guide/2018_10/controller_storage.rst similarity index 80% rename from doc/source/installation_guide/controller_storage.rst rename to doc/source/installation_guide/2018_10/controller_storage.rst index 191705488..263e8f27a 100644 --- a/doc/source/installation_guide/controller_storage.rst +++ b/doc/source/installation_guide/2018_10/controller_storage.rst @@ -1,8 +1,6 @@ -.. _controller-storage: - -=================================== -Controller Storage Deployment Guide -=================================== +=============================================== +Controller storage deployment guide stx.2018.10 +=============================================== .. contents:: :local: @@ -15,94 +13,94 @@ For approved instructions, see the `StarlingX Cloud with Controller Storage wiki page `__. ---------------------- -Deployment Description +Deployment description ---------------------- -The Controller Storage deployment option provides a 2x Node High Availability -Controller / Storage Cluster with: +The Controller Storage deployment option provides a 2x node high availability +controller / storage cluster with: -- A pool of up to seven Compute Nodes (pool size limit due to the capacity of - the Storage Function). -- A growth path for Storage to the full Standard solution with an independent - CEPH Storage Cluster. -- High Availability Services runnning across the Controller Nodes in either - Active/Active or Active/Standby mode. -- Storage Function running on top of LVM on single second disk, DRBD-sync'd - between the Controller Nodes. +- A pool of up to seven compute nodes (pool size limit due to the capacity of + the storage function). +- A growth path for storage to the full standard solution with an independent + CEPH storage cluster. +- High availability services runnning across the controller nodes in either + active/active or active/standby mode. +- Storage function running on top of LVM on single second disk, DRBD-sync'd + between the controller nodes. .. figure:: figures/starlingx-deployment-options-controller-storage.png :scale: 50% - :alt: Controller Storage Deployment Configuration + :alt: Controller Storage deployment configuration - *Controller Storage Deployment Configuration* + *Controller Storage deployment configuration* -A Controller Storage deployment provides protection against overall Controller -Node and Compute Node failure: +A Controller Storage deployment provides protection against overall controller +node and compute node failure: -- On overall Controller Node failure, all Controller High Availability Services - go Active on the remaining healthy Controller Node. -- On overall Compute Node failure, Virtual Machines on failed Compute Node are - recovered on the remaining healthy Compute Nodes. +- On overall controller node failure, all controller high availability services + go active on the remaining healthy controller node. +- On overall compute node failure, virtual machines on failed compute node are + recovered on the remaining healthy compute nodes. ------------------------------------ -Preparing Controller Storage Servers +Preparing controller storage servers ------------------------------------ ********** -Bare Metal +Bare metal ********** -Required Servers: +Required servers: - Controllers: 2 - Computes: 2 - 100 ^^^^^^^^^^^^^^^^^^^^^ -Hardware Requirements +Hardware requirements ^^^^^^^^^^^^^^^^^^^^^ The recommended minimum requirements for the physical servers where Controller Storage will be deployed, include: -- Minimum Processor: +- Minimum processor: - - Dual-CPU Intel® Xeon® E5 26xx Family (SandyBridge) 8 cores/socket + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket - Memory: - - 64 GB Controller - - 32 GB Compute + - 64 GB controller + - 32 GB compute - BIOS: - - Hyper-Threading Tech: Enabled - - Virtualization Technology: Enabled - - VT for Directed I/O: Enabled - - CPU Power and Performance Policy: Performance - - CPU C State Control: Disabled - - Plug & Play BMC Detection: Disabled + - Hyper-Threading technology: Enabled + - Virtualization technology: Enabled + - VT for directed I/O: Enabled + - CPU power and performance policy: Performance + - CPU C state control: Disabled + - Plug & play BMC detection: Disabled -- Primary Disk: +- Primary disk: - - 500 GB SDD or NVMe Controller - - 120 GB (min. 10K RPM) Compute + - 500 GB SDD or NVMe controller + - 120 GB (min. 10K RPM) compute -- Additional Disks: +- Additional disks: - - 1 or more 500 GB disks (min. 10K RPM) Compute + - 1 or more 500 GB disks (min. 10K RPM) compute -- Network Ports\* +- Network ports\* - - Management: 10GE Controller, Compute - - OAM: 10GE Controller - - Data: n x 10GE Compute + - Management: 10GE controller, compute + - OAM: 10GE controller + - Data: n x 10GE compute ******************* -Virtual Environment +Virtual environment ******************* Run the libvirt qemu setup scripts. Setting up virtualized OAM and -Management networks: +management networks: :: @@ -123,7 +121,7 @@ are: - controllerstorage-compute-1 ^^^^^^^^^^^^^^^^^^^^^^^^^ -Power Up a Virtual Server +Power up a virtual server ^^^^^^^^^^^^^^^^^^^^^^^^^ To power up a virtual server, run the following command: @@ -139,7 +137,7 @@ e.g. $ sudo virsh start controllerstorage-controller-0 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Access Virtual Server Consoles +Access virtual server consoles ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The XML for virtual servers in stx-tools repo, deployment/libvirt, @@ -151,9 +149,9 @@ domain (the server) and selecting "Open". Access the textual console with the command "virsh console $DOMAIN", where DOMAIN is the name of the server shown in virsh. -When booting the Controller-0 for the first time, both the serial and +When booting the controller-0 for the first time, both the serial and graphical consoles will present the initial configuration menu for the -cluster. One can select serial or graphical console for Controller-0. +cluster. One can select serial or graphical console for controller-0. For the other nodes however only serial is used, regardless of which option is selected. @@ -164,35 +162,35 @@ sequence which follows the boot device selection. One has a few seconds to do this. -------------------------------- -Installing the Controller-0 Host +Installing the controller-0 host -------------------------------- -Installing Controller-0 involves initializing a host with software and +Installing controller-0 involves initializing a host with software and then applying a bootstrap configuration from the command line. The -configured bootstrapped host becomes Controller-0. +configured bootstrapped host becomes controller-0. Procedure: -#. Power on the server that will be Controller-0 with the StarlingX ISO +#. Power on the server that will be controller-0 with the StarlingX ISO on a USB in a bootable USB slot. #. Configure the controller using the config_controller script. ************************* -Initializing Controller-0 +Initializing controller-0 ************************* -This section describes how to initialize StarlingX in host Controller-0. +This section describes how to initialize StarlingX in host controller-0. Except where noted, all the commands must be executed from a console of the host. -Power on the host to be configured as Controller-0, with the StarlingX +Power on the host to be configured as controller-0, with the StarlingX ISO on a USB in a bootable USB slot. Wait for the console to show the StarlingX ISO booting options: - **Standard Controller Configuration** - When the installer is loaded and the installer welcome screen - appears in the Controller-0 host, select the type of installation + appears in the controller-0 host, select the type of installation "Standard Controller Configuration". - **Graphical Console** @@ -202,13 +200,13 @@ StarlingX ISO booting options: - **Standard Security Boot Profile** - - Select "Standard Security Boot Profile" as the Security Profile. + - Select "Standard Security Boot Profile" as the security profile. Monitor the initialization. When it is complete, a reboot is initiated -on the Controller-0 host, briefly displays a GNU GRUB screen, and then +on the controller-0 host, briefly displays a GNU GRUB screen, and then boots automatically into the StarlingX image. -Log into Controller-0 as user wrsroot, with password wrsroot. The +Log into controller-0 as user wrsroot, with password wrsroot. The first time you log in as wrsroot, you are required to change your password. Enter the current password (wrsroot): @@ -229,28 +227,27 @@ Enter the new password again to confirm it: Retype new password: -Controller-0 is initialized with StarlingX, and is ready for -configuration. +controller-0 is initialized with StarlingX, and is ready for configuration. ************************ -Configuring Controller-0 +Configuring controller-0 ************************ -This section describes how to perform the Controller-0 configuration +This section describes how to perform the controller-0 configuration interactively just to bootstrap system with minimum critical data. Except where noted, all the commands must be executed from the console -of the active controller (here assumed to be Controller-0). +of the active controller (here assumed to be controller-0). When run interactively, the config_controller script presents a series of prompts for initial configuration of StarlingX: -- For the Virtual Environment, you can accept all the default values +- For the virtual environment, you can accept all the default values immediately after ‘system date and time’. -- For a Physical Deployment, answer the bootstrap configuration +- For a physical deployment, answer the bootstrap configuration questions with answers applicable to your particular physical setup. The script is used to configure the first controller in the StarlingX -cluster as Controller-0. The prompts are grouped by configuration +cluster as controller-0. The prompts are grouped by configuration area. To start the script interactively, use the following command with no parameters: @@ -283,21 +280,21 @@ Accept all the default values immediately after ‘system date and time’. Please complete any out of service commissioning steps with system commands and unlock controller to proceed. After config_controller bootstrap configuration, REST API, CLI and -Horizon interfaces are enabled on the Controller-0 OAM IP Address. The +Horizon interfaces are enabled on the controller-0 OAM IP address. The remaining installation instructions will use the CLI. ------------------------------------ -Provisioning Controller-0 and System +Provisioning controller-0 and system ------------------------------------ -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc ********************************************* -Configuring Provider Networks at Installation +Configuring provider networks at installation ********************************************* You must set up provider networks at installation so that you can attach @@ -311,11 +308,11 @@ Set up one provider network of the vlan type, named providernet-a: [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-range-create --name providernet-a-range1 --range 100-400 providernet-a ************************************* -Configuring Cinder on Controller Disk +Configuring Cinder on controller disk ************************************* Review the available disk space and capacity and obtain the uuid of the -physical disk +physical disk: :: @@ -328,7 +325,7 @@ physical disk | 89694799-0dd8-4532-8636-c0d8aabfe215 | /dev/sdb | 2064 | HDD | 200.0 | 199.997 | |... +--------------------------------------+-----------+---------+---------+-------+------------+--------------+... -Create the 'cinder-volumes' local volume group +Create the 'cinder-volumes' local volume group: :: @@ -353,7 +350,7 @@ Create the 'cinder-volumes' local volume group | parameters | {u'lvm_type': u'thin'} | +-----------------+--------------------------------------+ -Create a disk partition to add to the volume group +Create a disk partition to add to the volume group: :: @@ -377,7 +374,7 @@ Create a disk partition to add to the volume group | updated_at | None | +-------------+--------------------------------------------------+ -Wait for the new partition to be created (i.e. status=Ready) +Wait for the new partition to be created (i.e. status=Ready): :: @@ -391,7 +388,7 @@ Wait for the new partition to be created (i.e. status=Ready) | |...| |...| | | | +--------------------------------------+...+------------+...+---------------------+----------+--------+ -Add the partition to the volume group +Add the partition to the volume group: :: @@ -416,14 +413,14 @@ Add the partition to the volume group | updated_at | None | +--------------------------+--------------------------------------------------+ -Enable LVM Backend. +Enable LVM backend: :: [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add lvm -s cinder --confirmed Wait for the storage backend to leave "configuring" state. Confirm LVM -Backend storage is configured: +backend storage is configured: :: @@ -436,11 +433,11 @@ Backend storage is configured: +--------------------------------------+------------+---------+------------+------+----------+... ********************** -Unlocking Controller-0 +Unlocking controller-0 ********************** -You must unlock Controller-0 so that you can use it to install the -remaining hosts. On Controller-0, acquire Keystone administrative +You must unlock controller-0 so that you can use it to install the +remaining hosts. On controller-0, acquire Keystone administrative privileges. Use the system host-unlock command: :: @@ -449,13 +446,13 @@ privileges. Use the system host-unlock command: The host is rebooted. During the reboot, the command line is unavailable, and any ssh connections are dropped. To monitor the -progress of the reboot, use the Controller-0 console. +progress of the reboot, use the controller-0 console. **************************************** -Verifying the Controller-0 Configuration +Verifying the controller-0 configuration **************************************** -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: @@ -475,7 +472,7 @@ Verify that the StarlingX controller services are running: ... +-----+-------------------------------+--------------+----------------+ -Verify that Controller-0 is unlocked, enabled, and available: +Verify that controller-0 is unlocked, enabled, and available: :: @@ -487,7 +484,7 @@ Verify that Controller-0 is unlocked, enabled, and available: +----+--------------+-------------+----------------+-------------+--------------+ --------------------------------------- -Installing Controller-1 / Compute Hosts +Installing controller-1 / compute hosts --------------------------------------- After initializing and configuring an active controller, you can add and @@ -495,7 +492,7 @@ configure a backup controller and additional compute hosts. For each host do the following: ***************** -Initializing Host +Initializing host ***************** Power on Host. In host console you will see: @@ -508,16 +505,16 @@ Power on Host. In host console you will see: controller node in order to proceed. *************************************** -Updating Host Host Name and Personality +Updating host hostname and personality *************************************** -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc -Wait for Controller-0 to discover new host, list the host until new +Wait for controller-0 to discover new host, list the host until new UNKNOWN host shows up in table: :: @@ -542,22 +539,22 @@ Or for compute-0: [wrsroot@controller-0 ~(keystone_admin)]$ system host-update 3 personality=compute hostname=compute-0 -See also: 'system help host-update' +See also: 'system help host-update'. -Unless it is known that the host's configuration can support the -installation of more than one node, it is recommended that the -installation and configuration of each node be serialized. For example, -if the entire cluster has its virtual disks hosted on the host's root -disk which happens to be a single rotational type hard disk, then the -host cannot (reliably) support parallel node installation. +Unless it is known that the host's configuration can support the installation of +more than one node, it is recommended that the installation and configuration of +each node be serialized. For example, if the entire cluster has its virtual +disks hosted on the host's root disk which happens to be a single rotational +type hard disk, then the host cannot (reliably) support parallel node +installation. *************** -Monitoring Host +Monitoring host *************** -On Controller-0, you can monitor the installation progress by running -the system host-show command for the host periodically. Progress is -shown in the install_state field. +On controller-0, you can monitor the installation progress by running the system +host-show command for the host periodically. Progress is shown in the +install_state field: :: @@ -566,16 +563,16 @@ shown in the install_state field. | install_state | booting | | install_state_info | None | -Wait while the host is configured and rebooted. Up to 20 minutes may be -required for a reboot, depending on hardware. When the reboot is -complete, the host is reported as Locked, Disabled, and Online. +Wait while the host is configured and rebooted. Up to 20 minutes may be required +for a reboot, depending on hardware. When the reboot is complete, the host is +reported as locked, disabled, and online. ************* -Listing Hosts +Listing hosts ************* -Once all Nodes have been installed, configured and rebooted, on -Controller-0 list the hosts: +Once all nodes have been installed, configured and rebooted, on controller-0 +list the hosts: :: @@ -590,10 +587,10 @@ Controller-0 list the hosts: +----+--------------+-------------+----------------+-------------+--------------+ ------------------------- -Provisioning Controller-1 +Provisioning controller-1 ------------------------- -On Controller-0, list hosts +On controller-0, list hosts: :: @@ -607,28 +604,28 @@ On Controller-0, list hosts +----+--------------+-------------+----------------+-------------+--------------+ *********************************************** -Provisioning Network Interfaces on Controller-1 +Provisioning network interfaces on controller-1 *********************************************** -In order to list out hardware port names, types, pci-addresses that have +In order to list out hardware port names, types, PCI addresses that have been discovered: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list controller-1 -Provision the oam interface for Controller-1: +Provision the OAM interface for controller-1: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -n -c platform --networks oam controller-1 ************************************ -Provisioning Storage on Controller-1 +Provisioning storage on controller-1 ************************************ Review the available disk space and capacity and obtain the uuid of the -physical disk +physical disk: :: @@ -641,7 +638,7 @@ physical disk | 70b83394-968e-4f0d-8a99-7985cd282a21 | /dev/sdb | 2064 | HDD | 200.0 | 199.997 | +--------------------------------------+-----------+---------+---------+-------+------------+ -Assign Cinder storage to the physical disk +Assign Cinder storage to the physical disk: :: @@ -667,7 +664,7 @@ Assign Cinder storage to the physical disk +-----------------+--------------------------------------+ Create a disk partition to add to the volume group based on uuid of the -physical disk +physical disk: :: @@ -691,7 +688,7 @@ physical disk | updated_at | None | +-------------+--------------------------------------------------+ -Wait for the new partition to be created (i.e. status=Ready) +Wait for the new partition to be created (i.e. status=Ready): :: @@ -705,7 +702,7 @@ Wait for the new partition to be created (i.e. status=Ready) | |...| | ... | | | +--------------------------------------+...+------------+...+--------+----------------------+ -Add the partition to the volume group +Add the partition to the volume group: :: @@ -731,25 +728,24 @@ Add the partition to the volume group +--------------------------+--------------------------------------------------+ ********************** -Unlocking Controller-1 +Unlocking controller-1 ********************** -Unlock Controller-1 +Unlock controller-1: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-1 -Wait while the Controller-1 is rebooted. Up to 10 minutes may be -required for a reboot, depending on hardware. +Wait while the controller-1 is rebooted. Up to 10 minutes may be required for a +reboot, depending on hardware. -**REMARK:** Controller-1 will remain in 'degraded' state until -data-syncing is complete. The duration is dependant on the -virtualization host's configuration - i.e., the number and configuration -of physical disks used to host the nodes' virtual disks. Also, the -management network is expected to have link capacity of 10000 (1000 is -not supported due to excessive data-sync time). Use 'fm alarm-list' to -confirm status. +**REMARK:** controller-1 will remain in 'degraded' state until data-syncing is +complete. The duration is dependant on the virtualization host's configuration - +i.e., the number and configuration of physical disks used to host the nodes' +virtual disks. Also, the management network is expected to have link capacity of +10000 (1000 is not supported due to excessive data-sync time). Use +'fm alarm-list' to confirm status. :: @@ -762,26 +758,26 @@ confirm status. ... --------------------------- -Provisioning a Compute Host +Provisioning a compute host --------------------------- You must configure the network interfaces and the storage disks on a -host before you can unlock it. For each Compute Host do the following: +host before you can unlock it. For each compute host do the following: -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc ************************************************* -Provisioning Network Interfaces on a Compute Host +Provisioning network interfaces on a compute host ************************************************* -On Controller-0, in order to list out hardware port names, types, +On controller-0, in order to list out hardware port names, types, pci-addresses that have been discovered: -- **Only in Virtual Environment**: Ensure that the interface used is +- **Only in virtual environment**: Ensure that the interface used is one of those attached to host bridge with model type "virtio" (i.e., eth1000 and eth1001). The model type "e1000" emulated devices will not work for provider networks: @@ -790,21 +786,21 @@ pci-addresses that have been discovered: [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list compute-0 -Provision the data interface for Compute: +Provision the data interface for compute: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -p providernet-a -c data compute-0 eth1000 *************************** -VSwitch Virtual Environment +VSwitch virtual environment *************************** -**Only in Virtual Environment**. If the compute has more than 4 cpus, -the system will auto-configure the vswitch to use 2 cores. However some -virtual environments do not properly support multi-queue required in a -multi-cpu environment. Therefore run the following command to reduce the -vswitch cores to 1: +**Only in virtual environment**. If the compute has more than 4 cpus, the system +will auto-configure the vswitch to use 2 cores. However some virtual +environments do not properly support multi-queue required in a multi-CPU +environment. Therefore run the following command to reduce the vswitch cores to +1: :: @@ -820,7 +816,7 @@ vswitch cores to 1: +--------------------------------------+-------+-----------+-------+--------+... ************************************** -Provisioning Storage on a Compute Host +Provisioning storage on a compute host ************************************** Review the available disk space and capacity and obtain the uuid(s) of @@ -915,31 +911,30 @@ nova-local: +-----------------+-------------------------------------------------------------------+ ************************ -Unlocking a Compute Host +Unlocking a compute host ************************ -On Controller-0, use the system host-unlock command to unlock the -Compute node: +On controller-0, use the system host-unlock command to unlock the compute node: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock compute-0 -Wait while the Compute node is rebooted. Up to 10 minutes may be +Wait while the compute node is rebooted. Up to 10 minutes may be required for a reboot, depending on hardware. The host is rebooted, and -its Availability State is reported as In-Test, followed by +its availability state is reported as in-test, followed by unlocked/enabled. ------------------- -System Health Check +System health check ------------------- *********************** -Listing StarlingX Nodes +Listing StarlingX nodes *********************** -On Controller-0, after a few minutes, all nodes shall be reported as -Unlocked, Enabled, and Available: +On controller-0, after a few minutes, all nodes shall be reported as +unlocked, enabled, and available: :: @@ -954,18 +949,20 @@ Unlocked, Enabled, and Available: +----+--------------+-------------+----------------+-------------+--------------+ ***************** -System Alarm List +System alarm-list ***************** -When all nodes are Unlocked, Enabled and Available: check 'fm alarm-list' for issues. +When all nodes are unlocked, enabled and available: check 'fm alarm-list' for +issues. -Your StarlingX deployment is now up and running with 2x HA Controllers with Cinder -Storage, 2x Computes and all OpenStack services up and running. You can now proceed -with standard OpenStack APIs, CLIs and/or Horizon to load Glance Images, configure -Nova Flavors, configure Neutron networks and launch Nova Virtual Machines. +Your StarlingX deployment is now up and running with 2x HA controllers with +Cinder storage, 2x computes, and all OpenStack services up and running. You can +now proceed with standard OpenStack APIs, CLIs and/or Horizon to load Glance +images, configure Nova Flavors, configure Neutron networks and launch Nova +virtual machines. ---------------------- -Deployment Terminology +Deployment terminology ---------------------- .. include:: deployment_terminology.rst diff --git a/doc/source/installation_guide/dedicated_storage.rst b/doc/source/installation_guide/2018_10/dedicated_storage.rst similarity index 81% rename from doc/source/installation_guide/dedicated_storage.rst rename to doc/source/installation_guide/2018_10/dedicated_storage.rst index c716dc455..73c2827e7 100644 --- a/doc/source/installation_guide/dedicated_storage.rst +++ b/doc/source/installation_guide/2018_10/dedicated_storage.rst @@ -1,8 +1,6 @@ -.. _dedicated-storage: - -================================== -Dedicated Storage Deployment Guide -================================== +============================================== +Dedicated storage deployment guide stx.2018.10 +============================================== .. contents:: :local: @@ -15,11 +13,11 @@ For approved instructions, see the `StarlingX Cloud with Dedicated Storage wiki page `__. ---------------------- -Deployment Description +Deployment description ---------------------- Cloud with Dedicated Storage is the standard StarlingX deployment option with -independent Controller, Compute, and Storage Nodes. +independent controller, compute, and storage nodes. This deployment option provides the maximum capacity for a single region deployment, with a supported growth path to a multi-region deployment option by @@ -27,31 +25,31 @@ adding a secondary region. .. figure:: figures/starlingx-deployment-options-dedicated-storage.png :scale: 50% - :alt: Dedicated Storage Deployment Configuration + :alt: Dedicated Storage deployment configuration - *Dedicated Storage Deployment Configuration* + *Dedicated Storage deployment configuration* Cloud with Dedicated Storage includes: -- 2x Node HA Controller Cluster with HA Services running across the Controller - Nodes in either Active/Active or Active/Standby mode. -- Pool of up to 100 Compute Nodes for hosting virtual machines and virtual +- 2x node HA controller cluster with HA services running across the controller + nodes in either active/active or active/standby mode. +- Pool of up to 100 compute nodes for hosting virtual machines and virtual networks. -- 2-9x Node HA CEPH Storage Cluster for hosting virtual volumes, images, and +- 2-9x node HA CEPH storage cluster for hosting virtual volumes, images, and object storage that supports a replication factor of 2 or 3. - Storage Nodes are deployed in replication groups of 2 or 3. Replication + Storage nodes are deployed in replication groups of 2 or 3. Replication of objects is done strictly within the replication group. - Supports up to 4 groups of 2x Storage Nodes, or up to 3 groups of 3x Storage - Nodes. + Supports up to 4 groups of 2x storage nodes, or up to 3 groups of 3x storage + nodes. ----------------------------------- -Preparing Dedicated Storage Servers +Preparing dedicated storage servers ----------------------------------- ********** -Bare Metal +Bare metal ********** Required Servers: @@ -65,51 +63,51 @@ Required Servers: - Computes: 2 - 100 ^^^^^^^^^^^^^^^^^^^^^ -Hardware Requirements +Hardware requirements ^^^^^^^^^^^^^^^^^^^^^ The recommended minimum requirements for the physical servers where Dedicated Storage will be deployed, include: -- Minimum Processor: +- Minimum processor: - - Dual-CPU Intel® Xeon® E5 26xx Family (SandyBridge) 8 cores/socket + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket - Memory: - - 64 GB Controller, Storage - - 32 GB Compute + - 64 GB controller, storage + - 32 GB compute - BIOS: - - Hyper-Threading Tech: Enabled - - Virtualization Technology: Enabled - - VT for Directed I/O: Enabled - - CPU Power and Performance Policy: Performance - - CPU C State Control: Disabled - - Plug & Play BMC Detection: Disabled + - Hyper-Threading technology: Enabled + - Virtualization technology: Enabled + - VT for directed I/O: Enabled + - CPU power and performance policy: Performance + - CPU C state control: Disabled + - Plug & play BMC detection: Disabled -- Primary Disk: +- Primary disk: - - 500 GB SDD or NVMe Controller - - 120 GB (min. 10K RPM) Compute, Storage + - 500 GB SDD or NVMe controller + - 120 GB (min. 10K RPM) compute and storage -- Additional Disks: +- Additional disks: - - 1 or more 500 GB disks (min. 10K RPM) Storage, Compute + - 1 or more 500 GB disks (min. 10K RPM) storage, compute -- Network Ports\* +- Network ports\* - - Management: 10GE Controller, Storage, Compute - - OAM: 10GE Controller - - Data: n x 10GE Compute + - Management: 10GE controller, storage, compute + - OAM: 10GE controller + - Data: n x 10GE compute ******************* -Virtual Environment +Virtual environment ******************* Run the libvirt qemu setup scripts. Setting up virtualized OAM and -Management networks: +management networks: :: @@ -132,7 +130,7 @@ are: - dedicatedstorage-storage-1 ^^^^^^^^^^^^^^^^^^^^^^^^^ -Power Up a Virtual Server +Power up a virtual server ^^^^^^^^^^^^^^^^^^^^^^^^^ To power up a virtual server, run the following command: @@ -148,7 +146,7 @@ e.g. $ sudo virsh start dedicatedstorage-controller-0 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Access Virtual Server Consoles +Access virtual server consoles ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The XML for virtual servers in stx-tools repo, deployment/libvirt, @@ -173,12 +171,12 @@ sequence which follows the boot device selection. One has a few seconds to do this. -------------------------------- -Installing the Controller-0 Host +Installing the controller-0 host -------------------------------- Installing controller-0 involves initializing a host with software and then applying a bootstrap configuration from the command line. The -configured bootstrapped host becomes Controller-0. +configured bootstrapped host becomes controller-0. Procedure: @@ -187,21 +185,21 @@ Procedure: #. Configure the controller using the config_controller script. ************************* -Initializing Controller-0 +Initializing controller-0 ************************* -This section describes how to initialize StarlingX in host Controller-0. +This section describes how to initialize StarlingX in host controller-0. Except where noted, all the commands must be executed from a console of the host. -Power on the host to be configured as Controller-0, with the StarlingX +Power on the host to be configured as controller-0, with the StarlingX ISO on a USB in a bootable USB slot. Wait for the console to show the StarlingX ISO booting options: - **Standard Controller Configuration** - When the installer is loaded and the installer welcome screen - appears in the Controller-0 host, select the type of installation + appears in the controller-0 host, select the type of installation "Standard Controller Configuration". - **Graphical Console** @@ -211,13 +209,13 @@ StarlingX ISO booting options: - **Standard Security Boot Profile** - - Select "Standard Security Boot Profile" as the Security Profile. + - Select "Standard Security Boot Profile" as the security profile. Monitor the initialization. When it is complete, a reboot is initiated -on the Controller-0 host, briefly displays a GNU GRUB screen, and then +on the controller-0 host, briefly displays a GNU GRUB screen, and then boots automatically into the StarlingX image. -Log into Controller-0 as user wrsroot, with password wrsroot. The +Log into controller-0 as user wrsroot, with password wrsroot. The first time you log in as wrsroot, you are required to change your password. Enter the current password (wrsroot): @@ -238,14 +236,13 @@ Enter the new password again to confirm it: Retype new password: -Controller-0 is initialized with StarlingX, and is ready for -configuration. +controller-0 is initialized with StarlingX, and is ready for configuration. ************************ -Configuring Controller-0 +Configuring controller-0 ************************ -This section describes how to perform the Controller-0 configuration +This section describes how to perform the controller-0 configuration interactively just to bootstrap system with minimum critical data. Except where noted, all the commands must be executed from the console of the active controller (here assumed to be controller-0). @@ -253,9 +250,9 @@ of the active controller (here assumed to be controller-0). When run interactively, the config_controller script presents a series of prompts for initial configuration of StarlingX: -- For the Virtual Environment, you can accept all the default values +- For the virtual environment, you can accept all the default values immediately after ‘system date and time’. -- For a Physical Deployment, answer the bootstrap configuration +- For a physical deployment, answer the bootstrap configuration questions with answers applicable to your particular physical setup. The script is used to configure the first controller in the StarlingX @@ -271,7 +268,7 @@ with no parameters: Enter ! at any prompt to abort... ... -Accept all the default values immediately after ‘system date and time’ +Accept all the default values immediately after ‘system date and time’: :: @@ -292,21 +289,21 @@ Accept all the default values immediately after ‘system date and time’ Please complete any out of service commissioning steps with system commands and unlock controller to proceed. After config_controller bootstrap configuration, REST API, CLI and -Horizon interfaces are enabled on the controller-0 OAM IP Address. The +Horizon interfaces are enabled on the controller-0 OAM IP address. The remaining installation instructions will use the CLI. ------------------------------------ -Provisioning Controller-0 and System +Provisioning controller-0 and system ------------------------------------ -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc ********************************************* -Configuring Provider Networks at Installation +Configuring provider networks at installation ********************************************* You must set up provider networks at installation so that you can attach @@ -320,7 +317,7 @@ Set up one provider network of the vlan type, named providernet-a: [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-range-create --name providernet-a-range1 --range 100-400 providernet-a ********************************************* -Adding a Ceph Storage Backend at Installation +Adding a Ceph storage backend at installation ********************************************* Add CEPH Storage backend: @@ -353,7 +350,7 @@ Add CEPH Storage backend: | 55f49f86-3e01-4d03-a014-42e1b55ba487 | file-store | file | configured | None | glance |... +--------------------------------------+------------+---------+-------------+--------------------+----------+... -Confirm CEPH storage is configured +Confirm CEPH storage is configured: :: @@ -370,25 +367,25 @@ Confirm CEPH storage is configured +--------------------------------------+------------+---------+------------+-------------------+-----------+... ********************** -Unlocking Controller-0 +Unlocking controller-0 ********************** -You must unlock controller-0 so that you can use it to install the -remaining hosts. Use the system host-unlock command: +You must unlock controller-0 so that you can use it to install the remaining +hosts. Use the system host-unlock command: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-0 -The host is rebooted. During the reboot, the command line is -unavailable, and any ssh connections are dropped. To monitor the -progress of the reboot, use the controller-0 console. +The host is rebooted. During the reboot, the command line is unavailable, and +any ssh connections are dropped. To monitor the progress of the reboot, use the +controller-0 console. **************************************** -Verifying the Controller-0 Configuration +Verifying the controller-0 configuration **************************************** -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: @@ -420,10 +417,10 @@ Verify that controller-0 is unlocked, enabled, and available: +----+--------------+-------------+----------------+-------------+--------------+ ******************************* -Provisioning Filesystem Storage +Provisioning filesystem storage ******************************* -List the controller filesystems with status and current sizes +List the controller file systems with status and current sizes: :: @@ -449,7 +446,7 @@ Modify filesystem sizes [wrsroot@controller-0 ~(keystone_admin)]$ system controllerfs-modify backup=42 database=12 img-conversions=12 ------------------------------------------------------- -Installing Controller-1 / Storage Hosts / Compute Hosts +Installing controller-1 / storage hosts / compute hosts ------------------------------------------------------- After initializing and configuring an active controller, you can add and @@ -457,7 +454,7 @@ configure a backup controller and additional compute or storage hosts. For each host do the following: ***************** -Initializing Host +Initializing host ***************** Power on Host. In host console you will see: @@ -470,16 +467,16 @@ Power on Host. In host console you will see: controller node in order to proceed. ********************************** -Updating Host Name and Personality +Updating host name and personality ********************************** -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc -Wait for Controller-0 to discover new host, list the host until new +Wait for controller-0 to discover new host, list the host until new UNKNOWN host shows up in table: :: @@ -498,19 +495,19 @@ Use the system host-add to update host personality attribute: [wrsroot@controller-0 ~(keystone_admin)]$ system host-add -n -p -m -**REMARK:** use the Mac Address for the specific network interface you -are going to be connected. e.g. OAM network interface for "Controller-1" -node, Management network interface for "Computes" and "Storage" nodes. +**REMARK:** use the Mac address for the specific network interface you +are going to be connected. e.g. OAM network interface for controller-1 +node, management network interface for compute and storage nodes. -Check the **NIC** MAC Address from "Virtual Manager GUI" under *"Show +Check the **NIC** MAC address from "Virtual Manager GUI" under *"Show virtual hardware details -*\ **i**\ *" Main Banner --> NIC: --> specific -"Bridge name:" under MAC Address text field.* +"Bridge name:" under MAC address text field.* *************** -Monitoring Host +Monitoring host *************** -On Controller-0, you can monitor the installation progress by running +On controller-0, you can monitor the installation progress by running the system host-show command for the host periodically. Progress is shown in the install_state field. @@ -524,14 +521,14 @@ shown in the install_state field. Wait while the host is configured and rebooted. Up to 20 minutes may be required for a reboot, depending on hardware. When the reboot is -complete, the host is reported as Locked, Disabled, and Online. +complete, the host is reported as locked, disabled, and online. ************* -Listing Hosts +Listing hosts ************* -Once all Nodes have been installed, configured and rebooted, on -Controller-0 list the hosts: +Once all nodes have been installed, configured and rebooted, on +controller-0 list the hosts: :: @@ -548,10 +545,10 @@ Controller-0 list the hosts: +----+--------------+-------------+----------------+-------------+--------------+ ------------------------- -Provisioning Controller-1 +Provisioning controller-1 ------------------------- -On Controller-0, list hosts +On controller-0, list hosts: :: @@ -565,36 +562,36 @@ On Controller-0, list hosts +----+--------------+-------------+----------------+-------------+--------------+ *********************************************** -Provisioning Network Interfaces on Controller-1 +Provisioning network interfaces on controller-1 *********************************************** -In order to list out hardware port names, types, pci-addresses that have +In order to list out hardware port names, types, PCI addresses that have been discovered: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list controller-1 -Provision the oam interface for Controller-1: +Provision the OAM interface for controller-1: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -n -c platform --networks oam controller-1 ********************** -Unlocking Controller-1 +Unlocking controller-1 ********************** -Unlock Controller-1 +Unlock controller-1: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-1 -Wait while the Controller-1 is rebooted. Up to 10 minutes may be +Wait while the controller-1 is rebooted. Up to 10 minutes may be required for a reboot, depending on hardware. -**REMARK:** Controller-1 will remain in 'degraded' state until +**REMARK:** controller-1 will remain in degraded state until data-syncing is complete. The duration is dependant on the virtualization host's configuration - i.e., the number and configuration of physical disks used to host the nodes' virtual disks. Also, the @@ -613,14 +610,14 @@ confirm status. ... ------------------------- -Provisioning Storage Host +Provisioning storage host ------------------------- ************************************** -Provisioning Storage on a Storage Host +Provisioning storage on a storage host ************************************** -Available physical disks in Storage-N +Available physical disks in storage-N: :: @@ -640,7 +637,7 @@ Available physical disks in Storage-N | | | | | | | |... +--------------------------------------+-----------+---------+---------+-------+------------+--------------+... -Available storage tiers in Storage-N +Available storage tiers in storage-N: :: @@ -651,9 +648,9 @@ Available storage tiers in Storage-N | 4398d910-75e4-4e99-a57f-fc147fb87bdb | storage | in-use | 5131a848-25ea-4cd8-bbce-0d65c84183df | +--------------------------------------+---------+--------+--------------------------------------+ -Create a storage function (i.e. OSD) in Storage-N. At least two unlocked and -enabled hosts with monitors are required. Candidates are: Controller-0, -Controller-1, and Storage-0. +Create a storage function (i.e. OSD) in storage-N. At least two unlocked and +enabled hosts with monitors are required. Candidates are: controller-0, +controller-1, and storage-0. :: @@ -676,7 +673,7 @@ Controller-1, and Storage-0. | updated_at | 2018-08-16T00:40:07.626762+00:00 | +------------------+--------------------------------------------------+ -Create remaining available storage function (an OSD) in Storage-N +Create remaining available storage function (an OSD) in storage-N based in the number of available physical disks. List the OSDs: @@ -690,7 +687,7 @@ List the OSDs: | 34989bad-67fc-49ea-9e9c-38ca4be95fad | osd | 0 | {} | c7cc08e6-ff18-4229-a79d-a04187de7b8d | +--------------------------------------+----------+-------+--------------+--------------------------------------+ -Unlock Storage-N +Unlock storage-N: :: @@ -700,26 +697,26 @@ Unlock Storage-N remaining storage nodes. --------------------------- -Provisioning a Compute Host +Provisioning a compute host --------------------------- You must configure the network interfaces and the storage disks on a -host before you can unlock it. For each Compute Host do the following: +host before you can unlock it. For each compute host do the following: -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc ************************************************* -Provisioning Network Interfaces on a Compute Host +Provisioning network interfaces on a compute host ************************************************* -On Controller-0, in order to list out hardware port names, types, +On controller-0, in order to list out hardware port names, types, pci-addresses that have been discovered: -- **Only in Virtual Environment**: Ensure that the interface used is +- **Only in virtual environment**: Ensure that the interface used is one of those attached to host bridge with model type "virtio" (i.e., eth1000 and eth1001). The model type "e1000" emulated devices will not work for provider networks. @@ -728,20 +725,20 @@ pci-addresses that have been discovered: [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list compute-0 -Provision the data interface for Compute: +Provision the data interface for compute: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -p providernet-a -c data compute-0 eth1000 *************************** -VSwitch Virtual Environment +VSwitch virtual environment *************************** -**Only in Virtual Environment**. If the compute has more than 4 cpus, +**Only in virtual environment**. If the compute has more than 4 CPUs, the system will auto-configure the vswitch to use 2 cores. However some virtual environments do not properly support multi-queue required in a -multi-cpu environment. Therefore run the following command to reduce the +multi-CPU environment. Therefore run the following command to reduce the vswitch cores to 1: :: @@ -758,7 +755,7 @@ vswitch cores to 1: +--------------------------------------+-------+-----------+-------+--------+... ************************************** -Provisioning Storage on a Compute Host +Provisioning storage on a compute host ************************************** Review the available disk space and capacity and obtain the uuid(s) of @@ -834,30 +831,30 @@ volumes: [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-modify -b remote compute-0 nova-local ************************ -Unlocking a Compute Host +Unlocking a compute host ************************ -On Controller-0, use the system host-unlock command to unlock the -Compute-N: +On controller-0, use the system host-unlock command to unlock the +compute-N: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock compute-0 -Wait while the Compute-N is rebooted. Up to 10 minutes may be required +Wait while the compute-N is rebooted. Up to 10 minutes may be required for a reboot, depending on hardware. The host is rebooted, and its -Availability State is reported as In-Test, followed by unlocked/enabled. +availability state is reported as in-test, followed by unlocked/enabled. ------------------- -System Health Check +System health check ------------------- *********************** -Listing StarlingX Nodes +Listing StarlingX nodes *********************** -On Controller-0, after a few minutes, all nodes shall be reported as -Unlocked, Enabled, and Available: +On controller-0, after a few minutes, all nodes shall be reported as +unlocked, enabled, and available: :: @@ -874,7 +871,7 @@ Unlocked, Enabled, and Available: +----+--------------+-------------+----------------+-------------+--------------+ ****************************** -Checking StarlingX CEPH Health +Checking StarlingX CEPH health ****************************** :: @@ -892,18 +889,20 @@ Checking StarlingX CEPH Health controller-0:~$ ***************** -System Alarm List +System alarm list ***************** -When all nodes are Unlocked, Enabled and Available: check 'fm alarm-list' for issues. +When all nodes are unlocked, enabled and available: check 'fm alarm-list' for +issues. -Your StarlingX deployment is now up and running with 2x HA Controllers with Cinder -Storage, 1x Compute, 3x Storages and all OpenStack services up and running. You can -now proceed with standard OpenStack APIs, CLIs and/or Horizon to load Glance Images, -configure Nova Flavors, configure Neutron networks and launch Nova Virtual Machines. +Your StarlingX deployment is now up and running with 2x HA controllers with +Cinder storage, 1x compute, 3x storages and all OpenStack services up and +running. You can now proceed with standard OpenStack APIs, CLIs and/or Horizon +to load Glance images, configure Nova Flavors, configure Neutron networks and +launch Nova virtual machines. ---------------------- -Deployment Terminology +Deployment terminology ---------------------- .. include:: deployment_terminology.rst diff --git a/doc/source/installation_guide/2018_10/deployment_terminology.rst b/doc/source/installation_guide/2018_10/deployment_terminology.rst new file mode 100644 index 000000000..a88b4695f --- /dev/null +++ b/doc/source/installation_guide/2018_10/deployment_terminology.rst @@ -0,0 +1,119 @@ +.. _incl-simplex-deployment-terminology: + +**All-in-one controller node** + A single physical node that provides a controller function, compute + function, and storage function. + +.. _incl-simplex-deployment-terminology-end: + + +.. _incl-standard-controller-deployment-terminology: + +**Controller node / function** + A node that runs cloud control function for managing cloud resources. + + - Runs cloud control functions for managing cloud resources. + - Runs all OpenStack control functions (e.g. managing images, virtual + volumes, virtual network, and virtual machines). + - Can be part of a two-node HA control node cluster for running control + functions either active/active or active/standby. + +**Compute ( & network ) node / function** + A node that hosts applications in virtual machines using compute resources + such as CPU, memory, and disk. + + - Runs virtual switch for realizing virtual networks. + - Provides L3 routing and NET services. + +.. _incl-standard-controller-deployment-terminology-end: + + +.. _incl-dedicated-storage-deployment-terminology: + +**Storage node / function** + A node that contains a set of disks (e.g. SATA, SAS, SSD, and/or NVMe). + + - Runs CEPH distributed storage software. + - Part of an HA multi-node CEPH storage cluster supporting a replication + factor of two or three, journal caching, and class tiering. + - Provides HA persistent storage for images, virtual volumes + (i.e. block storage), and object storage. + +.. _incl-dedicated-storage-deployment-terminology-end: + +.. _incl-common-deployment-terminology: + +**OAM network** + The network on which all external StarlingX platform APIs are exposed, + (i.e. REST APIs, Horizon web server, SSH, and SNMP), typically 1GE. + + Only controller type nodes are required to be connected to the OAM + network. + +**Management network** + A private network (i.e. not connected externally), tipically 10GE, + used for the following: + + - Internal OpenStack / StarlingX monitoring and control. + - VM I/O access to a storage cluster. + + All nodes are required to be connected to the management network. + +**Data network(s)** + Networks on which the OpenStack / Neutron provider networks are realized + and become the VM tenant networks. + + Only compute type and all-in-one type nodes are required to be connected + to the data network(s); these node types require one or more interface(s) + on the data network(s). + +**IPMI network** + An optional network on which IPMI interfaces of all nodes are connected. + The network must be reachable using L3/IP from the controller's OAM + interfaces. + + You can optionally connect all node types to the IPMI network. + +**PXEBoot network** + An optional network for controllers to boot/install other nodes over the + network. + + By default, controllers use the management network for boot/install of other + nodes in the openstack cloud. If this optional network is used, all node + types are required to be connected to the PXEBoot network. + + A PXEBoot network is required for a variety of special case situations: + + - Cases where the management network must be IPv6: + + - IPv6 does not support PXEBoot. Therefore, IPv4 PXEBoot network must be + configured. + + - Cases where the management network must be VLAN tagged: + + - Most server's BIOS do not support PXEBooting over tagged networks. + Therefore, you must configure an untagged PXEBoot network. + + - Cases where a management network must be shared across regions but + individual regions' controllers want to only network boot/install nodes + of their own region: + + - You must configure separate, per-region PXEBoot networks. + +**Infra network** + A deprecated optional network that was historically used for access to the + storage cluster. + + If this optional network is used, all node types are required to be + connected to the INFRA network, + +**Node interfaces** + All nodes' network interfaces can, in general, optionally be either: + + - Untagged single port. + - Untagged two-port LAG and optionally split between redudant L2 switches + running vPC (Virtual Port-Channel), also known as multichassis + EtherChannel (MEC). + - VLAN on either single-port ETH interface or two-port LAG interface. + +.. _incl-common-deployment-terminology-end: diff --git a/doc/source/installation_guide/duplex.rst b/doc/source/installation_guide/2018_10/duplex.rst similarity index 87% rename from doc/source/installation_guide/duplex.rst rename to doc/source/installation_guide/2018_10/duplex.rst index d56ec40e4..0eb427cb1 100644 --- a/doc/source/installation_guide/duplex.rst +++ b/doc/source/installation_guide/2018_10/duplex.rst @@ -1,29 +1,27 @@ -.. _duplex: - -================================== -All-In-One Duplex Deployment Guide -================================== +============================================== +All-In-One Duplex deployment guide stx.2018.10 +============================================== .. contents:: :local: :depth: 1 -**NOTE:** The instructions to setup a StarlingX All-in-One Duplex +**NOTE:** The instructions to setup a StarlingX All-in-One Duplex (AIO-DX) with containerized openstack services in this guide are under development. For approved instructions, see the `All in One Duplex Configuration wiki page `__. ---------------------- -Deployment Description +Deployment description ---------------------- ***************** All-In-One Duplex ***************** -The All-In-One Duplex deployment option provides all three Cloud Functions -(Controller, Compute, and Storage) on two physical servers. With cloud +The All-In-One Duplex (AIO-DX) deployment option provides all three cloud +functions (controller, compute, and storage) on two physical servers. With cloud technologies, multiple diverse application types can be deployed and consolidated onto a protected pair of physical servers. For example: @@ -35,100 +33,100 @@ consolidated onto a protected pair of physical servers. For example: .. figure:: figures/starlingx-deployment-options-duplex.png :scale: 50% - :alt: All-In-One Duplex Deployment Configuration + :alt: All-In-One Duplex deployment configuration - *All-In-One Duplex Deployment Configuration* + *All-In-One Duplex deployment configuration* This two node cluster enables: -- High Availability Services running on the Controller Function across the - two physical servers in either Active/Active or Active/Standby mode. -- Storage Function running on top of LVM on single second disk, DRBD-sync'd +- High availability services running on the controller function across the + two physical servers in either active/active or active/standby mode. +- Storage function running on top of LVM on single second disk, DRBD-sync'd between the servers. -- Virtual Machines being scheduled on both Compute Functions. +- Virtual machines being scheduled on both compute functions. A All-In-One Duplex deployment provides protection against overall server hardware fault. Should an overall server hardware fault occur: -- All Controller High Availability Services go Active on remaining +- All controller high availability services go active on remaining healthy server. -- All Virtual Machines are recovered on remaining healthy server. +- All virtual machines are recovered on remaining healthy server. The All-In-One Duplex deployment solution is required for a variety of special case situations, for example: -- Small amount of Cloud Processing/Storage. +- Small amount of cloud processing/storage. - Protection against overall server hardware fault. ************************** -All-In-One Duplex Extended +All-In-One Duplex extended ************************** The All-In-One Duplex Extended deployment option extends the capacity of the -All-In-One Duplex Deployment by adding up to four Compute Nodes to the +All-In-One Duplex deployment by adding up to four compute nodes to the deployment. The extended deployment option provides a capacity growth path for someone starting with an All-In-One Duplex deployment. With this option, virtual machines can be scheduled on either of the -All-In-One Controller Nodes and/or the Compute Nodes. +all-in-one controller nodes and/or the compute nodes. .. figure:: figures/starlingx-deployment-options-duplex-extended.png :scale: 50% :alt: All-In-One Duplex Extended Deployment Configuration - *All-In-One Duplex Extended Deployment Configuration* + *All-In-One Duplex Extended deployment configuration* -This configuration is limited to four Compute Nodes as the Controller Function -on the All-In-One Controllers has only a portion of the processing power of the +This configuration is limited to four compute nodes as the controller function +on the all-in-one controllers has only a portion of the processing power of the overall server. ----------------------------------- -Preparing All-In-One Duplex Servers +Preparing All-In-One Duplex servers ----------------------------------- ********** -Bare Metal +Bare metal ********** Required Servers: -- Combined Servers (Controller + Compute): 2 +- Combined servers (controller + compute): 2 ^^^^^^^^^^^^^^^^^^^^^ -Hardware Requirements +Hardware requirements ^^^^^^^^^^^^^^^^^^^^^ The recommended minimum requirements for the physical servers where All-In-One Duplex will be deployed, include: -- Minimum Processor: +- Minimum processor: - - Typical Hardware Form Factor: + - Typical hardware form factor: - - Dual-CPU Intel® Xeon® E5 26xx Family (SandyBridge) 8 cores/socket - - Low Cost / Low Power Hardware Form Factor + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket + - Low cost / low power hardware form factor - - Single-CPU Intel Xeon D-15xx Family, 8 cores + - Single-CPU Intel Xeon D-15xx family, 8 cores - Memory: 64 GB - BIOS: - - Hyper-Threading Tech: Enabled - - Virtualization Technology: Enabled - - VT for Directed I/O: Enabled - - CPU Power and Performance Policy: Performance - - CPU C State Control: Disabled - - Plug & Play BMC Detection: Disabled + - Hyper-Threading technology: Enabled + - Virtualization technology: Enabled + - VT for directed I/O: Enabled + - CPU power and performance policy: Performance + - CPU C state control: Disabled + - Plug & play BMC detection: Disabled -- Primary Disk: +- Primary disk: - 500 GB SDD or NVMe -- Additional Disks: +- Additional disks: - Zero or more 500 GB disks (min. 10K RPM) -- Network Ports: +- Network ports: **NOTE:** The All-In-One Duplex configuration requires one or more data ports. @@ -137,11 +135,11 @@ All-In-One Duplex will be deployed, include: - Data: n x 10GE ******************* -Virtual Environment +Virtual environment ******************* -Run the libvirt qemu setup scripts. Setting up virtualized OAM and -Management networks: +Run the libvirt QEMU setup scripts. Setting up virtualized OAM and +management networks: :: @@ -160,7 +158,7 @@ are: - duplex-controller-1 ^^^^^^^^^^^^^^^^^^^^^^^^^ -Power Up a Virtual Server +Power up a virtual server ^^^^^^^^^^^^^^^^^^^^^^^^^ To power up a virtual server, run the following command: @@ -176,7 +174,7 @@ e.g. $ sudo virsh start duplex-controller-0 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Access Virtual Server Consoles +Access virtual server consoles ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The XML for virtual servers in stx-tools repo, deployment/libvirt, @@ -201,12 +199,12 @@ sequence which follows the boot device selection. One has a few seconds to do this. -------------------------------- -Installing the Controller-0 Host +Installing the controller-0 host -------------------------------- Installing controller-0 involves initializing a host with software and then applying a bootstrap configuration from the command line. The -configured bootstrapped host becomes Controller-0. +configured bootstrapped host becomes controller-0. Procedure: @@ -215,21 +213,21 @@ Procedure: #. Configure the controller using the config_controller script. ************************* -Initializing Controller-0 +Initializing controller-0 ************************* -This section describes how to initialize StarlingX in host Controller-0. +This section describes how to initialize StarlingX in host controller-0. Except where noted, all the commands must be executed from a console of the host. -Power on the host to be configured as Controller-0, with the StarlingX +Power on the host to be configured as controller-0, with the StarlingX ISO on a USB in a bootable USB slot. Wait for the console to show the StarlingX ISO booting options: - **All-in-one Controller Configuration** - When the installer is loaded and the installer welcome screen - appears in the Controller-0 host, select the type of installation + appears in the controller-0 host, select the type of installation "All-in-one Controller Configuration". - **Graphical Console** @@ -239,13 +237,13 @@ StarlingX ISO booting options: - **Standard Security Boot Profile** - - Select "Standard Security Boot Profile" as the Security Profile. + - Select "Standard Security Boot Profile" as the security profile. Monitor the initialization. When it is complete, a reboot is initiated -on the Controller-0 host, briefly displays a GNU GRUB screen, and then +on the controller-0 host, briefly displays a GNU GRUB screen, and then boots automatically into the StarlingX image. -Log into Controller-0 as user wrsroot, with password wrsroot. The +Log into controller-0 as user wrsroot, with password wrsroot. The first time you log in as wrsroot, you are required to change your password. Enter the current password (wrsroot): @@ -266,14 +264,14 @@ Enter the new password again to confirm it: Retype new password: -Controller-0 is initialized with StarlingX, and is ready for +controller-0 is initialized with StarlingX, and is ready for configuration. ************************ -Configuring Controller-0 +Configuring controller-0 ************************ -This section describes how to perform the Controller-0 configuration +This section describes how to perform the controller-0 configuration interactively just to bootstrap system with minimum critical data. Except where noted, all the commands must be executed from the console of the active controller (here assumed to be controller-0). @@ -281,9 +279,9 @@ of the active controller (here assumed to be controller-0). When run interactively, the config_controller script presents a series of prompts for initial configuration of StarlingX: -- For the Virtual Environment, you can accept all the default values +- For the virtual environment, you can accept all the default values immediately after ‘system date and time’. -- For a Physical Deployment, answer the bootstrap configuration +- For a physical deployment, answer the bootstrap configuration questions with answers applicable to your particular physical setup. The script is used to configure the first controller in the StarlingX @@ -299,7 +297,7 @@ with no parameters: Enter ! at any prompt to abort... ... -Select [y] for System Date and Time: +Select [y] for System date and time: :: @@ -320,7 +318,7 @@ For System mode choose "duplex": 3) simplex - single node non-redundant configuration System mode [duplex-direct]: 2 -After System Date / Time and System mode: +After System date and time and System mode: :: @@ -340,21 +338,21 @@ After System Date / Time and System mode: Please complete any out of service commissioning steps with system commands and unlock controller to proceed. After config_controller bootstrap configuration, REST API, CLI and -Horizon interfaces are enabled on the controller-0 OAM IP Address. The +Horizon interfaces are enabled on the controller-0 OAM IP address. The remaining installation instructions will use the CLI. ---------------------------------- -Provisioning the Controller-0 Host +Provisioning the controller-0 host ---------------------------------- -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc ********************************************* -Configuring Provider Networks at Installation +Configuring provider networks at installation ********************************************* Set up one provider network of the vlan type, named providernet-a: @@ -365,10 +363,10 @@ Set up one provider network of the vlan type, named providernet-a: [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-range-create --name providernet-a-range1 --range 100-400 providernet-a ***************************************** -Providing Data Interfaces on Controller-0 +Providing data interfaces on controller-0 ***************************************** -List all interfaces +List all interfaces: :: @@ -383,7 +381,7 @@ List all interfaces | f59b9469-7702-4b46-bad5-683b95f0a1cb | enp0s8 | platform |...| None | [u'enp0s8'] | [] | [] | MTU=1500 |.. +--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+.. -Configure the data interfaces +Configure the data interfaces: :: @@ -415,11 +413,11 @@ Configure the data interfaces +------------------+--------------------------------------+ ************************************* -Configuring Cinder on Controller Disk +Configuring Cinder on controller disk ************************************* Review the available disk space and capacity and obtain the uuid of the -physical disk +physical disk: :: @@ -439,7 +437,7 @@ physical disk | | | | | | |... +--------------------------------------+-----------+---------+---------+---------+------------+... -Create the 'cinder-volumes' local volume group +Create the 'cinder-volumes' local volume group: :: @@ -462,7 +460,7 @@ Create the 'cinder-volumes' local volume group | parameters | {u'lvm_type': u'thin'} | +-----------------+--------------------------------------+ -Create a disk partition to add to the volume group +Create a disk partition to add to the volume group: :: @@ -486,7 +484,7 @@ Create a disk partition to add to the volume group | updated_at | None | +-------------+--------------------------------------------------+ -Wait for the new partition to be created (i.e. status=Ready) +Wait for the new partition to be created (i.e. status=Ready): :: @@ -500,7 +498,7 @@ Wait for the new partition to be created (i.e. status=Ready) | |...| |...| | | | +--------------------------------------+...+------------+...+---------------------+----------+--------+ -Add the partition to the volume group +Add the partition to the volume group: :: @@ -526,10 +524,10 @@ Add the partition to the volume group +--------------------------+--------------------------------------------------+ ********************************************* -Adding an LVM Storage Backend at Installation +Adding an LVM storage backend at installation ********************************************* -Ensure requirements are met to add LVM storage +Ensure requirements are met to add LVM storage: :: @@ -543,7 +541,7 @@ Ensure requirements are met to add LVM storage storage. Set the 'confirmed' field to execute this operation for the lvm backend. -Add the LVM storage backend +Add the LVM storage backend: :: @@ -559,8 +557,7 @@ Add the LVM storage backend | e2697426-2d79-4a83-beb7-2eafa9ceaee5 | lvm-store | lvm | configuring |...| cinder | {} | +--------------------------------------+------------+---------+-------------+...+----------+--------------+ -Wait for the LVM storage backend to be configured (i.e. -state=Configured) +Wait for the LVM storage backend to be configured (i.e. state=configured): :: @@ -573,11 +570,11 @@ state=Configured) +--------------------------------------+------------+---------+------------+------+----------+--------------+ *********************************************** -Configuring VM Local Storage on Controller Disk +Configuring VM local storage on controller disk *********************************************** Review the available disk space and capacity and obtain the uuid of the -physical disk +physical disk: :: @@ -597,7 +594,7 @@ physical disk | | | | | | |... +--------------------------------------+-----------+---------+---------+---------+------------+... -Create the 'noval-local' volume group +Create the 'noval-local' volume group: :: @@ -622,7 +619,7 @@ Create the 'noval-local' volume group | parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} | +-----------------+-------------------------------------------------------------------+ -Create a disk partition to add to the volume group +Create a disk partition to add to the volume group: :: @@ -646,7 +643,7 @@ Create a disk partition to add to the volume group | updated_at | None | +-------------+--------------------------------------------------+ -Wait for the new partition to be created (i.e. status=Ready) +Wait for the new partition to be created (i.e. status=Ready): :: @@ -660,7 +657,7 @@ Wait for the new partition to be created (i.e. status=Ready) | |...| |...| | | | +--------------------------------------+...+------------+...+---------------------+----------+--------+ -Add the partition to the volume group +Add the partition to the volume group: :: @@ -686,11 +683,11 @@ Add the partition to the volume group +--------------------------+--------------------------------------------------+ ********************** -Unlocking Controller-0 +Unlocking controller-0 ********************** You must unlock controller-0 so that you can use it to install -Controller-1. Use the system host-unlock command: +controller-1. Use the system host-unlock command: :: @@ -701,10 +698,10 @@ unavailable, and any ssh connections are dropped. To monitor the progress of the reboot, use the controller-0 console. **************************************** -Verifying the Controller-0 Configuration +Verifying the controller-0 configuration **************************************** -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: @@ -724,7 +721,7 @@ Verify that the controller-0 services are running: ... +-----+-------------------------------+--------------+----------------+ -Verify that controller-0 has controller and compute subfunctions +Verify that controller-0 has controller and compute subfunctions: :: @@ -743,17 +740,17 @@ Verify that controller-0 is unlocked, enabled, and available: +----+--------------+-------------+----------------+-------------+--------------+ -------------------------------- -Installing the Controller-1 Host +Installing the controller-1 host -------------------------------- After initializing and configuring controller-0, you can add and configure a backup controller controller-1. ****************************** -Initializing Controller-1 Host +Initializing controller-1 host ****************************** -Power on Controller-1. In Controller-1 console you will see: +Power on controller-1. In controller-1 console you will see: :: @@ -763,16 +760,16 @@ Power on Controller-1. In Controller-1 console you will see: controller node in order to proceed. **************************************************** -Updating Controller-1 Host Host Name and Personality +Updating controller-1 host hostname and personality **************************************************** -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc -Wait for Controller-0 to discover new host, list the host until new +Wait for controller-0 to discover new host, list the host until new UNKNOWN host shows up in table: :: @@ -785,7 +782,7 @@ UNKNOWN host shows up in table: | 2 | None | None | locked | disabled | offline | +----+--------------+-------------+----------------+-------------+--------------+ -Use the system host-update to update Contoller-1 host personality +Use the system host-update to update contoller-1 host personality attribute: :: @@ -835,10 +832,10 @@ attribute: +---------------------+--------------------------------------+ **************************** -Monitoring Controller-1 Host +Monitoring controller-1 host **************************** -On Controller-0, you can monitor the installation progress by running +On controller-0, you can monitor the installation progress by running the system host-show command for the host periodically. Progress is shown in the install_state field. @@ -849,16 +846,16 @@ shown in the install_state field. | install_state | booting | | install_state_info | None | -Wait while the Controller-1 is configured and rebooted. Up to 20 minutes +Wait while the controller-1 is configured and rebooted. Up to 20 minutes may be required for a reboot, depending on hardware. When the reboot is -complete, the Controller-1 is reported as Locked, Disabled, and Online. +complete, the controller-1 is reported as locked, disabled, and online. ************************* -Listing Controller-1 Host +Listing controller-1 host ************************* -Once Controller-1 has been installed, configured and rebooted, on -Controller-0 list the hosts: +Once controller-1 has been installed, configured and rebooted, on +controller-0 list the hosts: :: @@ -871,10 +868,10 @@ Controller-0 list the hosts: +----+--------------+-------------+----------------+-------------+--------------+ ---------------------------------- -Provisioning the Controller-1 Host +Provisioning the controller-1 host ---------------------------------- -On Controller-0, list hosts +On controller-0, list hosts: :: @@ -887,17 +884,17 @@ On Controller-0, list hosts +----+--------------+-------------+----------------+-------------+--------------+ *********************************************** -Provisioning Network Interfaces on Controller-1 +Provisioning network interfaces on controller-1 *********************************************** -In order to list out hardware port names, types, pci-addresses that have +In order to list out hardware port names, types, PCI addresses that have been discovered: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list controller-1 -Provision the Controller-1 oam interface +Provision the controller-1 OAM interface :: @@ -929,10 +926,10 @@ Provision the Controller-1 oam interface +------------------+--------------------------------------+ ***************************************** -Providing Data Interfaces on Controller-1 +Providing data interfaces on controller-1 ***************************************** -List all interfaces +List all interfaces: :: @@ -948,7 +945,7 @@ List all interfaces | e78ad9a9-e74d-4c6c-9de8-0e41aad8d7b7 | eth1000 | None |...| None | [u'eth1000'] | [] | [] | MTU=1500 |.. +--------------------------------------+---------+---------+...+------+--------------+------+------+------------+.. -Configure the data interfaces +Configure the data interfaces: :: @@ -980,11 +977,11 @@ Configure the data interfaces +------------------+--------------------------------------+ ************************************ -Provisioning Storage on Controller-1 +Provisioning storage on controller-1 ************************************ Review the available disk space and capacity and obtain the uuid of the -physical disk +physical disk: :: @@ -997,7 +994,7 @@ physical disk | 623bbfc0-2b38-432a-acf4-a28db6066cce | /dev/sdc | 2080 | HDD | 16240 | 16237 |... +--------------------------------------+-------------+------------+-------------+----------+---------------+... -Assign Cinder storage to the physical disk +Assign Cinder storage to the physical disk: :: @@ -1023,7 +1020,7 @@ Assign Cinder storage to the physical disk +-----------------+--------------------------------------+ Create a disk partition to add to the volume group based on uuid of the -physical disk +physical disk: :: @@ -1047,7 +1044,7 @@ physical disk | updated_at | None | +-------------+--------------------------------------------------+ -Wait for the new partition to be created (i.e. status=Ready) +Wait for the new partition to be created (i.e. status=Ready): :: @@ -1058,7 +1055,7 @@ Wait for the new partition to be created (i.e. status=Ready) | 7a41aab0-6695-4d16-9003-73238adda75b |...| /dev/sdb1 |...| None | 16237 | Creating (on unlock) | +--------------------------------------+...+-------------+...+-----------+----------+----------------------+ -Add the partition to the volume group +Add the partition to the volume group: :: @@ -1083,14 +1080,12 @@ Add the partition to the volume group | updated_at | None | +--------------------------+--------------------------------------------------+ -.. _configuring-vm-local-storage-on-controller-disk-1: - *********************************************** -Configuring VM Local Storage on Controller Disk +Configuring VM local storage on controller disk *********************************************** Review the available disk space and capacity and obtain the uuid of the -physical disk +physical disk: :: @@ -1103,7 +1098,7 @@ physical disk | 623bbfc0-2b38-432a-acf4-a28db6066cce | /dev/sdc | 2080 | HDD | 16240 | 16237 |... +--------------------------------------+-------------+------------+-------------+----------+---------------+... -Create the 'cinder-volumes' local volume group +Create the 'cinder-volumes' local volume group: :: @@ -1128,7 +1123,7 @@ Create the 'cinder-volumes' local volume group | parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} | +-----------------+-------------------------------------------------------------------+ -Create a disk partition to add to the volume group +Create a disk partition to add to the volume group: :: @@ -1152,7 +1147,7 @@ Create a disk partition to add to the volume group | updated_at | None | +-------------+--------------------------------------------------+ -Wait for the new partition to be created (i.e. status=Ready) +Wait for the new partition to be created (i.e. status=Ready): :: @@ -1164,7 +1159,7 @@ Wait for the new partition to be created (i.e. status=Ready) | f7bc6095-9375-49fe-83c7-12601c202376 |...| /dev/sdc1 |...| None | 16237 | Creating (on unlock) | +--------------------------------------+...+-------------+...+-----------+----------+----------------------+ -Add the partition to the volume group +Add the partition to the volume group: :: @@ -1190,19 +1185,19 @@ Add the partition to the volume group +--------------------------+--------------------------------------------------+ ********************** -Unlocking Controller-1 +Unlocking controller-1 ********************** -Unlock Controller-1 +Unlock controller-1: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-1 -Wait while the Controller-1 is rebooted. Up to 10 minutes may be +Wait while the controller-1 is rebooted. Up to 10 minutes may be required for a reboot, depending on hardware. -REMARK: Controller-1 will remain in 'degraded' state until data-syncing +REMARK: controller-1 will remain in degraded state until data-syncing is complete. The duration is dependant on the virtualization host's configuration - i.e., the number and configuration of physical disks used to host the nodes' virtual disks. Also, the management network is @@ -1220,20 +1215,20 @@ excessive data-sync time). Use 'fm alarm-list' to confirm status. +----+--------------+-------------+----------------+-------------+--------------+ ----------------------------------- -Extending the Compute Node Capacity +Extending the compute node capacity ----------------------------------- -You can add up to four Compute Nodes to the All-in-One Duplex deployment. +You can add up to four compute nodes to the All-in-One Duplex deployment. ************************** -Compute Hosts Installation +Compute hosts installation ************************** After initializing and configuring the two controllers, you can add up to four additional compute hosts. To add a host, do the following: ^^^^^^^^^^^^^^^^^ -Initializing Host +Initializing host ^^^^^^^^^^^^^^^^^ Power on the host. The following appears in the host console: @@ -1246,16 +1241,16 @@ Power on the host. The following appears in the host console: controller node in order to proceed. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Updating the Host Name and Personality +Updating the hostname and personality ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc -Wait for the Controller-0 to both discover the new host and to list that host +Wait for the controller-0 to both discover the new host and to list that host as UNKNOWN in the table: :: @@ -1275,7 +1270,7 @@ Use the system host-update command to update the host personality attribute: [wrsroot@controller-0 ~(keystone_admin)]$ system host-update 3 personality=compute hostname=compute-0 -See also: 'system help host-update' +See also: 'system help host-update'. Unless it is known that the host's configuration can support the installation of more than one node, it is recommended that the @@ -1285,10 +1280,10 @@ root disk and that disk happens to be a single rotational type hard disk, then the host cannot reliably support parallel node installation. ^^^^^^^^^^^^^^^ -Monitoring Host +Monitoring host ^^^^^^^^^^^^^^^ -On Controller-0, you can monitor the installation progress by periodically +On controller-0, you can monitor the installation progress by periodically running the system host-show command for the host. Progress is shown in the install_state field. @@ -1301,11 +1296,11 @@ shown in the install_state field. Wait while the host is installed, configured, and rebooted. Depending on hardware, it could take up to 20 minutes for this process to complete. -When the reboot is complete, the host is reported as Locked, Disabled, -and Online. +When the reboot is complete, the host is reported as locked, disabled, +and online. ^^^^^^^^^^^^^ -Listing Hosts +Listing hosts ^^^^^^^^^^^^^ You can use the system host-list command to list the hosts once the node @@ -1323,26 +1318,26 @@ has been installed, configured, and rebooted: +----+--------------+-------------+----------------+-------------+--------------+ ***************************** -Provisioning the Compute Host +Provisioning the compute host ***************************** You must configure the network interfaces and the storage disks on a -host before you can unlock it. For each Compute Host, do the following: +host before you can unlock it. For each compute host, do the following: -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Provisioning Network Interfaces on a Compute Host +Provisioning network interfaces on a compute host ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In order to identify hardware port names, types, and discovered -pci-addresses on Controller-0, list the host ports: +pci-addresses on controller-0, list the host ports: -- **Only in Virtual Environment**: Ensure that the interface used is +- **Only in virtual environment**: Ensure that the interface used is one of those attached to the host bridge with model type "virtio" (i.e. eth1000 and eth1001). The model type "e1000" emulated devices will not work for provider networks: @@ -1359,17 +1354,17 @@ pci-addresses on Controller-0, list the host ports: | c1694675-643d-4ba7-b821-cd147450112e | eth1001 | ethernet | 0000:02:04.0 |... +--------------------------------------+---------+----------+--------------+... -Use the following command to provision the data interface for Compute: +Use the following command to provision the data interface for compute: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -p providernet-a -c data compute-0 eth1000 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -VSwitch Virtual Environment +VSwitch virtual environment ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -**Only in Virtual Environment**. If the compute node has more than four CPUs, +**Only in virtual environment**. If the compute node has more than four CPUs, the system auto-configures the vswitch to use two cores. However, some virtual environments do not properly support multi-queue, which is required in a multi-CPU environment. Therefore, run the following command to reduce the @@ -1388,7 +1383,7 @@ vswitch cores to one: +--------------------------------------+-------+-----------+-------+--------+... ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Provisioning Storage on a Compute Host +Provisioning storage on a compute host ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Review the available disk space and capacity and then obtain the uuid(s) of @@ -1458,31 +1453,31 @@ group based on the uuid of the physical disk: +--------------------------+--------------------------------------------+ ^^^^^^^^^^^^^^^^^^^^^^^^ -Unlocking a Compute Host +Unlocking a compute host ^^^^^^^^^^^^^^^^^^^^^^^^ -On Controller-0, use the system host-unlock command to unlock the -Compute node: +On controller-0, use the system host-unlock command to unlock the +compute node: :: [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock compute-0 -Wait while the Compute node is rebooted and re-configured. Depending on +Wait while the compute node is rebooted and re-configured. Depending on hardware, it can take up to 10 minutes for the reboot to complete. Once -the reboot is complete, the nodes Availability State reports as "In-Test" +the reboot is complete, the nodes availability state reports as "in-test" and is followed by unlocked/enabled. ------------------- -System Health Check +System health check ------------------- *********************** -Listing StarlingX Nodes +Listing StarlingX nodes *********************** -On Controller-0, after a few minutes, all nodes are reported as -Unlocked, Enabled, and Available: +On controller-0, after a few minutes, all nodes are reported as +unlocked, enabled, and available: :: @@ -1496,18 +1491,20 @@ Unlocked, Enabled, and Available: +----+--------------+-------------+----------------+-------------+--------------+ ***************** -System Alarm List +System alarm list ***************** -When all nodes are Unlocked, Enabled and Available: check 'fm alarm-list' for issues. +When all nodes are unlocked, enabled, and available: check 'fm alarm-list' for +issues. -Your StarlingX deployment is now up and running with 2x HA Controllers with Cinder -Storage and all OpenStack services up and running. You can now proceed with standard -OpenStack APIs, CLIs and/or Horizon to load Glance Images, configure Nova Flavors, -configure Neutron networks and launch Nova Virtual Machines. +Your StarlingX deployment is now up and running with 2x HA controllers with +Cinder storage and all OpenStack services up and running. You can now proceed +with standard OpenStack APIs, CLIs and/or Horizon to load Glance images, +configure Nova Flavors, configure Neutron networks and launch Nova virtual +machines. ---------------------- -Deployment Terminology +Deployment terminology ---------------------- .. include:: deployment_terminology.rst diff --git a/doc/source/installation_guide/figures/starlingx-deployment-options-controller-storage.png b/doc/source/installation_guide/2018_10/figures/starlingx-deployment-options-controller-storage.png similarity index 100% rename from doc/source/installation_guide/figures/starlingx-deployment-options-controller-storage.png rename to doc/source/installation_guide/2018_10/figures/starlingx-deployment-options-controller-storage.png diff --git a/doc/source/installation_guide/figures/starlingx-deployment-options-dedicated-storage.png b/doc/source/installation_guide/2018_10/figures/starlingx-deployment-options-dedicated-storage.png similarity index 100% rename from doc/source/installation_guide/figures/starlingx-deployment-options-dedicated-storage.png rename to doc/source/installation_guide/2018_10/figures/starlingx-deployment-options-dedicated-storage.png diff --git a/doc/source/installation_guide/figures/starlingx-deployment-options-duplex-extended.png b/doc/source/installation_guide/2018_10/figures/starlingx-deployment-options-duplex-extended.png similarity index 100% rename from doc/source/installation_guide/figures/starlingx-deployment-options-duplex-extended.png rename to doc/source/installation_guide/2018_10/figures/starlingx-deployment-options-duplex-extended.png diff --git a/doc/source/installation_guide/figures/starlingx-deployment-options-duplex.png b/doc/source/installation_guide/2018_10/figures/starlingx-deployment-options-duplex.png similarity index 100% rename from doc/source/installation_guide/figures/starlingx-deployment-options-duplex.png rename to doc/source/installation_guide/2018_10/figures/starlingx-deployment-options-duplex.png diff --git a/doc/source/installation_guide/figures/starlingx-deployment-options-simplex.png b/doc/source/installation_guide/2018_10/figures/starlingx-deployment-options-simplex.png similarity index 100% rename from doc/source/installation_guide/figures/starlingx-deployment-options-simplex.png rename to doc/source/installation_guide/2018_10/figures/starlingx-deployment-options-simplex.png diff --git a/doc/source/installation_guide/2018_10/index.rst b/doc/source/installation_guide/2018_10/index.rst new file mode 100644 index 000000000..57ee6d119 --- /dev/null +++ b/doc/source/installation_guide/2018_10/index.rst @@ -0,0 +1,288 @@ +============================== +Installation guide stx.2018.10 +============================== + +This is the installation guide for release stx.2018.10. If an installation +guide is needed for a previous release, review the +:doc:`installation guides for previous releases `. + +------------ +Introduction +------------ + +StarlingX may be installed in: + +- **Bare metal**: Real deployments of StarlingX are only supported on + physical servers. +- **Virtual environment**: It should only be used for evaluation or + development purposes. + +StarlingX installed in virtual environments has two options: + +- :doc:`Libvirt/QEMU ` +- VirtualBox + +------------ +Requirements +------------ + +Different use cases require different configurations. + +********** +Bare metal +********** + +The minimum requirements for the physical servers where StarlingX might +be deployed, include: + +- **Controller hosts** + + - Minimum processor is: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 + cores/socket + + - Minimum memory: 64 GB + - Hard drives: + + - Primary hard drive, minimum 500 GB for OS and system databases. + - Secondary hard drive, minimum 500 GB for persistent VM storage. + + - 2 physical Ethernet interfaces: OAM and MGMT network. + - USB boot support. + - PXE boot support. + +- **Storage hosts** + + - Minimum processor is: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 + cores/socket. + + - Minimum memory: 64 GB. + - Hard drives: + + - Primary hard drive, minimum 500 GB for OS. + - 1 or more additional hard drives for CEPH OSD storage, and + - Optionally 1 or more SSD or NVMe drives for CEPH journals. + + - 1 physical Ethernet interface: MGMT network + - PXE boot support. + +- **Compute hosts** + + - Minimum processor is: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 + cores/socket. + + - Minimum memory: 32 GB. + - Hard drives: + + - Primary hard drive, minimum 500 GB for OS. + - 1 or more additional hard drives for ephemeral VM storage. + + - 2 or more physical Ethernet interfaces: MGMT network and 1 or more + provider networks. + - PXE boot support. + +- **All-In-One Simplex or Duplex, controller + compute hosts** + + - Minimum processor is: + + - Typical hardware form factor: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket + - Low cost / low power hardware form factor + + - Single-CPU Intel Xeon D-15xx family, 8 cores + + - Minimum memory: 64 GB. + - Hard drives: + + - Primary hard drive, minimum 500 GB SDD or NVMe. + - 0 or more 500 GB disks (min. 10K RPM). + + - Network ports: + + **NOTE:** Duplex and Simplex configurations require one or more data + ports. + The Duplex configuration requires a management port. + + - Management: 10GE (Duplex only) + - OAM: 10GE + - Data: n x 10GE + +The recommended minimum requirements for the physical servers are +described later in each StarlingX deployment guide. + +^^^^^^^^^^^^^^^^^^^^^^^^ +NVMe drive as boot drive +^^^^^^^^^^^^^^^^^^^^^^^^ + +To use a Non-Volatile Memory Express (NVMe) drive as the boot drive for any of +your nodes, you must configure your host and adjust kernel parameters during +installation: + +- Configure the host to be in UEFI mode. +- Edit the kernel boot parameter. After you are presented with the StarlingX + ISO boot options and after you have selected the preferred installation option + (e.g. Standard Configuration / All-in-One Controller Configuration), press the + TAB key to edit the kernel boot parameters. Modify the **boot_device** and + **rootfs_device** from the default **sda** so that it is the correct device + name for the NVMe drive (e.g. "nvme0n1"). + + :: + + vmlinuz rootwait console=tty0 inst.text inst.stage2=hd:LABEL=oe_iso_boot + inst.ks=hd:LABEL=oe_iso_boot:/smallsystem_ks.cfg boot_device=nvme0n1 + rootfs_device=nvme0n1 biosdevname=0 usbcore.autosuspend=-1 inst.gpt + security_profile=standard user_namespace.enable=1 initrd=initrd.img + + +******************* +Virtual environment +******************* + +The recommended minimum requirements for the workstation, hosting the +virtual machine(s) where StarlingX will be deployed, include: + +^^^^^^^^^^^^^^^^^^^^^ +Hardware requirements +^^^^^^^^^^^^^^^^^^^^^ + +A workstation computer with: + +- Processor: x86_64 only supported architecture with BIOS enabled + hardware virtualization extensions +- Cores: 8 (4 with careful monitoring of cpu load) +- Memory: At least 32GB RAM +- Hard Disk: 500GB HDD +- Network: Two network adapters with active Internet connection + +^^^^^^^^^^^^^^^^^^^^^ +Software requirements +^^^^^^^^^^^^^^^^^^^^^ + +A workstation computer with: + +- Operating System: Freshly installed Ubuntu 16.04 LTS 64-bit +- Proxy settings configured (if applies) +- Git +- KVM/VirtManager +- Libvirt library +- QEMU full-system emulation binaries +- stx-tools project +- StarlingX ISO image + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Deployment environment setup +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This section describes how to set up the workstation computer which will +host the virtual machine(s) where StarlingX will be deployed. + +'''''''''''''''''''''''''''''' +Updating your operating system +'''''''''''''''''''''''''''''' + +Before proceeding with the build, ensure your OS is up to date. You’ll +first need to update the local database list of available packages: + +:: + + $ sudo apt-get update + +''''''''''''''''''''''''' +Install stx-tools project +''''''''''''''''''''''''' + +Clone the stx-tools project. Usually you’ll want to clone it under your +user’s home directory. + +:: + + $ cd $HOME + $ git clone https://git.starlingx.io/stx-tools + + +'''''''''''''''''''''''''''''''''''''''' +Installing requirements and dependencies +'''''''''''''''''''''''''''''''''''''''' + +Navigate to the stx-tools installation libvirt directory: + +:: + + $ cd $HOME/stx-tools/deployment/libvirt/ + + +Install the required packages: + +:: + + $ bash install_packages.sh + + +'''''''''''''''''' +Disabling firewall +'''''''''''''''''' + +Unload firewall and disable firewall on boot: + +:: + + $ sudo ufw disable + Firewall stopped and disabled on system startup + $ sudo ufw status + Status: inactive + + +------------------------------- +Getting the StarlingX ISO image +------------------------------- + +Follow the instructions from the :doc:`/developer_guide/2018_10/index` to build a +StarlingX ISO image. + +********** +Bare metal +********** + +A bootable USB flash drive containing StarlingX ISO image. + + +******************* +Virtual environment +******************* + +Copy the StarlingX ISO Image to the stx-tools deployment libvirt project +directory: + +:: + + $ cp $HOME/stx-tools/deployment/libvirt/ + + +------------------ +Deployment options +------------------ + +- Standard controller + + - :doc:`StarlingX Cloud with Dedicated Storage ` + - :doc:`StarlingX Cloud with Controller Storage ` + +- All-in-one + + - :doc:`StarlingX Cloud Duplex ` + - :doc:`StarlingX Cloud Simplex ` + +.. toctree:: + :hidden: + + installation_libvirt_qemu + controller_storage + dedicated_storage + duplex + simplex diff --git a/doc/source/installation_guide/2018_10/installation_libvirt_qemu.rst b/doc/source/installation_guide/2018_10/installation_libvirt_qemu.rst new file mode 100644 index 000000000..5caf027b7 --- /dev/null +++ b/doc/source/installation_guide/2018_10/installation_libvirt_qemu.rst @@ -0,0 +1,204 @@ +===================================== +Installation libvirt qemu stx.2018.10 +===================================== + +Installation for StarlingX stx.2018.10 using Libvirt/QEMU virtualization. + +--------------------- +Hardware requirements +--------------------- + +A workstation computer with: + +- Processor: x86_64 only supported architecture with BIOS enabled + hardware virtualization extensions +- Memory: At least 32GB RAM +- Hard disk: 500GB HDD +- Network: One network adapter with active Internet connection + +--------------------- +Software requirements +--------------------- + +A workstation computer with: + +- Operating system: This process is known to work on Ubuntu 16.04 and + is likely to work on other Linux OS's with some appropriate adjustments. +- Proxy settings configured (if applies) +- Git +- KVM/VirtManager +- Libvirt library +- QEMU full-system emulation binaries +- stx-tools project +- StarlingX ISO image + +---------------------------- +Deployment environment setup +---------------------------- + +************* +Configuration +************* + +These scripts are configured using environment variables that all have +built-in defaults. On shared systems you probably do not want to use the +defaults. The simplest way to handle this is to keep an rc file that can +be sourced into an interactive shell that configures everything. Here's +an example called stxcloud.rc: + +:: + + export CONTROLLER=stxcloud + export COMPUTE=stxnode + export STORAGE=stxstorage + export BRIDGE_INTERFACE=stxbr + export INTERNAL_NETWORK=172.30.20.0/24 + export INTERNAL_IP=172.30.20.1/24 + export EXTERNAL_NETWORK=192.168.20.0/24 + export EXTERNAL_IP=192.168.20.1/24 + + +This rc file shows the defaults baked into the scripts: + +:: + + export CONTROLLER=controller + export COMPUTE=compute + export STORAGE=storage + export BRIDGE_INTERFACE=stxbr + export INTERNAL_NETWORK=10.10.10.0/24 + export INTERNAL_IP=10.10.10.1/24 + export EXTERNAL_NETWORK=192.168.204.0/24 + export EXTERNAL_IP=192.168.204.1/24 + + +************************* +Install stx-tools project +************************* + +Clone the stx-tools project into a working directory. + +:: + + git clone git://git.openstack.org/openstack/stx-tools.git + + +It is convenient to set up a shortcut to the deployment script +directory: + +:: + + SCRIPTS=$(pwd)/stx-tools/deployment/libvirt + + +If you created a configuration, load it from stxcloud.rc: + +:: + + source stxcloud.rc + + +**************************************** +Installing requirements and dependencies +**************************************** + +Install the required packages and configure QEMU. This only needs to be +done once per host. (NOTE: this script only knows about Ubuntu at this +time): + +:: + + $SCRIPTS/install_packages.sh + + +****************** +Disabling firewall +****************** + +Unload firewall and disable firewall on boot: + +:: + + sudo ufw disable + sudo ufw status + + +****************** +Configure networks +****************** + +Configure the network bridges using setup_network.sh before doing +anything else. It will create 4 bridges named stxbr1, stxbr2, stxbr3 and +stxbr4. Set the BRIDGE_INTERFACE environment variable if you need to +change stxbr to something unique. + +:: + + $SCRIPTS/setup_network.sh + + +The destroy_network.sh script does the reverse, and should not be used +lightly. It should also only be used after all of the VMs created below +have been destroyed. + +There is also a script cleanup_network.sh that will remove networking +configuration from libvirt. + +********************* +Configure controllers +********************* + +One script exists for building different StarlingX cloud configurations: +setup_configuration.sh. + +The script uses the cloud configuration with the -c option: + +- simplex +- duplex +- controllerstorage +- dedicatedstorage + +You need an ISO file for the installation, the script takes a file name +with the -i option: + +:: + + $SCRIPTS/setup_configuration.sh -c -i + + +And the setup will begin. The scripts create one or more VMs and start +the boot of the first controller, named oddly enough \``controller-0``. +If you have Xwindows available you will get virt-manager running. If +not, Ctrl-C out of that attempt if it doesn't return to a shell prompt. +Then connect to the serial console: + +:: + + virsh console controller-0 + + +Continue the usual StarlingX installation from this point forward. + +Tear down the VMs using destroy_configuration.sh. + +:: + + $SCRIPTS/destroy_configuration.sh -c + + +-------- +Continue +-------- + +Pick up the installation in one of the existing guides at the initializing +controller-0 step. + +- Standard controller + + - :doc:`StarlingX Cloud with Dedicated Storage Virtual Environment ` + - :doc:`StarlingX Cloud with Controller Storage Virtual Environment ` + +- All-in-one + + - :doc:`StarlingX Cloud Duplex Virtual Environment ` + - :doc:`StarlingX Cloud Simplex Virtual Environment ` diff --git a/doc/source/installation_guide/simplex.rst b/doc/source/installation_guide/2018_10/simplex.rst similarity index 86% rename from doc/source/installation_guide/simplex.rst rename to doc/source/installation_guide/2018_10/simplex.rst index 0ddaa3bd8..c639cba32 100644 --- a/doc/source/installation_guide/simplex.rst +++ b/doc/source/installation_guide/2018_10/simplex.rst @@ -1,8 +1,6 @@ -.. _simplex: - -=================================== -All-In-One Simplex Deployment Guide -=================================== +=============================================== +All-In-One Simplex deployment guide stx.2018.10 +=============================================== .. contents:: :local: @@ -15,14 +13,14 @@ For approved instructions, see the `One Node Configuration wiki page `__. ---------------------- -Deployment Description +Deployment description ---------------------- -The All-In-One Simplex deployment option provides all three Cloud Functions -(Controller, Compute, and Storage) on a single physical server. With these Cloud -Functions, multiple application types can be deployed and consolidated onto a -single physical server. For example, with a All-In-One Simplex deployment you -can: +The All-In-One Simplex (AIO-SX) deployment option provides all three cloud +gunctions (controller, compute, and storage) on a single physical server. With +these cloud functions, multiple application types can be deployed and +consolidated onto a single physical server. For example, with a AIO-SX +deployment you can: - Consolidate legacy applications that must run standalone on a server by using multiple virtual machines on a single physical server. @@ -30,14 +28,14 @@ can: different distributions of operating systems by using multiple virtual machines on a single physical server. -Only a small amount of Cloud Processing / Storage power is required with an +Only a small amount of cloud processing / storage power is required with an All-In-One Simplex deployment. .. figure:: figures/starlingx-deployment-options-simplex.png :scale: 50% - :alt: All-In-One Simplex Deployment Configuration + :alt: All-In-One Simplex deployment configuration - *All-In-One Simplex Deployment Configuration* + *All-In-One Simplex deployment configuration* An All-In-One Simplex deployment provides no protection against an overall server hardware fault. Protection against overall server hardware fault is @@ -46,52 +44,52 @@ could be enabled if, for example, an HW RAID or 2x Port LAG is used in the deployment. -------------------------------------- -Preparing an All-In-One Simplex Server +Preparing an All-In-One Simplex server -------------------------------------- ********** -Bare Metal +Bare metal ********** Required Server: -- Combined Server (Controller + Compute): 1 +- Combined server (controller + compute): 1 ^^^^^^^^^^^^^^^^^^^^^ -Hardware Requirements +Hardware requirements ^^^^^^^^^^^^^^^^^^^^^ The recommended minimum requirements for the physical servers where All-In-One Simplex will be deployed are: -- Minimum Processor: +- Minimum processor: - - Typical Hardware Form Factor: + - Typical hardware form factor: - - Dual-CPU Intel® Xeon® E5 26xx Family (SandyBridge) 8 cores/socket - - Low Cost / Low Power Hardware Form Factor + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket + - Low cost / low power hardware form factor - - Single-CPU Intel Xeon D-15xx Family, 8 cores + - Single-CPU Intel Xeon D-15xx family, 8 cores - Memory: 64 GB - BIOS: - - Hyper-Threading Tech: Enabled - - Virtualization Technology: Enabled - - VT for Directed I/O: Enabled - - CPU Power and Performance Policy: Performance - - CPU C State Control: Disabled - - Plug & Play BMC Detection: Disabled + - Hyper-Threading technology: Enabled + - Virtualization technology: Enabled + - VT for directed I/O: Enabled + - CPU power and performance policy: Performance + - CPU C state control: Disabled + - Plug & play BMC detection: Disabled -- Primary Disk: +- Primary disk: - 500 GB SDD or NVMe -- Additional Disks: +- Additional disks: - Zero or more 500 GB disks (min. 10K RPM) -- Network Ports +- Network ports **NOTE:** All-In-One Simplex configuration requires one or more data ports. This configuration does not require a management port. @@ -100,11 +98,11 @@ All-In-One Simplex will be deployed are: - Data: n x 10GE ******************* -Virtual Environment +Virtual environment ******************* Run the libvirt qemu setup scripts. Setting up virtualized OAM and -Management networks: +management networks: :: @@ -121,7 +119,7 @@ The default XML server definition created by the previous script is: - simplex-controller-0 ^^^^^^^^^^^^^^^^^^^^^^^^^ -Power Up a Virtual Server +Power up a virtual server ^^^^^^^^^^^^^^^^^^^^^^^^^ To power up the virtual server, run the following command: @@ -137,7 +135,7 @@ e.g. $ sudo virsh start simplex-controller-0 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Access a Virtual Server Console +Access a virtual server console ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The XML for virtual servers in stx-tools repo, deployment/libvirt, @@ -162,12 +160,12 @@ sequence which follows the boot device selection. One has a few seconds to do this. ------------------------------ -Installing the Controller Host +Installing the controller host ------------------------------ Installing controller-0 involves initializing a host with software and then applying a bootstrap configuration from the command line. The -configured bootstrapped host becomes Controller-0. +configured bootstrapped host becomes controller-0. Procedure: @@ -176,21 +174,21 @@ Procedure: #. Configure the controller using the config_controller script. ************************* -Initializing Controller-0 +Initializing controller-0 ************************* -This section describes how to initialize StarlingX in host Controller-0. +This section describes how to initialize StarlingX in host controller-0. Except where noted, all the commands must be executed from a console of the host. -Power on the host to be configured as Controller-0, with the StarlingX +Power on the host to be configured as controller-0, with the StarlingX ISO on a USB in a bootable USB slot. Wait for the console to show the StarlingX ISO booting options: - **All-in-one Controller Configuration** - When the installer is loaded and the installer welcome screen - appears in the Controller-0 host, select the type of installation + appears in the controller-0 host, select the type of installation "All-in-one Controller Configuration". - **Graphical Console** @@ -203,10 +201,10 @@ StarlingX ISO booting options: - Select "Standard Security Boot Profile" as the Security Profile. Monitor the initialization. When it is complete, a reboot is initiated -on the Controller-0 host, briefly displays a GNU GRUB screen, and then +on the controller-0 host, briefly displays a GNU GRUB screen, and then boots automatically into the StarlingX image. -Log into Controller-0 as user wrsroot, with password wrsroot. The +Log into controller-0 as user wrsroot, with password wrsroot. The first time you log in as wrsroot, you are required to change your password. Enter the current password (wrsroot): @@ -228,14 +226,13 @@ Enter the new password again to confirm it: Retype new password: -Controller-0 is initialized with StarlingX, and is ready for -configuration. +controller-0 is initialized with StarlingX, and is ready for configuration. ************************ -Configuring Controller-0 +Configuring controller-0 ************************ -This section describes how to perform the Controller-0 configuration +This section describes how to perform the controller-0 configuration interactively just to bootstrap system with minimum critical data. Except where noted, all the commands must be executed from the console of the active controller (here assumed to be controller-0). @@ -243,9 +240,9 @@ of the active controller (here assumed to be controller-0). When run interactively, the config_controller script presents a series of prompts for initial configuration of StarlingX: -- For the Virtual Environment, you can accept all the default values +- For the virtual environment, you can accept all the default values immediately after ‘system date and time’. -- For a Physical Deployment, answer the bootstrap configuration +- For a physical deployment, answer the bootstrap configuration questions with answers applicable to your particular physical setup. The script is used to configure the first controller in the StarlingX @@ -261,7 +258,7 @@ with no parameters: Enter ! at any prompt to abort... ... -Select [y] for System Date and Time: +Select [y] for System date and time: :: @@ -281,7 +278,7 @@ For System mode choose "simplex": 3) simplex - single node non-redundant configuration System mode [duplex-direct]: 3 -After System Date / Time and System mode: +After System date and time and System mode: :: @@ -302,21 +299,21 @@ After System Date / Time and System mode: commands and unlock controller to proceed. After config_controller bootstrap configuration, REST API, CLI and -Horizon interfaces are enabled on the controller-0 OAM IP Address. The +Horizon interfaces are enabled on the controller-0 OAM IP address. The remaining installation instructions will use the CLI. -------------------------------- -Provisioning the Controller Host +Provisioning the controller host -------------------------------- -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: controller-0:~$ source /etc/nova/openrc ********************************************* -Configuring Provider Networks at Installation +Configuring provider networks at installation ********************************************* Set up one provider network of the vlan type, named providernet-a: @@ -327,10 +324,10 @@ Set up one provider network of the vlan type, named providernet-a: [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-range-create --name providernet-a-range1 --range 100-400 providernet-a ***************************************** -Providing Data Interfaces on Controller-0 +Providing data interfaces on controller-0 ***************************************** -List all interfaces +List all interfaces: :: @@ -345,7 +342,7 @@ List all interfaces | f59b9469-7702-4b46-bad5-683b95f0a1cb | enp0s8 | platform |...| None | [u'enp0s8'] | [] | [] | MTU=1500 |.. +--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+.. -Configure the data interfaces +Configure the data interfaces: :: @@ -377,11 +374,11 @@ Configure the data interfaces +------------------+--------------------------------------+ ************************************* -Configuring Cinder on Controller Disk +Configuring Cinder on controller disk ************************************* Review the available disk space and capacity and obtain the uuid of the -physical disk +physical disk: :: @@ -401,7 +398,7 @@ physical disk | | | | | | |... +--------------------------------------+-----------+---------+---------+---------+------------+... -Create the 'cinder-volumes' local volume group +Create the 'cinder-volumes' local volume group: :: @@ -424,7 +421,7 @@ Create the 'cinder-volumes' local volume group | parameters | {u'lvm_type': u'thin'} | +-----------------+--------------------------------------+ -Create a disk partition to add to the volume group +Create a disk partition to add to the volume group: :: @@ -448,7 +445,7 @@ Create a disk partition to add to the volume group | updated_at | None | +-------------+--------------------------------------------------+ -Wait for the new partition to be created (i.e. status=Ready) +Wait for the new partition to be created (i.e. status=Ready): :: @@ -462,7 +459,7 @@ Wait for the new partition to be created (i.e. status=Ready) | |...| |...| | | | +--------------------------------------+...+------------+...+---------------------+----------+--------+ -Add the partition to the volume group +Add the partition to the volume group: :: @@ -488,10 +485,10 @@ Add the partition to the volume group +--------------------------+--------------------------------------------------+ ********************************************* -Adding an LVM Storage Backend at Installation +Adding an LVM storage backend at installation ********************************************* -Ensure requirements are met to add LVM storage +Ensure requirements are met to add LVM storage: :: @@ -505,7 +502,7 @@ Ensure requirements are met to add LVM storage storage. Set the 'confirmed' field to execute this operation for the lvm backend. -Add the LVM storage backend +Add the LVM storage backend: :: @@ -521,8 +518,7 @@ Add the LVM storage backend | e2697426-2d79-4a83-beb7-2eafa9ceaee5 | lvm-store | lvm | configuring |...| cinder | {} | +--------------------------------------+------------+---------+-------------+...+----------+--------------+ -Wait for the LVM storage backend to be configured (i.e. -state=Configured) +Wait for the LVM storage backend to be configured (i.e. state=configured): :: @@ -535,11 +531,11 @@ state=Configured) +--------------------------------------+------------+---------+------------+------+----------+--------------+ *********************************************** -Configuring VM Local Storage on Controller Disk +Configuring VM local storage on controller disk *********************************************** Review the available disk space and capacity and obtain the uuid of the -physical disk +physical disk: :: @@ -559,7 +555,7 @@ physical disk | | | | | | |... +--------------------------------------+-----------+---------+---------+---------+------------+... -Create the 'nova-local' volume group +Create the 'nova-local' volume group: :: @@ -584,7 +580,7 @@ Create the 'nova-local' volume group | parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} | +-----------------+-------------------------------------------------------------------+ -Create a disk partition to add to the volume group +Create a disk partition to add to the volume group: :: @@ -608,7 +604,7 @@ Create a disk partition to add to the volume group | updated_at | None | +-------------+--------------------------------------------------+ -Wait for the new partition to be created (i.e. status=Ready) +Wait for the new partition to be created (i.e. status=Ready): :: @@ -622,7 +618,7 @@ Wait for the new partition to be created (i.e. status=Ready) | |...| |...| | | | +--------------------------------------+...+------------+...+---------------------+----------+--------+ -Add the partition to the volume group +Add the partition to the volume group: :: @@ -648,11 +644,11 @@ Add the partition to the volume group +--------------------------+--------------------------------------------------+ ********************** -Unlocking Controller-0 +Unlocking controller-0 ********************** You must unlock controller-0 so that you can use it to install -Controller-1. Use the system host-unlock command: +controller-1. Use the system host-unlock command: :: @@ -663,10 +659,10 @@ unavailable, and any ssh connections are dropped. To monitor the progress of the reboot, use the controller-0 console. **************************************** -Verifying the Controller-0 Configuration +Verifying the controller-0 configuration **************************************** -On Controller-0, acquire Keystone administrative privileges: +On controller-0, acquire Keystone administrative privileges: :: @@ -686,7 +682,7 @@ Verify that the controller-0 services are running: ... +-----+-------------------------------+--------------+----------------+ -Verify that controller-0 has controller and compute subfunctions +Verify that controller-0 has controller and compute subfunctions: :: @@ -705,18 +701,19 @@ Verify that controller-0 is unlocked, enabled, and available: +----+--------------+-------------+----------------+-------------+--------------+ ***************** -System Alarm List +System alarm list ***************** -When all nodes are Unlocked, Enabled and Available: check 'fm alarm-list' for issues. +When all nodes are unlocked, enabled, and available: check 'fm alarm-list' for +issues. -Your StarlingX deployment is now up and running with 1 Controller with Cinder Storage -and all OpenStack services up and running. You can now proceed with standard OpenStack -APIs, CLIs and/or Horizon to load Glance Images, configure Nova Flavors, configure -Neutron networks and launch Nova Virtual Machines. +Your StarlingX deployment is now up and running with one controller with Cinder +storage and all OpenStack services up and running. You can now proceed with +standard OpenStack APIs, CLIs and/or Horizon to load Glance images, configure +Nova Flavors, configure Neutron networks and launch Nova virtual machines. ---------------------- -Deployment Terminology +Deployment terminology ---------------------- .. include:: deployment_terminology.rst diff --git a/doc/source/installation_guide/deployment_terminology.rst b/doc/source/installation_guide/deployment_terminology.rst deleted file mode 100644 index 90a628eac..000000000 --- a/doc/source/installation_guide/deployment_terminology.rst +++ /dev/null @@ -1,119 +0,0 @@ -.. _incl-simplex-deployment-terminology: - -**All-In-One Controller Node** - A single physical node that provides a Controller Function, Compute - Function, and Storage Function. - -.. _incl-simplex-deployment-terminology-end: - - -.. _incl-standard-controller-deployment-terminology: - -**Controller Node / Function** - A node that runs Cloud Control Function for managing Cloud Resources. - - - Runs Cloud Control Functions for managing Cloud Resources. - - Runs all OpenStack Control Functions (e.g. managing Images, Virtual - Volumes, Virtual Network, and Virtual Machines). - - Can be part of a two-node HA Control Node Cluster for running Control - Functions either Active/Active or Active/Standby. - -**Compute ( & Network ) Node / Function** - A node that hosts applications in Virtual Machines using Compute Resources - such as CPU, Memory, and Disk. - - - Runs Virtual Switch for realizing virtual networks. - - Provides L3 Routing and NET Services. - -.. _incl-standard-controller-deployment-terminology-end: - - -.. _incl-dedicated-storage-deployment-terminology: - -**Storage Node / Function** - A node that contains a set of Disks (e.g. SATA, SAS, SSD, and/or NVMe). - - - Runs CEPH Distributed Storage Software. - - Part of an HA multi-node CEPH Storage Cluster supporting a replication - factor of two or three, Journal Caching, and Class Tiering. - - Provides HA Persistent Storage for Images, Virtual Volumes - (i.e. Block Storage), and Object Storage. - -.. _incl-dedicated-storage-deployment-terminology-end: - -.. _incl-common-deployment-terminology: - -**OAM Network** - The network on which all external StarlingX Platform APIs are exposed, - (i.e. REST APIs, Horizon Web Server, SSH, and SNMP), typically 1GE. - - Only Controller type nodes are required to be connected to the OAM - Network. - -**Management Network** - A private network (i.e. not connected externally), tipically 10GE, - used for the following: - - - Internal OpenStack / StarlingX monitoring and control. - - VM I/O access to a storage cluster. - - All nodes are required to be connected to the Management Network. - -**Data Network(s)** - Networks on which the OpenStack / Neutron Provider Networks are realized - and become the VM Tenant Networks. - - Only Compute type and All-in-One type nodes are required to be connected - to the Data Network(s); these node types require one or more interface(s) - on the Data Network(s). - -**IPMI Network** - An optional network on which IPMI interfaces of all nodes are connected. - The network must be reachable using L3/IP from the Controller's OAM - Interfaces. - - You can optionally connect all node types to the IPMI Network. - -**PXEBoot Network** - An optional network for Controllers to boot/install other nodes over the - network. - - By default, Controllers use the Management Network for boot/install of other - nodes in the openstack cloud. If this optional network is used, all node - types are required to be connected to the PXEBoot Network. - - A PXEBoot network is required for a variety of special case situations: - - - Cases where the Management Network must be IPv6: - - - IPv6 does not support PXEBoot. Therefore, IPv4 PXEBoot network must be - configured. - - - Cases where the Management Network must be VLAN tagged: - - - Most Server's BIOS do not support PXEBooting over tagged networks. - Therefore, you must configure an untagged PXEBoot network. - - - Cases where a Management Network must be shared across regions but - individual regions' Controllers want to only network boot/install nodes - of their own region: - - - You must configure separate, per-region PXEBoot Networks. - -**Infra Network** - A deprecated optional network that was historically used for access to the - Storage cluster. - - If this optional network is used, all node types are required to be - connected to the INFRA Network, - -**Node Interfaces** - All Nodes' Network Interfaces can, in general, optionally be either: - - - Untagged single port. - - Untagged two-port LAG and optionally split between redudant L2 Switches - running vPC (Virtual Port-Channel), also known as multichassis - EtherChannel (MEC). - - VLAN on either single-port ETH interface or two-port LAG interface. - -.. _incl-common-deployment-terminology-end: diff --git a/doc/source/installation_guide/index.rst b/doc/source/installation_guide/index.rst index 5e19d0454..d8a1a0ad1 100644 --- a/doc/source/installation_guide/index.rst +++ b/doc/source/installation_guide/index.rst @@ -1,287 +1,38 @@ -================== -Installation Guide -================== +=================== +Installation guides +=================== ------ -Intro ------ - -StarlingX may be installed in: - -- **Bare Metal**: Real deployments of StarlingX are only supported on - physical servers. -- **Virtual Environment**: It should only be used for evaluation or - development purposes. - - -StarlingX installed in virtual environments has two options: - -- :ref:`Libvirt/QEMU ` -- VirtualBox - ------------- -Requirements ------------- - -Different use cases require different configurations. - -********** -Bare Metal -********** - -The minimum requirements for the physical servers where StarlingX might -be deployed, include: - -- **Controller Hosts** - - - Minimum Processor is: - - - Dual-CPU Intel® Xeon® E5 26xx Family (SandyBridge) 8 - cores/socket - - - Minimum Memory: 64 GB - - Hard Drives: - - - Primary Hard Drive, minimum 500 GB for OS and system databases. - - Secondary Hard Drive, minimum 500 GB for persistent VM storage. - - - 2 physical Ethernet interfaces: OAM and MGMT Network. - - USB boot support. - - PXE boot support. - -- **Storage Hosts** - - - Minimum Processor is: - - - Dual-CPU Intel® Xeon® E5 26xx Family (SandyBridge) 8 - cores/socket. - - - Minimum Memory: 64 GB. - - Hard Drives: - - - Primary Hard Drive, minimum 500 GB for OS. - - 1 or more additional Hard Drives for CEPH OSD storage, and - - Optionally 1 or more SSD or NVMe Drives for CEPH Journals. - - - 1 physical Ethernet interface: MGMT Network - - PXE boot support. - -- **Compute Hosts** - - - Minimum Processor is: - - - Dual-CPU Intel® Xeon® E5 26xx Family (SandyBridge) 8 - cores/socket. - - - Minimum Memory: 32 GB. - - Hard Drives: - - - Primary Hard Drive, minimum 500 GB for OS. - - 1 or more additional Hard Drives for ephemeral VM Storage. - - - 2 or more physical Ethernet interfaces: MGMT Network and 1 or more - Provider Networks. - - PXE boot support. - -- **All-In-One Simplex or Duplex, Controller + Compute Hosts** - - - Minimum Processor is: - - - Typical Hardware Form Factor: - - - Dual-CPU Intel® Xeon® E5 26xx Family (SandyBridge) 8 cores/socket - - Low Cost / Low Power Hardware Form Factor - - - Single-CPU Intel Xeon D-15xx Family, 8 cores - - - Minimum Memory: 64 GB. - - Hard Drives: - - - Primary Hard Drive, minimum 500 GB SDD or NVMe. - - 0 or more 500 GB disks (min. 10K RPM). - - - Network Ports: - - **NOTE:** Duplex and Simplex configurations require one or more data - ports. - The Duplex configuration requires a management port. - - - Management: 10GE (Duplex only) - - OAM: 10GE - - Data: n x 10GE - -The recommended minimum requirements for the physical servers are -described later in each StarlingX Deployment Options guide. - -^^^^^^^^^^^^^^^^^^^^^^^^ -NVMe Drive as Boot Drive -^^^^^^^^^^^^^^^^^^^^^^^^ - -To use a Non-Volatile Memory Express (NVMe) drive as the boot drive for any of -your nodes, you must configure your host and adjust kernel parameters during -installation: - -- Configure the host to be in UEFI mode. -- Edit the kernel boot parameter. After you are presented with the StarlingX - ISO boot options and after you have selected the preferred installation option - (e.g. Standard Configuration / All-in-One Controller Configuration), press the - TAB key to edit the Kernel boot parameters. Modify the **boot_device** and - **rootfs_device** from the default **sda** so that it is the correct device - name for the NVMe drive (e.g. "nvme0n1"). - - :: - - vmlinuz rootwait console=tty0 inst.text inst.stage2=hd:LABEL=oe_iso_boot - inst.ks=hd:LABEL=oe_iso_boot:/smallsystem_ks.cfg boot_device=nvme0n1 - rootfs_device=nvme0n1 biosdevname=0 usbcore.autosuspend=-1 inst.gpt - security_profile=standard user_namespace.enable=1 initrd=initrd.img - - -******************* -Virtual Environment -******************* - -The recommended minimum requirements for the workstation, hosting the -Virtual Machine(s) where StarlingX will be deployed, include: - -^^^^^^^^^^^^^^^^^^^^^ -Hardware Requirements -^^^^^^^^^^^^^^^^^^^^^ - -A workstation computer with: - -- Processor: x86_64 only supported architecture with BIOS enabled - hardware virtualization extensions -- Cores: 8 (4 with careful monitoring of cpu load) -- Memory: At least 32GB RAM -- Hard Disk: 500GB HDD -- Network: Two network adapters with active Internet connection - -^^^^^^^^^^^^^^^^^^^^^ -Software Requirements -^^^^^^^^^^^^^^^^^^^^^ - -A workstation computer with: - -- Operating System: Freshly installed Ubuntu 16.04 LTS 64-bit -- Proxy settings configured (if applies) -- Git -- KVM/VirtManager -- Libvirt Library -- QEMU Full System Emulation Binaries -- stx-tools project -- StarlingX ISO Image - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Deployment Environment Setup -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This section describes how to set up the workstation computer which will -host the Virtual Machine(s) where StarlingX will be deployed. - -'''''''''''''''''''''''''''''' -Updating Your Operating System -'''''''''''''''''''''''''''''' - -Before proceeding with the build, ensure your OS is up to date. You’ll -first need to update the local database list of available packages: - -:: - - $ sudo apt-get update - -''''''''''''''''''''''''' -Install stx-tools project -''''''''''''''''''''''''' - -Clone the stx-tools project. Usually you’ll want to clone it under your -user’s home directory. - -:: - - $ cd $HOME - $ git clone https://git.starlingx.io/stx-tools - - -'''''''''''''''''''''''''''''''''''''''' -Installing Requirements and Dependencies -'''''''''''''''''''''''''''''''''''''''' - -Navigate to the stx-tools installation libvirt directory: - -:: - - $ cd $HOME/stx-tools/deployment/libvirt/ - - -Install the required packages: - -:: - - $ bash install_packages.sh - - -'''''''''''''''''' -Disabling Firewall -'''''''''''''''''' - -Unload firewall and disable firewall on boot: - -:: - - $ sudo ufw disable - Firewall stopped and disabled on system startup - $ sudo ufw status - Status: inactive - - -------------------------------- -Getting the StarlingX ISO Image -------------------------------- - -Follow the instructions from the :ref:`developer-guide` to build a -StarlingX ISO image. - - -********** -Bare Metal -********** - -A bootable USB flash drive containing StarlingX ISO image. - - -******************* -Virtual Environment -******************* - -Copy the StarlingX ISO Image to the stx-tools deployment libvirt project -directory: - -:: - - $ cp $HOME/stx-tools/deployment/libvirt/ - - ------------------- -Deployment Options ------------------- - -- Standard Controller - - - :ref:`StarlingX Cloud with Dedicated Storage ` - - :ref:`StarlingX Cloud with Controller Storage ` - -- All-in-one - - - :ref:`StarlingX Cloud Duplex ` - - :ref:`StarlingX Cloud Simplex ` +Installation steps for StarlingX are release specific. To install the +latest release of StarlingX, use the :doc:`/installation_guide/2018_10/index`. +To install a previous release of StarlingX, use the installation guide +for your specific release: .. toctree:: - :hidden: + :maxdepth: 1 - installation_libvirt_qemu - controller_storage - dedicated_storage - duplex - simplex + /installation_guide/latest/index + /installation_guide/2018_10/index + +.. How to add a new release (installer and developer guides): + 1. Archive previous release + 1. Rename old 'latest' folder to the release name e.g. Year_Month + 2. Update links in old 'latest' to use new path e.g. + :doc:`Libvirt/QEMU ` + becomes + :doc:`Libvirt/QEMU ` + 2. Add new release + 1. Add a new 'latest' dir and add the new version - likely this will be a copy of the previous version, with updates applied + 2. Make sure the new files have the correct version in the page title and intro sentence e.g. '2018.10.rc1 Installation Guide' + 3. Make sure all files in new 'latest' link to the correct versions of supporting docs (do this via doc link, so that it goes to top of page e.g. :doc:`/installation_guide/latest/index`) + 4. Make sure the new release index is labeled with the correct version name e.g + .. _index-2019-05: + 3. Add the archived version to the toctree on this page + 4. If adding a new version *before* it is available (e.g. to begin work on new docs), + make sure page text still directs user to the *actual* current release, not the + future-not-yet-released version. + 5. When the release is *actually* available, make sure to update these pages: + - index + - installation guide + - developer guide + - release notes diff --git a/doc/source/installation_guide/latest/controller_storage.rst b/doc/source/installation_guide/latest/controller_storage.rst new file mode 100644 index 000000000..1e8dc4912 --- /dev/null +++ b/doc/source/installation_guide/latest/controller_storage.rst @@ -0,0 +1,974 @@ +=============================================== +Controller storage deployment guide stx.2019.05 +=============================================== + +.. contents:: + :local: + :depth: 1 + +**NOTE:** The instructions to setup a StarlingX Cloud with Controller +Storage with containerized openstack services in this guide +are under development. +For approved instructions, see the +`StarlingX Cloud with Controller Storage wiki page `__. + +---------------------- +Deployment description +---------------------- + +The Controller Storage deployment option provides a 2x node high availability +controller / storage cluster with: + +- A pool of up to seven compute nodes (pool size limit due to the capacity of + the storage function). +- A growth path for storage to the full standard solution with an independent + CEPH storage cluster. +- High availability services runnning across the controller nodes in either + active/active or active/standby mode. +- Storage function running on top of LVM on single second disk, DRBD-sync'd + between the controller nodes. + +.. figure:: figures/starlingx-deployment-options-controller-storage.png + :scale: 50% + :alt: Controller Storage deployment configuration + + *Controller Storage deployment configuration* + +A Controller Storage deployment provides protection against overall controller +node and compute node failure: + +- On overall controller node failure, all controller high availability services + go active on the remaining healthy controller node. +- On overall compute node failure, virtual machines on failed compute node are + recovered on the remaining healthy compute nodes. + +------------------------------------ +Preparing controller storage servers +------------------------------------ + +********** +Bare metal +********** + +Required servers: + +- Controllers: 2 +- Computes: 2 - 100 + +^^^^^^^^^^^^^^^^^^^^^ +Hardware requirements +^^^^^^^^^^^^^^^^^^^^^ + +The recommended minimum requirements for the physical servers where +Controller Storage will be deployed, include: + +- Minimum processor: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket + +- Memory: + + - 64 GB controller + - 32 GB compute + +- BIOS: + + - Hyper-Threading technology: Enabled + - Virtualization technology: Enabled + - VT for directed I/O: Enabled + - CPU power and performance policy: Performance + - CPU C state control: Disabled + - Plug & play BMC detection: Disabled + +- Primary disk: + + - 500 GB SDD or NVMe controller + - 120 GB (min. 10K RPM) compute + +- Additional disks: + + - 1 or more 500 GB disks (min. 10K RPM) compute + +- Network ports\* + + - Management: 10GE controller, compute + - OAM: 10GE controller + - Data: n x 10GE compute + +******************* +Virtual environment +******************* + +Run the libvirt qemu setup scripts. Setting up virtualized OAM and +management networks: + +:: + + $ bash setup_network.sh + +Building XML for definition of virtual servers: + +:: + + $ bash setup_configuration.sh -c controllerstorage -i + +The default XML server definitions that are created by the previous script +are: + +- controllerstorage-controller-0 +- controllerstorage-controller-1 +- controllerstorage-compute-0 +- controllerstorage-compute-1 + +^^^^^^^^^^^^^^^^^^^^^^^^^ +Power up a virtual server +^^^^^^^^^^^^^^^^^^^^^^^^^ + +To power up a virtual server, run the following command: + +:: + + $ sudo virsh start + +e.g. + +:: + + $ sudo virsh start controllerstorage-controller-0 + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Access virtual server consoles +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The XML for virtual servers in stx-tools repo, deployment/libvirt, +provides both graphical and text consoles. + +Access the graphical console in virt-manager by right-click on the +domain (the server) and selecting "Open". + +Access the textual console with the command "virsh console $DOMAIN", +where DOMAIN is the name of the server shown in virsh. + +When booting the controller-0 for the first time, both the serial and +graphical consoles will present the initial configuration menu for the +cluster. One can select serial or graphical console for controller-0. +For the other nodes however only serial is used, regardless of which +option is selected. + +Open the graphic console on all servers before powering them on to +observe the boot device selection and PXI boot progress. Run "virsh +console $DOMAIN" command promptly after power on to see the initial boot +sequence which follows the boot device selection. One has a few seconds +to do this. + +-------------------------------- +Installing the controller-0 host +-------------------------------- + +Installing controller-0 involves initializing a host with software and +then applying a bootstrap configuration from the command line. The +configured bootstrapped host becomes controller-0. + +Procedure: + +#. Power on the server that will be controller-0 with the StarlingX ISO + on a USB in a bootable USB slot. +#. Configure the controller using the config_controller script. + +************************* +Initializing controller-0 +************************* + +This section describes how to initialize StarlingX in host controller-0. +Except where noted, all the commands must be executed from a console of +the host. + +Power on the host to be configured as controller-0, with the StarlingX +ISO on a USB in a bootable USB slot. Wait for the console to show the +StarlingX ISO booting options: + +- **Standard Controller Configuration** + + - When the installer is loaded and the installer welcome screen + appears in the controller-0 host, select the type of installation + "Standard Controller Configuration". + +- **Graphical Console** + + - Select the "Graphical Console" as the console to use during + installation. + +- **Standard Security Boot Profile** + + - Select "Standard Security Boot Profile" as the security profile. + +Monitor the initialization. When it is complete, a reboot is initiated +on the controller-0 host, briefly displays a GNU GRUB screen, and then +boots automatically into the StarlingX image. + +Log into controller-0 as user wrsroot, with password wrsroot. The +first time you log in as wrsroot, you are required to change your +password. Enter the current password (wrsroot): + +:: + + Changing password for wrsroot. + (current) UNIX Password: + +Enter a new password for the wrsroot account: + +:: + + New password: + +Enter the new password again to confirm it: + +:: + + Retype new password: + +controller-0 is initialized with StarlingX, and is ready for configuration. + +************************ +Configuring controller-0 +************************ + +This section describes how to perform the controller-0 configuration +interactively just to bootstrap system with minimum critical data. +Except where noted, all the commands must be executed from the console +of the active controller (here assumed to be controller-0). + +When run interactively, the config_controller script presents a series +of prompts for initial configuration of StarlingX: + +- For the virtual environment, you can accept all the default values + immediately after ‘system date and time’. +- For a physical deployment, answer the bootstrap configuration + questions with answers applicable to your particular physical setup. + +The script is used to configure the first controller in the StarlingX +cluster as controller-0. The prompts are grouped by configuration +area. To start the script interactively, use the following command +with no parameters: + +:: + + controller-0:~$ sudo config_controller + System Configuration + ================ + Enter ! at any prompt to abort... + ... + +Accept all the default values immediately after ‘system date and time’. + +:: + + ... + Applying configuration (this will take several minutes): + + 01/08: Creating bootstrap configuration ... DONE + 02/08: Applying bootstrap manifest ... DONE + 03/08: Persisting local configuration ... DONE + 04/08: Populating initial system inventory ... DONE + 05:08: Creating system configuration ... DONE + 06:08: Applying controller manifest ... DONE + 07:08: Finalize controller configuration ... DONE + 08:08: Waiting for service activation ... DONE + + Configuration was applied + + Please complete any out of service commissioning steps with system commands and unlock controller to proceed. + +After config_controller bootstrap configuration, REST API, CLI and +Horizon interfaces are enabled on the controller-0 OAM IP address. The +remaining installation instructions will use the CLI. + +------------------------------------ +Provisioning controller-0 and system +------------------------------------ + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +********************************************* +Configuring provider networks at installation +********************************************* + +You must set up provider networks at installation so that you can attach +data interfaces and unlock the compute nodes. + +Set up one provider network of the vlan type, named providernet-a: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-create providernet-a --type=vlan + [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-range-create --name providernet-a-range1 --range 100-400 providernet-a + +************************************* +Configuring Cinder on controller disk +************************************* + +Review the available disk space and capacity and obtain the uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 + +--------------------------------------+-----------+---------+---------+-------+------------+--------------+... + | uuid | device_no | device_ | device_ | size_ | available_ | rpm |... + | | de | num | type | gib | gib | |... + +--------------------------------------+-----------+---------+---------+-------+------------+--------------+... + | 004f4c09-2f61-46c5-8def-99b2bdeed83c | /dev/sda | 2048 | HDD | 200.0 | 0.0 | |... + | 89694799-0dd8-4532-8636-c0d8aabfe215 | /dev/sdb | 2064 | HDD | 200.0 | 199.997 | |... + +--------------------------------------+-----------+---------+---------+-------+------------+--------------+... + +Create the 'cinder-volumes' local volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-0 cinder-volumes + +-----------------+--------------------------------------+ + | Property | Value | + +-----------------+--------------------------------------+ + | lvm_vg_name | cinder-volumes | + | vg_state | adding | + | uuid | ece4c755-241c-4363-958e-85e9e3d12917 | + | ihost_uuid | 150284e2-fb60-4169-ae75-7f444b8ca9bf | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size_gib | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-22T03:59:30.685718+00:00 | + | updated_at | None | + | parameters | {u'lvm_type': u'thin'} | + +-----------------+--------------------------------------+ + +Create a disk partition to add to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-0 89694799-0dd8-4532-8636-c0d8aabfe215 199 -t lvm_phys_vol + +-------------+--------------------------------------------------+ + | Property | Value | + +-------------+--------------------------------------------------+ + | device_path | /dev/disk/by-path/pci-0000:00:03.0-ata-2.0-part1 | + | device_node | /dev/sdb1 | + | type_guid | ba5eba11-0000-1111-2222-000000000001 | + | type_name | None | + | start_mib | None | + | end_mib | None | + | size_mib | 203776 | + | uuid | 9ba2d76a-6ae2-4bfa-ad48-57b62d102e80 | + | ihost_uuid | 150284e2-fb60-4169-ae75-7f444b8ca9bf | + | idisk_uuid | 89694799-0dd8-4532-8636-c0d8aabfe215 | + | ipv_uuid | None | + | status | Creating | + | created_at | 2018-08-22T04:03:40.761221+00:00 | + | updated_at | None | + +-------------+--------------------------------------------------+ + +Wait for the new partition to be created (i.e. status=Ready): + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-0 --disk 89694799-0dd8-4532-8636-c0d8aabfe215 + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + | uuid |...| device_nod |...| type_name | size_mib | status | + | |...| e |...| | | | + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + | 9ba2d76a-6ae2-4bfa-ad48-57b62d102e80 |...| /dev/sdb1 |...| LVM Physical Volume | 199.0 | Ready | + | |...| |...| | | | + | |...| |...| | | | + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + +Add the partition to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-0 cinder-volumes 9ba2d76a-6ae2-4bfa-ad48-57b62d102e80 + +--------------------------+--------------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------------+ + | uuid | 060dc47e-bc17-40f4-8f09-5326ef0e86a5 | + | pv_state | adding | + | pv_type | partition | + | disk_or_part_uuid | 9ba2d76a-6ae2-4bfa-ad48-57b62d102e80 | + | disk_or_part_device_node | /dev/sdb1 | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:03.0-ata-2.0-part1 | + | lvm_pv_name | /dev/sdb1 | + | lvm_vg_name | cinder-volumes | + | lvm_pv_uuid | None | + | lvm_pv_size_gib | 0.0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | 150284e2-fb60-4169-ae75-7f444b8ca9bf | + | created_at | 2018-08-22T04:06:54.008632+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------------+ + +Enable LVM backend: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add lvm -s cinder --confirmed + +Wait for the storage backend to leave "configuring" state. Confirm LVM +backend storage is configured: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-list + +--------------------------------------+------------+---------+------------+------+----------+... + | uuid | name | backend | state | task | services |... + +--------------------------------------+------------+---------+------------+------+----------+... + | 1daf3e5b-4122-459f-9dba-d2e92896e718 | file-store | file | configured | None | glance |... + | a4607355-be7e-4c5c-bf87-c71a0e2ad380 | lvm-store | lvm | configured | None | cinder |... + +--------------------------------------+------------+---------+------------+------+----------+... + +********************** +Unlocking controller-0 +********************** + +You must unlock controller-0 so that you can use it to install the +remaining hosts. On controller-0, acquire Keystone administrative +privileges. Use the system host-unlock command: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-0 + +The host is rebooted. During the reboot, the command line is +unavailable, and any ssh connections are dropped. To monitor the +progress of the reboot, use the controller-0 console. + +**************************************** +Verifying the controller-0 configuration +**************************************** + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +Verify that the StarlingX controller services are running: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system service-list + +-----+-------------------------------+--------------+----------------+ + | id | service_name | hostname | state | + +-----+-------------------------------+--------------+----------------+ + ... + | 1 | oam-ip | controller-0 | enabled-active | + | 2 | management-ip | controller-0 | enabled-active | + ... + +-----+-------------------------------+--------------+----------------+ + +Verify that controller-0 is unlocked, enabled, and available: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + +----+--------------+-------------+----------------+-------------+--------------+ + +--------------------------------------- +Installing controller-1 / compute hosts +--------------------------------------- + +After initializing and configuring an active controller, you can add and +configure a backup controller and additional compute hosts. For each +host do the following: + +***************** +Initializing host +***************** + +Power on Host. In host console you will see: + +:: + + Waiting for this node to be configured. + + Please configure the personality for this node from the + controller node in order to proceed. + +*************************************** +Updating host hostname and personality +*************************************** + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +Wait for controller-0 to discover new host, list the host until new +UNKNOWN host shows up in table: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +Use the system host-update to update host personality attribute: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-update 2 personality=controller hostname=controller-1 + +Or for compute-0: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-update 3 personality=compute hostname=compute-0 + +See also: 'system help host-update'. + +Unless it is known that the host's configuration can support the installation of +more than one node, it is recommended that the installation and configuration of +each node be serialized. For example, if the entire cluster has its virtual +disks hosted on the host's root disk which happens to be a single rotational +type hard disk, then the host cannot (reliably) support parallel node +installation. + +*************** +Monitoring host +*************** + +On controller-0, you can monitor the installation progress by running the system +host-show command for the host periodically. Progress is shown in the +install_state field: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-show | grep install + | install_output | text | + | install_state | booting | + | install_state_info | None | + +Wait while the host is configured and rebooted. Up to 20 minutes may be required +for a reboot, depending on hardware. When the reboot is complete, the host is +reported as locked, disabled, and online. + +************* +Listing hosts +************* + +Once all nodes have been installed, configured and rebooted, on controller-0 +list the hosts: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | locked | disabled | online | + | 3 | compute-0 | compute | locked | disabled | online | + | 4 | compute-1 | compute | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +------------------------- +Provisioning controller-1 +------------------------- + +On controller-0, list hosts: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + ... + | 2 | controller-1 | controller | locked | disabled | online | + ... + +----+--------------+-------------+----------------+-------------+--------------+ + +*********************************************** +Provisioning network interfaces on controller-1 +*********************************************** + +In order to list out hardware port names, types, PCI addresses that have +been discovered: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list controller-1 + +Provision the OAM interface for controller-1: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -n -c platform --networks oam controller-1 + +************************************ +Provisioning storage on controller-1 +************************************ + +Review the available disk space and capacity and obtain the uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 + +--------------------------------------+-----------+---------+---------+-------+------------+ + | uuid | device_no | device_ | device_ | size_ | available_ | + | | de | num | type | gib | gib | + +--------------------------------------+-----------+---------+---------+-------+------------+ + | f7ce53db-7843-457e-8422-3c8f9970b4f2 | /dev/sda | 2048 | HDD | 200.0 | 0.0 | + | 70b83394-968e-4f0d-8a99-7985cd282a21 | /dev/sdb | 2064 | HDD | 200.0 | 199.997 | + +--------------------------------------+-----------+---------+---------+-------+------------+ + +Assign Cinder storage to the physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-1 cinder-volumes + +-----------------+--------------------------------------+ + | Property | Value | + +-----------------+--------------------------------------+ + | lvm_vg_name | cinder-volumes | + | vg_state | adding | + | uuid | 22d8b94a-200a-4fd5-b1f5-7015ddf10d0b | + | ihost_uuid | 06827025-eacb-45e6-bb88-1a649f7404ec | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size_gib | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-22T05:33:44.608913+00:00 | + | updated_at | None | + | parameters | {u'lvm_type': u'thin'} | + +-----------------+--------------------------------------+ + +Create a disk partition to add to the volume group based on uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-1 70b83394-968e-4f0d-8a99-7985cd282a21 199 -t lvm_phys_vol + +-------------+--------------------------------------------------+ + | Property | Value | + +-------------+--------------------------------------------------+ + | device_path | /dev/disk/by-path/pci-0000:00:03.0-ata-2.0-part1 | + | device_node | /dev/sdb1 | + | type_guid | ba5eba11-0000-1111-2222-000000000001 | + | type_name | None | + | start_mib | None | + | end_mib | None | + | size_mib | 203776 | + | uuid | 16a1c5cb-620c-47a3-be4b-022eafd122ee | + | ihost_uuid | 06827025-eacb-45e6-bb88-1a649f7404ec | + | idisk_uuid | 70b83394-968e-4f0d-8a99-7985cd282a21 | + | ipv_uuid | None | + | status | Creating (on unlock) | + | created_at | 2018-08-22T05:36:42.123770+00:00 | + | updated_at | None | + +-------------+--------------------------------------------------+ + +Wait for the new partition to be created (i.e. status=Ready): + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-1 --disk 70b83394-968e-4f0d-8a99-7985cd282a21 + +--------------------------------------+...+------------+...+-------+--------+----------------------+ + | uuid |...| device_nod | ... | size_g | status | + | |...| e | ... | ib | | + +--------------------------------------+...+------------+ ... +--------+----------------------+ + | 16a1c5cb-620c-47a3-be4b-022eafd122ee |...| /dev/sdb1 | ... | 199.0 | Creating (on unlock) | + | |...| | ... | | | + | |...| | ... | | | + +--------------------------------------+...+------------+...+--------+----------------------+ + +Add the partition to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-1 cinder-volumes 16a1c5cb-620c-47a3-be4b-022eafd122ee + +--------------------------+--------------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------------+ + | uuid | 01d79ed2-717f-428e-b9bc-23894203b35b | + | pv_state | adding | + | pv_type | partition | + | disk_or_part_uuid | 16a1c5cb-620c-47a3-be4b-022eafd122ee | + | disk_or_part_device_node | /dev/sdb1 | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:03.0-ata-2.0-part1 | + | lvm_pv_name | /dev/sdb1 | + | lvm_vg_name | cinder-volumes | + | lvm_pv_uuid | None | + | lvm_pv_size_gib | 0.0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | 06827025-eacb-45e6-bb88-1a649f7404ec | + | created_at | 2018-08-22T05:44:34.715289+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------------+ + +********************** +Unlocking controller-1 +********************** + +Unlock controller-1: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-1 + +Wait while the controller-1 is rebooted. Up to 10 minutes may be required for a +reboot, depending on hardware. + +**REMARK:** controller-1 will remain in 'degraded' state until data-syncing is +complete. The duration is dependant on the virtualization host's configuration - +i.e., the number and configuration of physical disks used to host the nodes' +virtual disks. Also, the management network is expected to have link capacity of +10000 (1000 is not supported due to excessive data-sync time). Use +'fm alarm-list' to confirm status. + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | unlocked | enabled | available | + ... + +--------------------------- +Provisioning a compute host +--------------------------- + +You must configure the network interfaces and the storage disks on a +host before you can unlock it. For each compute host do the following: + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +************************************************* +Provisioning network interfaces on a compute host +************************************************* + +On controller-0, in order to list out hardware port names, types, +pci-addresses that have been discovered: + +- **Only in virtual environment**: Ensure that the interface used is + one of those attached to host bridge with model type "virtio" (i.e., + eth1000 and eth1001). The model type "e1000" emulated devices will + not work for provider networks: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list compute-0 + +Provision the data interface for compute: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -p providernet-a -c data compute-0 eth1000 + +*************************** +VSwitch virtual environment +*************************** + +**Only in virtual environment**. If the compute has more than 4 cpus, the system +will auto-configure the vswitch to use 2 cores. However some virtual +environments do not properly support multi-queue required in a multi-CPU +environment. Therefore run the following command to reduce the vswitch cores to +1: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-cpu-modify compute-0 -f vswitch -p0 1 + +--------------------------------------+-------+-----------+-------+--------+... + | uuid | log_c | processor | phy_c | thread |... + | | ore | | ore | |... + +--------------------------------------+-------+-----------+-------+--------+... + | a3b5620c-28b1-4fe0-9e97-82950d8582c2 | 0 | 0 | 0 | 0 |... + | f2e91c2b-bfc5-4f2a-9434-bceb7e5722c3 | 1 | 0 | 1 | 0 |... + | 18a98743-fdc4-4c0c-990f-3c1cb2df8cb3 | 2 | 0 | 2 | 0 |... + | 690d25d2-4f99-4ba1-a9ba-0484eec21cc7 | 3 | 0 | 3 | 0 |... + +--------------------------------------+-------+-----------+-------+--------+... + +************************************** +Provisioning storage on a compute host +************************************** + +Review the available disk space and capacity and obtain the uuid(s) of +the physical disk(s) to be used for nova local: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list compute-0 + +--------------------------------------+-----------+---------+---------+-------+------------+... + | uuid | device_no | device_ | device_ | size_ | available_ |... + | | de | num | type | gib | gib |... + +--------------------------------------+-----------+---------+---------+-------+------------+... + | 8a9d2c09-d3a7-4781-bd06-f7abf603713a | /dev/sda | 2048 | HDD | 200.0 | 172.164 |... + | 5ad61bd1-795a-4a76-96ce-39433ef55ca5 | /dev/sdb | 2064 | HDD | 200.0 | 199.997 |... + +--------------------------------------+-----------+---------+---------+-------+------------+... + +Create the 'nova-local' local volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add compute-0 nova-local + +-----------------+-------------------------------------------------------------------+ + | Property | Value | + +-----------------+-------------------------------------------------------------------+ + | lvm_vg_name | nova-local | + | vg_state | adding | + | uuid | 18898640-c8b7-4bbd-a323-4bf3e35fee4d | + | ihost_uuid | da1cbe93-cec5-4f64-b211-b277e4860ab3 | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size_gib | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-22T08:00:51.945160+00:00 | + | updated_at | None | + | parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} | + +-----------------+-------------------------------------------------------------------+ + +Create a disk partition to add to the volume group based on uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add compute-0 nova-local 5ad61bd1-795a-4a76-96ce-39433ef55ca5 + +--------------------------+--------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------+ + | uuid | 4c81745b-286a-4850-ba10-305e19cee78c | + | pv_state | adding | + | pv_type | disk | + | disk_or_part_uuid | 5ad61bd1-795a-4a76-96ce-39433ef55ca5 | + | disk_or_part_device_node | /dev/sdb | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:03.0-ata-2.0 | + | lvm_pv_name | /dev/sdb | + | lvm_vg_name | nova-local | + | lvm_pv_uuid | None | + | lvm_pv_size_gib | 0.0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | da1cbe93-cec5-4f64-b211-b277e4860ab3 | + | created_at | 2018-08-22T08:07:14.205690+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------+ + +Specify the local storage space as local copy-on-write image volumes in +nova-local: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-modify -b image -s 10240 compute-0 nova-local + +-----------------+-------------------------------------------------------------------+ + | Property | Value | + +-----------------+-------------------------------------------------------------------+ + | lvm_vg_name | nova-local | + | vg_state | adding | + | uuid | 18898640-c8b7-4bbd-a323-4bf3e35fee4d | + | ihost_uuid | da1cbe93-cec5-4f64-b211-b277e4860ab3 | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size_gib | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-22T08:00:51.945160+00:00 | + | updated_at | None | + | parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} | + +-----------------+-------------------------------------------------------------------+ + +************************ +Unlocking a compute host +************************ + +On controller-0, use the system host-unlock command to unlock the compute node: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock compute-0 + +Wait while the compute node is rebooted. Up to 10 minutes may be +required for a reboot, depending on hardware. The host is rebooted, and +its availability state is reported as in-test, followed by +unlocked/enabled. + +------------------- +System health check +------------------- + +*********************** +Listing StarlingX nodes +*********************** + +On controller-0, after a few minutes, all nodes shall be reported as +unlocked, enabled, and available: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | unlocked | enabled | available | + | 3 | compute-0 | compute | unlocked | enabled | available | + | 4 | compute-1 | compute | unlocked | enabled | available | + +----+--------------+-------------+----------------+-------------+--------------+ + +***************** +System alarm-list +***************** + +When all nodes are unlocked, enabled and available: check 'fm alarm-list' for +issues. + +Your StarlingX deployment is now up and running with 2x HA controllers with +Cinder storage, 2x computes, and all OpenStack services up and running. You can +now proceed with standard OpenStack APIs, CLIs and/or Horizon to load Glance +images, configure Nova Flavors, configure Neutron networks and launch Nova +virtual machines. + +---------------------- +Deployment terminology +---------------------- + +.. include:: deployment_terminology.rst + :start-after: incl-standard-controller-deployment-terminology: + :end-before: incl-standard-controller-deployment-terminology-end: + +.. include:: deployment_terminology.rst + :start-after: incl-common-deployment-terminology: + :end-before: incl-common-deployment-terminology-end: diff --git a/doc/source/installation_guide/latest/dedicated_storage.rst b/doc/source/installation_guide/latest/dedicated_storage.rst new file mode 100644 index 000000000..42efea424 --- /dev/null +++ b/doc/source/installation_guide/latest/dedicated_storage.rst @@ -0,0 +1,918 @@ +============================================== +Dedicated storage deployment guide stx.2019.05 +============================================== + +.. contents:: + :local: + :depth: 1 + +**NOTE:** The instructions to setup a StarlingX Cloud with Dedicated +Storage with containerized openstack services in this guide +are under development. +For approved instructions, see the +`StarlingX Cloud with Dedicated Storage wiki page `__. + +---------------------- +Deployment description +---------------------- + +Cloud with Dedicated Storage is the standard StarlingX deployment option with +independent controller, compute, and storage nodes. + +This deployment option provides the maximum capacity for a single region +deployment, with a supported growth path to a multi-region deployment option by +adding a secondary region. + +.. figure:: figures/starlingx-deployment-options-dedicated-storage.png + :scale: 50% + :alt: Dedicated Storage deployment configuration + + *Dedicated Storage deployment configuration* + +Cloud with Dedicated Storage includes: + +- 2x node HA controller cluster with HA services running across the controller + nodes in either active/active or active/standby mode. +- Pool of up to 100 compute nodes for hosting virtual machines and virtual + networks. +- 2-9x node HA CEPH storage cluster for hosting virtual volumes, images, and + object storage that supports a replication factor of 2 or 3. + + Storage nodes are deployed in replication groups of 2 or 3. Replication + of objects is done strictly within the replication group. + + Supports up to 4 groups of 2x storage nodes, or up to 3 groups of 3x storage + nodes. + +----------------------------------- +Preparing dedicated storage servers +----------------------------------- + +********** +Bare metal +********** + +Required Servers: + +- Controllers: 2 +- Storage + + - Replication factor of 2: 2 - 8 + - Replication factor of 3: 3 - 9 + +- Computes: 2 - 100 + +^^^^^^^^^^^^^^^^^^^^^ +Hardware requirements +^^^^^^^^^^^^^^^^^^^^^ + +The recommended minimum requirements for the physical servers where +Dedicated Storage will be deployed, include: + +- Minimum processor: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket + +- Memory: + + - 64 GB controller, storage + - 32 GB compute + +- BIOS: + + - Hyper-Threading technology: Enabled + - Virtualization technology: Enabled + - VT for directed I/O: Enabled + - CPU power and performance policy: Performance + - CPU C state control: Disabled + - Plug & play BMC detection: Disabled + +- Primary disk: + + - 500 GB SDD or NVMe controller + - 120 GB (min. 10K RPM) compute and storage + +- Additional disks: + + - 1 or more 500 GB disks (min. 10K RPM) storage, compute + +- Network ports\* + + - Management: 10GE controller, storage, compute + - OAM: 10GE controller + - Data: n x 10GE compute + +******************* +Virtual environment +******************* + +Run the libvirt qemu setup scripts. Setting up virtualized OAM and +management networks: + +:: + + $ bash setup_network.sh + +Building XML for definition of virtual servers: + +:: + + $ bash setup_configuration.sh -c dedicatedstorage -i + +The default XML server definitions that are created by the previous script +are: + +- dedicatedstorage-controller-0 +- dedicatedstorage-controller-1 +- dedicatedstorage-compute-0 +- dedicatedstorage-compute-1 +- dedicatedstorage-storage-0 +- dedicatedstorage-storage-1 + +^^^^^^^^^^^^^^^^^^^^^^^^^ +Power up a virtual server +^^^^^^^^^^^^^^^^^^^^^^^^^ + +To power up a virtual server, run the following command: + +:: + + $ sudo virsh start + +e.g. + +:: + + $ sudo virsh start dedicatedstorage-controller-0 + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Access virtual server consoles +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The XML for virtual servers in stx-tools repo, deployment/libvirt, +provides both graphical and text consoles. + +Access the graphical console in virt-manager by right-click on the +domain (the server) and selecting "Open". + +Access the textual console with the command "virsh console $DOMAIN", +where DOMAIN is the name of the server shown in virsh. + +When booting the controller-0 for the first time, both the serial and +graphical consoles will present the initial configuration menu for the +cluster. One can select serial or graphical console for controller-0. +For the other nodes however only serial is used, regardless of which +option is selected. + +Open the graphic console on all servers before powering them on to +observe the boot device selection and PXI boot progress. Run "virsh +console $DOMAIN" command promptly after power on to see the initial boot +sequence which follows the boot device selection. One has a few seconds +to do this. + +-------------------------------- +Installing the controller-0 host +-------------------------------- + +Installing controller-0 involves initializing a host with software and +then applying a bootstrap configuration from the command line. The +configured bootstrapped host becomes controller-0. + +Procedure: + +#. Power on the server that will be controller-0 with the StarlingX ISO + on a USB in a bootable USB slot. +#. Configure the controller using the config_controller script. + +************************* +Initializing controller-0 +************************* + +This section describes how to initialize StarlingX in host controller-0. +Except where noted, all the commands must be executed from a console of +the host. + +Power on the host to be configured as controller-0, with the StarlingX +ISO on a USB in a bootable USB slot. Wait for the console to show the +StarlingX ISO booting options: + +- **Standard Controller Configuration** + + - When the installer is loaded and the installer welcome screen + appears in the controller-0 host, select the type of installation + "Standard Controller Configuration". + +- **Graphical Console** + + - Select the "Graphical Console" as the console to use during + installation. + +- **Standard Security Boot Profile** + + - Select "Standard Security Boot Profile" as the security profile. + +Monitor the initialization. When it is complete, a reboot is initiated +on the controller-0 host, briefly displays a GNU GRUB screen, and then +boots automatically into the StarlingX image. + +Log into controller-0 as user wrsroot, with password wrsroot. The +first time you log in as wrsroot, you are required to change your +password. Enter the current password (wrsroot): + +:: + + Changing password for wrsroot. + (current) UNIX Password: + +Enter a new password for the wrsroot account: + +:: + + New password: + +Enter the new password again to confirm it: + +:: + + Retype new password: + +controller-0 is initialized with StarlingX, and is ready for configuration. + +************************ +Configuring controller-0 +************************ + +This section describes how to perform the controller-0 configuration +interactively just to bootstrap system with minimum critical data. +Except where noted, all the commands must be executed from the console +of the active controller (here assumed to be controller-0). + +When run interactively, the config_controller script presents a series +of prompts for initial configuration of StarlingX: + +- For the virtual environment, you can accept all the default values + immediately after ‘system date and time’. +- For a physical deployment, answer the bootstrap configuration + questions with answers applicable to your particular physical setup. + +The script is used to configure the first controller in the StarlingX +cluster as controller-0. The prompts are grouped by configuration +area. To start the script interactively, use the following command +with no parameters: + +:: + + controller-0:~$ sudo config_controller + System Configuration + ================ + Enter ! at any prompt to abort... + ... + +Accept all the default values immediately after ‘system date and time’: + +:: + + ... + Applying configuration (this will take several minutes): + + 01/08: Creating bootstrap configuration ... DONE + 02/08: Applying bootstrap manifest ... DONE + 03/08: Persisting local configuration ... DONE + 04/08: Populating initial system inventory ... DONE + 05:08: Creating system configuration ... DONE + 06:08: Applying controller manifest ... DONE + 07:08: Finalize controller configuration ... DONE + 08:08: Waiting for service activation ... DONE + + Configuration was applied + + Please complete any out of service commissioning steps with system commands and unlock controller to proceed. + +After config_controller bootstrap configuration, REST API, CLI and +Horizon interfaces are enabled on the controller-0 OAM IP address. The +remaining installation instructions will use the CLI. + +------------------------------------ +Provisioning controller-0 and system +------------------------------------ + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +********************************************* +Configuring provider networks at installation +********************************************* + +You must set up provider networks at installation so that you can attach +data interfaces and unlock the compute nodes. + +Set up one provider network of the vlan type, named providernet-a: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-create providernet-a --type=vlan + [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-range-create --name providernet-a-range1 --range 100-400 providernet-a + +********************************************* +Adding a Ceph storage backend at installation +********************************************* + +Add CEPH Storage backend: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add ceph -s cinder,glance,swift,nova + + WARNING : THIS OPERATION IS NOT REVERSIBLE AND CANNOT BE CANCELLED. + + By confirming this operation, Ceph backend will be created. + A minimum of 2 storage nodes are required to complete the configuration. + Please set the 'confirmed' field to execute this operation for the ceph backend. + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add ceph -s cinder,glance,swift,nova --confirmed + + System configuration has changed. + Please follow the administrator guide to complete configuring the system. + + +--------------------------------------+------------+---------+-------------+--------------------+----------+... + | uuid | name | backend | state | task | services |... + +--------------------------------------+------------+---------+-------------+--------------------+----------+... + | 48ddb10a-206c-42da-bb3f-f7160a356724 | ceph-store | ceph | configuring | applying-manifests | cinder, |... + | | | | | | glance, |... + | | | | | | swift |... + | | | | | | nova |... + | | | | | | |... + | 55f49f86-3e01-4d03-a014-42e1b55ba487 | file-store | file | configured | None | glance |... + +--------------------------------------+------------+---------+-------------+--------------------+----------+... + +Confirm CEPH storage is configured: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-list + +--------------------------------------+------------+---------+------------+-------------------+-----------+... + | uuid | name | backend | state | task | services |... + +--------------------------------------+------------+---------+------------+-------------------+-----------+... + | 48ddb10a-206c-42da-bb3f-f7160a356724 | ceph-store | ceph | configured | provision-storage | cinder, |... + | | | | | | glance, |... + | | | | | | swift |... + | | | | | | nova |... + | | | | | | |... + | 55f49f86-3e01-4d03-a014-42e1b55ba487 | file-store | file | configured | None | glance |... + +--------------------------------------+------------+---------+------------+-------------------+-----------+... + +********************** +Unlocking controller-0 +********************** + +You must unlock controller-0 so that you can use it to install the remaining +hosts. Use the system host-unlock command: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-0 + +The host is rebooted. During the reboot, the command line is unavailable, and +any ssh connections are dropped. To monitor the progress of the reboot, use the +controller-0 console. + +**************************************** +Verifying the controller-0 configuration +**************************************** + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +Verify that the StarlingX controller services are running: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system service-list + +-----+-------------------------------+--------------+----------------+ + | id | service_name | hostname | state | + +-----+-------------------------------+--------------+----------------+ + ... + | 1 | oam-ip | controller-0 | enabled-active | + | 2 | management-ip | controller-0 | enabled-active | + ... + +-----+-------------------------------+--------------+----------------+ + +Verify that controller-0 is unlocked, enabled, and available: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + +----+--------------+-------------+----------------+-------------+--------------+ + +******************************* +Provisioning filesystem storage +******************************* + +List the controller file systems with status and current sizes: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system controllerfs-list + +--------------------------------------+-----------------+------+--------------------+------------+-------+ + | UUID | FS Name | Size | Logical Volume | Replicated | State | + | | | in | | | | + | | | GiB | | | | + +--------------------------------------+-----------------+------+--------------------+------------+-------+ + | 4e31c4ea-6970-4fc6-80ba-431fdcdae15f | backup | 5 | backup-lv | False | None | + | 6c689cd7-2bef-4755-a2fb-ddd9504692f3 | database | 5 | pgsql-lv | True | None | + | 44c7d520-9dbe-41be-ac6a-5d02e3833fd5 | extension | 1 | extension-lv | True | None | + | 809a5ed3-22c0-4385-9d1e-dd250f634a37 | glance | 8 | cgcs-lv | True | None | + | 9c94ef09-c474-425c-a8ba-264e82d9467e | gnocchi | 5 | gnocchi-lv | False | None | + | 895222b3-3ce5-486a-be79-9fe21b94c075 | img-conversions | 8 | img-conversions-lv | False | None | + | 5811713f-def2-420b-9edf-6680446cd379 | scratch | 8 | scratch-lv | False | None | + +--------------------------------------+-----------------+------+--------------------+------------+-------+ + +Modify filesystem sizes + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system controllerfs-modify backup=42 database=12 img-conversions=12 + +------------------------------------------------------- +Installing controller-1 / storage hosts / compute hosts +------------------------------------------------------- + +After initializing and configuring an active controller, you can add and +configure a backup controller and additional compute or storage hosts. +For each host do the following: + +***************** +Initializing host +***************** + +Power on Host. In host console you will see: + +:: + + Waiting for this node to be configured. + + Please configure the personality for this node from the + controller node in order to proceed. + +********************************** +Updating host name and personality +********************************** + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +Wait for controller-0 to discover new host, list the host until new +UNKNOWN host shows up in table: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +Use the system host-add to update host personality attribute: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-add -n -p -m + +**REMARK:** use the Mac address for the specific network interface you +are going to be connected. e.g. OAM network interface for controller-1 +node, management network interface for compute and storage nodes. + +Check the **NIC** MAC address from "Virtual Manager GUI" under *"Show +virtual hardware details -*\ **i**\ *" Main Banner --> NIC: --> specific +"Bridge name:" under MAC address text field.* + +*************** +Monitoring host +*************** + +On controller-0, you can monitor the installation progress by running +the system host-show command for the host periodically. Progress is +shown in the install_state field. + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-show | grep install + | install_output | text | + | install_state | booting | + | install_state_info | None | + + +Wait while the host is configured and rebooted. Up to 20 minutes may be +required for a reboot, depending on hardware. When the reboot is +complete, the host is reported as locked, disabled, and online. + +************* +Listing hosts +************* + +Once all nodes have been installed, configured and rebooted, on +controller-0 list the hosts: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | locked | disabled | online | + | 3 | compute-0 | compute | locked | disabled | online | + | 4 | compute-1 | compute | locked | disabled | online | + | 5 | storage-0 | storage | locked | disabled | online | + | 6 | storage-1 | storage | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +------------------------- +Provisioning controller-1 +------------------------- + +On controller-0, list hosts: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + ... + | 2 | controller-1 | controller | locked | disabled | online | + ... + +----+--------------+-------------+----------------+-------------+--------------+ + +*********************************************** +Provisioning network interfaces on controller-1 +*********************************************** + +In order to list out hardware port names, types, PCI addresses that have +been discovered: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list controller-1 + +Provision the OAM interface for controller-1: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -n -c platform --networks oam controller-1 + +********************** +Unlocking controller-1 +********************** + +Unlock controller-1: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-1 + +Wait while the controller-1 is rebooted. Up to 10 minutes may be +required for a reboot, depending on hardware. + +**REMARK:** controller-1 will remain in degraded state until +data-syncing is complete. The duration is dependant on the +virtualization host's configuration - i.e., the number and configuration +of physical disks used to host the nodes' virtual disks. Also, the +management network is expected to have link capacity of 10000 (1000 is +not supported due to excessive data-sync time). Use 'fm alarm-list' to +confirm status. + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | unlocked | enabled | available | + ... + +------------------------- +Provisioning storage host +------------------------- + +************************************** +Provisioning storage on a storage host +************************************** + +Available physical disks in storage-N: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list storage-0 + +--------------------------------------+-----------+---------+---------+-------+------------+--------------+... + | uuid | device_no | device_ | device_ | size_ | available_ | rpm |... + | | de | num | type | gib | gib | |... + +--------------------------------------+-----------+---------+---------+-------+------------+--------------+... + | a2bbfe1f-cf91-4d39-a2e8-a9785448aa56 | /dev/sda | 2048 | HDD | 292. | 0.0 | Undetermined |... + | | | | | 968 | | |... + | | | | | | | |... + | c7cc08e6-ff18-4229-a79d-a04187de7b8d | /dev/sdb | 2064 | HDD | 100.0 | 99.997 | Undetermined |... + | | | | | | | |... + | | | | | | | |... + | 1ece5d1b-5dcf-4e3c-9d10-ea83a19dd661 | /dev/sdc | 2080 | HDD | 4.0 | 3.997 |... + | | | | | | | |... + | | | | | | | |... + +--------------------------------------+-----------+---------+---------+-------+------------+--------------+... + +Available storage tiers in storage-N: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-tier-list ceph_cluster + +--------------------------------------+---------+--------+--------------------------------------+ + | uuid | name | status | backend_using | + +--------------------------------------+---------+--------+--------------------------------------+ + | 4398d910-75e4-4e99-a57f-fc147fb87bdb | storage | in-use | 5131a848-25ea-4cd8-bbce-0d65c84183df | + +--------------------------------------+---------+--------+--------------------------------------+ + +Create a storage function (i.e. OSD) in storage-N. At least two unlocked and +enabled hosts with monitors are required. Candidates are: controller-0, +controller-1, and storage-0. + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-stor-add storage-0 c7cc08e6-ff18-4229-a79d-a04187de7b8d + +------------------+--------------------------------------------------+ + | Property | Value | + +------------------+--------------------------------------------------+ + | osdid | 0 | + | function | osd | + | journal_location | 34989bad-67fc-49ea-9e9c-38ca4be95fad | + | journal_size_gib | 1024 | + | journal_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part2 | + | journal_node | /dev/sdb2 | + | uuid | 34989bad-67fc-49ea-9e9c-38ca4be95fad | + | ihost_uuid | 4a5ed4fc-1d2b-4607-acf9-e50a3759c994 | + | idisk_uuid | c7cc08e6-ff18-4229-a79d-a04187de7b8d | + | tier_uuid | 4398d910-75e4-4e99-a57f-fc147fb87bdb | + | tier_name | storage | + | created_at | 2018-08-16T00:39:44.409448+00:00 | + | updated_at | 2018-08-16T00:40:07.626762+00:00 | + +------------------+--------------------------------------------------+ + +Create remaining available storage function (an OSD) in storage-N +based in the number of available physical disks. + +List the OSDs: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-stor-list storage-0 + +--------------------------------------+----------+-------+--------------+--------------------------------------+ + | uuid | function | osdid | capabilities | idisk_uuid | + +--------------------------------------+----------+-------+--------------+--------------------------------------+ + | 34989bad-67fc-49ea-9e9c-38ca4be95fad | osd | 0 | {} | c7cc08e6-ff18-4229-a79d-a04187de7b8d | + +--------------------------------------+----------+-------+--------------+--------------------------------------+ + +Unlock storage-N: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock storage-0 + +**REMARK:** Before you continue, repeat Provisioning Storage steps on +remaining storage nodes. + +--------------------------- +Provisioning a compute host +--------------------------- + +You must configure the network interfaces and the storage disks on a +host before you can unlock it. For each compute host do the following: + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +************************************************* +Provisioning network interfaces on a compute host +************************************************* + +On controller-0, in order to list out hardware port names, types, +pci-addresses that have been discovered: + +- **Only in virtual environment**: Ensure that the interface used is + one of those attached to host bridge with model type "virtio" (i.e., + eth1000 and eth1001). The model type "e1000" emulated devices will + not work for provider networks. + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list compute-0 + +Provision the data interface for compute: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -p providernet-a -c data compute-0 eth1000 + +*************************** +VSwitch virtual environment +*************************** + +**Only in virtual environment**. If the compute has more than 4 CPUs, +the system will auto-configure the vswitch to use 2 cores. However some +virtual environments do not properly support multi-queue required in a +multi-CPU environment. Therefore run the following command to reduce the +vswitch cores to 1: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-cpu-modify compute-0 -f vswitch -p0 1 + +--------------------------------------+-------+-----------+-------+--------+... + | uuid | log_c | processor | phy_c | thread |... + | | ore | | ore | |... + +--------------------------------------+-------+-----------+-------+--------+... + | a3b5620c-28b1-4fe0-9e97-82950d8582c2 | 0 | 0 | 0 | 0 |... + | f2e91c2b-bfc5-4f2a-9434-bceb7e5722c3 | 1 | 0 | 1 | 0 |... + | 18a98743-fdc4-4c0c-990f-3c1cb2df8cb3 | 2 | 0 | 2 | 0 |... + | 690d25d2-4f99-4ba1-a9ba-0484eec21cc7 | 3 | 0 | 3 | 0 |... + +--------------------------------------+-------+-----------+-------+--------+... + +************************************** +Provisioning storage on a compute host +************************************** + +Review the available disk space and capacity and obtain the uuid(s) of +the physical disk(s) to be used for nova local: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list compute-0 + +--------------------------------------+-----------+---------+---------+-------+------------+... + | uuid | device_no | device_ | device_ | size_ | available_ |... + | | de | num | type | gib | gib |... + +--------------------------------------+-----------+---------+---------+-------+------------+ + | 14e52a55-f6a7-40ad-a0b1-11c2c3b6e7e9 | /dev/sda | 2048 | HDD | 292. | 265.132 |... + | a639914b-23a9-4071-9f25-a5f1960846cc | /dev/sdb | 2064 | HDD | 100.0 | 99.997 |... + +--------------------------------------+-----------+---------+---------+-------+------------+... + +Create the 'nova-local' local volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add compute-0 nova-local + +-----------------+-------------------------------------------------------------------+ + | Property | Value | + +-----------------+-------------------------------------------------------------------+ + | lvm_vg_name | nova-local | + | vg_state | adding | + | uuid | 37f4c178-f0fe-422d-b66e-24ae057da674 | + | ihost_uuid | f56921a6-8784-45ac-bd72-c0372cd95964 | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size_gib | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-16T00:57:46.340454+00:00 | + | updated_at | None | + | parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} | + +-----------------+-------------------------------------------------------------------+ + +Create a disk partition to add to the volume group based on uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add compute-0 nova-local a639914b-23a9-4071-9f25-a5f1960846cc + +--------------------------+--------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------+ + | uuid | 56fdb63a-1078-4394-b1ce-9a0b3bff46dc | + | pv_state | adding | + | pv_type | disk | + | disk_or_part_uuid | a639914b-23a9-4071-9f25-a5f1960846cc | + | disk_or_part_device_node | /dev/sdb | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0 | + | lvm_pv_name | /dev/sdb | + | lvm_vg_name | nova-local | + | lvm_pv_uuid | None | + | lvm_pv_size_gib | 0.0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | f56921a6-8784-45ac-bd72-c0372cd95964 | + | created_at | 2018-08-16T01:05:59.013257+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------+ + +Remote RAW Ceph storage backed will be used to back nova local ephemeral +volumes: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-modify -b remote compute-0 nova-local + +************************ +Unlocking a compute host +************************ + +On controller-0, use the system host-unlock command to unlock the +compute-N: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock compute-0 + +Wait while the compute-N is rebooted. Up to 10 minutes may be required +for a reboot, depending on hardware. The host is rebooted, and its +availability state is reported as in-test, followed by unlocked/enabled. + +------------------- +System health check +------------------- + +*********************** +Listing StarlingX nodes +*********************** + +On controller-0, after a few minutes, all nodes shall be reported as +unlocked, enabled, and available: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | unlocked | enabled | available | + | 3 | compute-0 | compute | unlocked | enabled | available | + | 4 | compute-1 | compute | unlocked | enabled | available | + | 5 | storage-0 | storage | unlocked | enabled | available | + | 6 | storage-1 | storage | unlocked | enabled | available | + +----+--------------+-------------+----------------+-------------+--------------+ + +****************************** +Checking StarlingX CEPH health +****************************** + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ ceph -s + cluster e14ebfd6-5030-4592-91c3-7e6146b3c910 + health HEALTH_OK + monmap e1: 3 mons at {controller-0=192.168.204.3:6789/0,controller-1=192.168.204.4:6789/0,storage-0=192.168.204.204:6789/0} + election epoch 22, quorum 0,1,2 controller-0,controller-1,storage-0 + osdmap e84: 2 osds: 2 up, 2 in + flags sortbitwise,require_jewel_osds + pgmap v168: 1600 pgs, 5 pools, 0 bytes data, 0 objects + 87444 kB used, 197 GB / 197 GB avail + 1600 active+clean + controller-0:~$ + +***************** +System alarm list +***************** + +When all nodes are unlocked, enabled and available: check 'fm alarm-list' for +issues. + +Your StarlingX deployment is now up and running with 2x HA controllers with +Cinder storage, 1x compute, 3x storages and all OpenStack services up and +running. You can now proceed with standard OpenStack APIs, CLIs and/or Horizon +to load Glance images, configure Nova Flavors, configure Neutron networks and +launch Nova virtual machines. + +---------------------- +Deployment terminology +---------------------- + +.. include:: deployment_terminology.rst + :start-after: incl-standard-controller-deployment-terminology: + :end-before: incl-standard-controller-deployment-terminology-end: + +.. include:: deployment_terminology.rst + :start-after: incl-dedicated-storage-deployment-terminology: + :end-before: incl-dedicated-storage-deployment-terminology-end: + +.. include:: deployment_terminology.rst + :start-after: incl-common-deployment-terminology: + :end-before: incl-common-deployment-terminology-end: diff --git a/doc/source/installation_guide/latest/deployment_terminology.rst b/doc/source/installation_guide/latest/deployment_terminology.rst new file mode 100644 index 000000000..a88b4695f --- /dev/null +++ b/doc/source/installation_guide/latest/deployment_terminology.rst @@ -0,0 +1,119 @@ +.. _incl-simplex-deployment-terminology: + +**All-in-one controller node** + A single physical node that provides a controller function, compute + function, and storage function. + +.. _incl-simplex-deployment-terminology-end: + + +.. _incl-standard-controller-deployment-terminology: + +**Controller node / function** + A node that runs cloud control function for managing cloud resources. + + - Runs cloud control functions for managing cloud resources. + - Runs all OpenStack control functions (e.g. managing images, virtual + volumes, virtual network, and virtual machines). + - Can be part of a two-node HA control node cluster for running control + functions either active/active or active/standby. + +**Compute ( & network ) node / function** + A node that hosts applications in virtual machines using compute resources + such as CPU, memory, and disk. + + - Runs virtual switch for realizing virtual networks. + - Provides L3 routing and NET services. + +.. _incl-standard-controller-deployment-terminology-end: + + +.. _incl-dedicated-storage-deployment-terminology: + +**Storage node / function** + A node that contains a set of disks (e.g. SATA, SAS, SSD, and/or NVMe). + + - Runs CEPH distributed storage software. + - Part of an HA multi-node CEPH storage cluster supporting a replication + factor of two or three, journal caching, and class tiering. + - Provides HA persistent storage for images, virtual volumes + (i.e. block storage), and object storage. + +.. _incl-dedicated-storage-deployment-terminology-end: + +.. _incl-common-deployment-terminology: + +**OAM network** + The network on which all external StarlingX platform APIs are exposed, + (i.e. REST APIs, Horizon web server, SSH, and SNMP), typically 1GE. + + Only controller type nodes are required to be connected to the OAM + network. + +**Management network** + A private network (i.e. not connected externally), tipically 10GE, + used for the following: + + - Internal OpenStack / StarlingX monitoring and control. + - VM I/O access to a storage cluster. + + All nodes are required to be connected to the management network. + +**Data network(s)** + Networks on which the OpenStack / Neutron provider networks are realized + and become the VM tenant networks. + + Only compute type and all-in-one type nodes are required to be connected + to the data network(s); these node types require one or more interface(s) + on the data network(s). + +**IPMI network** + An optional network on which IPMI interfaces of all nodes are connected. + The network must be reachable using L3/IP from the controller's OAM + interfaces. + + You can optionally connect all node types to the IPMI network. + +**PXEBoot network** + An optional network for controllers to boot/install other nodes over the + network. + + By default, controllers use the management network for boot/install of other + nodes in the openstack cloud. If this optional network is used, all node + types are required to be connected to the PXEBoot network. + + A PXEBoot network is required for a variety of special case situations: + + - Cases where the management network must be IPv6: + + - IPv6 does not support PXEBoot. Therefore, IPv4 PXEBoot network must be + configured. + + - Cases where the management network must be VLAN tagged: + + - Most server's BIOS do not support PXEBooting over tagged networks. + Therefore, you must configure an untagged PXEBoot network. + + - Cases where a management network must be shared across regions but + individual regions' controllers want to only network boot/install nodes + of their own region: + + - You must configure separate, per-region PXEBoot networks. + +**Infra network** + A deprecated optional network that was historically used for access to the + storage cluster. + + If this optional network is used, all node types are required to be + connected to the INFRA network, + +**Node interfaces** + All nodes' network interfaces can, in general, optionally be either: + + - Untagged single port. + - Untagged two-port LAG and optionally split between redudant L2 switches + running vPC (Virtual Port-Channel), also known as multichassis + EtherChannel (MEC). + - VLAN on either single-port ETH interface or two-port LAG interface. + +.. _incl-common-deployment-terminology-end: diff --git a/doc/source/installation_guide/latest/duplex.rst b/doc/source/installation_guide/latest/duplex.rst new file mode 100644 index 000000000..552aff31d --- /dev/null +++ b/doc/source/installation_guide/latest/duplex.rst @@ -0,0 +1,1520 @@ +============================================== +All-In-One Duplex deployment guide stx.2019.05 +============================================== + +.. contents:: + :local: + :depth: 1 + +**NOTE:** The instructions to setup a StarlingX All-in-One Duplex +(AIO-DX) with containerized openstack services in this guide +are under development. +For approved instructions, see the +`All in One Duplex Configuration wiki page `__. + +---------------------- +Deployment description +---------------------- + +***************** +All-In-One Duplex +***************** + +The All-In-One Duplex (AIO-DX) deployment option provides all three cloud +functions (controller, compute, and storage) on two physical servers. With cloud +technologies, multiple diverse application types can be deployed and +consolidated onto a protected pair of physical servers. For example: + +- Consolidate legacy applications that must run standalone on a server by + using multiple virtual machines. +- Consolidate legacy applications that run on different operating systems + or different distributions of operating systems by using multiple virtual + machines. + +.. figure:: figures/starlingx-deployment-options-duplex.png + :scale: 50% + :alt: All-In-One Duplex deployment configuration + + *All-In-One Duplex deployment configuration* + +This two node cluster enables: + +- High availability services running on the controller function across the + two physical servers in either active/active or active/standby mode. +- Storage function running on top of LVM on single second disk, DRBD-sync'd + between the servers. +- Virtual machines being scheduled on both compute functions. + +A All-In-One Duplex deployment provides protection against overall server +hardware fault. Should an overall server hardware fault occur: + +- All controller high availability services go active on remaining + healthy server. +- All virtual machines are recovered on remaining healthy server. + +The All-In-One Duplex deployment solution is required for a variety of special +case situations, for example: + +- Small amount of cloud processing/storage. +- Protection against overall server hardware fault. + +************************** +All-In-One Duplex extended +************************** + +The All-In-One Duplex Extended deployment option extends the capacity of the +All-In-One Duplex deployment by adding up to four compute nodes to the +deployment. The extended deployment option provides a capacity growth path for +someone starting with an All-In-One Duplex deployment. + +With this option, virtual machines can be scheduled on either of the +all-in-one controller nodes and/or the compute nodes. + +.. figure:: figures/starlingx-deployment-options-duplex-extended.png + :scale: 50% + :alt: All-In-One Duplex Extended Deployment Configuration + + *All-In-One Duplex Extended deployment configuration* + +This configuration is limited to four compute nodes as the controller function +on the all-in-one controllers has only a portion of the processing power of the +overall server. + +----------------------------------- +Preparing All-In-One Duplex servers +----------------------------------- + +********** +Bare metal +********** + +Required Servers: + +- Combined servers (controller + compute): 2 + +^^^^^^^^^^^^^^^^^^^^^ +Hardware requirements +^^^^^^^^^^^^^^^^^^^^^ + +The recommended minimum requirements for the physical servers where +All-In-One Duplex will be deployed, include: + +- Minimum processor: + + - Typical hardware form factor: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket + - Low cost / low power hardware form factor + + - Single-CPU Intel Xeon D-15xx family, 8 cores + +- Memory: 64 GB +- BIOS: + + - Hyper-Threading technology: Enabled + - Virtualization technology: Enabled + - VT for directed I/O: Enabled + - CPU power and performance policy: Performance + - CPU C state control: Disabled + - Plug & play BMC detection: Disabled + +- Primary disk: + + - 500 GB SDD or NVMe + +- Additional disks: + + - Zero or more 500 GB disks (min. 10K RPM) + +- Network ports: + + **NOTE:** The All-In-One Duplex configuration requires one or more data ports. + + - Management: 10GE + - OAM: 10GE + - Data: n x 10GE + +******************* +Virtual environment +******************* + +Run the libvirt QEMU setup scripts. Setting up virtualized OAM and +management networks: + +:: + + $ bash setup_network.sh + +Building XML for definition of virtual servers: + +:: + + $ bash setup_configuration.sh -c duplex -i + +The default XML server definitions that are created by the previous script +are: + +- duplex-controller-0 +- duplex-controller-1 + +^^^^^^^^^^^^^^^^^^^^^^^^^ +Power up a virtual server +^^^^^^^^^^^^^^^^^^^^^^^^^ + +To power up a virtual server, run the following command: + +:: + + $ sudo virsh start + +e.g. + +:: + + $ sudo virsh start duplex-controller-0 + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Access virtual server consoles +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The XML for virtual servers in stx-tools repo, deployment/libvirt, +provides both graphical and text consoles. + +Access the graphical console in virt-manager by right-click on the +domain (the server) and selecting "Open". + +Access the textual console with the command "virsh console $DOMAIN", +where DOMAIN is the name of the server shown in virsh. + +When booting the controller-0 for the first time, both the serial and +graphical consoles will present the initial configuration menu for the +cluster. One can select serial or graphical console for controller-0. +For the other nodes however only serial is used, regardless of which +option is selected. + +Open the graphic console on all servers before powering them on to +observe the boot device selection and PXI boot progress. Run "virsh +console $DOMAIN" command promptly after power on to see the initial boot +sequence which follows the boot device selection. One has a few seconds +to do this. + +-------------------------------- +Installing the controller-0 host +-------------------------------- + +Installing controller-0 involves initializing a host with software and +then applying a bootstrap configuration from the command line. The +configured bootstrapped host becomes controller-0. + +Procedure: + +#. Power on the server that will be controller-0 with the StarlingX ISO + on a USB in a bootable USB slot. +#. Configure the controller using the config_controller script. + +************************* +Initializing controller-0 +************************* + +This section describes how to initialize StarlingX in host controller-0. +Except where noted, all the commands must be executed from a console of +the host. + +Power on the host to be configured as controller-0, with the StarlingX +ISO on a USB in a bootable USB slot. Wait for the console to show the +StarlingX ISO booting options: + +- **All-in-one Controller Configuration** + + - When the installer is loaded and the installer welcome screen + appears in the controller-0 host, select the type of installation + "All-in-one Controller Configuration". + +- **Graphical Console** + + - Select the "Graphical Console" as the console to use during + installation. + +- **Standard Security Boot Profile** + + - Select "Standard Security Boot Profile" as the security profile. + +Monitor the initialization. When it is complete, a reboot is initiated +on the controller-0 host, briefly displays a GNU GRUB screen, and then +boots automatically into the StarlingX image. + +Log into controller-0 as user wrsroot, with password wrsroot. The +first time you log in as wrsroot, you are required to change your +password. Enter the current password (wrsroot): + +:: + + Changing password for wrsroot. + (current) UNIX Password: + +Enter a new password for the wrsroot account: + +:: + + New password: + +Enter the new password again to confirm it: + +:: + + Retype new password: + +controller-0 is initialized with StarlingX, and is ready for +configuration. + +************************ +Configuring controller-0 +************************ + +This section describes how to perform the controller-0 configuration +interactively just to bootstrap system with minimum critical data. +Except where noted, all the commands must be executed from the console +of the active controller (here assumed to be controller-0). + +When run interactively, the config_controller script presents a series +of prompts for initial configuration of StarlingX: + +- For the virtual environment, you can accept all the default values + immediately after ‘system date and time’. +- For a physical deployment, answer the bootstrap configuration + questions with answers applicable to your particular physical setup. + +The script is used to configure the first controller in the StarlingX +cluster as controller-0. The prompts are grouped by configuration +area. To start the script interactively, use the following command +with no parameters: + +:: + + controller-0:~$ sudo config_controller + System Configuration + ================ + Enter ! at any prompt to abort... + ... + +Select [y] for System date and time: + +:: + + System date and time: + ----------------------------- + + Is the current date and time correct? [y/N]: y + +For System mode choose "duplex": + +:: + + System mode. Available options are: + + 1) duplex-direct: two node-redundant configuration. Management and + infrastructure networks are directly connected to peer ports + 2) duplex - two node redundant configuration + 3) simplex - single node non-redundant configuration + System mode [duplex-direct]: 2 + +After System date and time and System mode: + +:: + + Applying configuration (this will take several minutes): + + 01/08: Creating bootstrap configuration ... DONE + 02/08: Applying bootstrap manifest ... DONE + 03/08: Persisting local configuration ... DONE + 04/08: Populating initial system inventory ... DONE + 05:08: Creating system configuration ... DONE + 06:08: Applying controller manifest ... DONE + 07:08: Finalize controller configuration ... DONE + 08:08: Waiting for service activation ... DONE + + Configuration was applied + + Please complete any out of service commissioning steps with system commands and unlock controller to proceed. + +After config_controller bootstrap configuration, REST API, CLI and +Horizon interfaces are enabled on the controller-0 OAM IP address. The +remaining installation instructions will use the CLI. + +---------------------------------- +Provisioning the controller-0 host +---------------------------------- + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +********************************************* +Configuring provider networks at installation +********************************************* + +Set up one provider network of the vlan type, named providernet-a: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-create providernet-a --type=vlan + [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-range-create --name providernet-a-range1 --range 100-400 providernet-a + +***************************************** +Providing data interfaces on controller-0 +***************************************** + +List all interfaces: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-list -a controller-0 + +--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+.. + | uuid | name | class |...| vlan | ports | uses | used by | attributes |.. + | | | |...| id | | i/f | i/f | |.. + +--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+.. + | 49fd8938-e76f-49f1-879e-83c431a9f1af | enp0s3 | platform |...| None | [u'enp0s3'] | [] | [] | MTU=1500 |.. + | 8957bb2c-fec3-4e5d-b4ed-78071f9f781c | eth1000 | None |...| None | [u'eth1000'] | [] | [] | MTU=1500 |.. + | bf6f4cad-1022-4dd7-962b-4d7c47d16d54 | eth1001 | None |...| None | [u'eth1001'] | [] | [] | MTU=1500 |.. + | f59b9469-7702-4b46-bad5-683b95f0a1cb | enp0s8 | platform |...| None | [u'enp0s8'] | [] | [] | MTU=1500 |.. + +--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+.. + +Configure the data interfaces: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -c data controller-0 eth1000 -p providernet-a + +------------------+--------------------------------------+ + | Property | Value | + +------------------+--------------------------------------+ + | ifname | eth1000 | + | iftype | ethernet | + | ports | [u'eth1000'] | + | providernetworks | providernet-a | + | imac | 08:00:27:c4:ad:3e | + | imtu | 1500 | + | ifclass | data | + | aemode | None | + | schedpolicy | None | + | txhashpolicy | None | + | uuid | 8957bb2c-fec3-4e5d-b4ed-78071f9f781c | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | vlan_id | None | + | uses | [] | + | used_by | [] | + | created_at | 2018-08-28T12:50:51.820151+00:00 | + | updated_at | 2018-08-28T14:46:18.333109+00:00 | + | sriov_numvfs | 0 | + | ipv4_mode | disabled | + | ipv6_mode | disabled | + | accelerated | [True] | + +------------------+--------------------------------------+ + +************************************* +Configuring Cinder on controller disk +************************************* + +Review the available disk space and capacity and obtain the uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 + +--------------------------------------+-----------+---------+---------+---------+------------+... + | uuid | device_no | device_ | device_ | size_mi | available_ |... + | | de | num | type | b | mib |... + +--------------------------------------+-----------+---------+---------+---------+------------+... + | 6b42c9dc-f7c0-42f1-a410-6576f5f069f1 | /dev/sda | 2048 | HDD | 600000 | 434072 |... + | | | | | | |... + | | | | | | |... + | 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 | /dev/sdb | 2064 | HDD | 16240 | 16237 |... + | | | | | | |... + | | | | | | |... + | 146195b2-f3d7-42f9-935d-057a53736929 | /dev/sdc | 2080 | HDD | 16240 | 16237 |... + | | | | | | |... + | | | | | | |... + +--------------------------------------+-----------+---------+---------+---------+------------+... + +Create the 'cinder-volumes' local volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-0 cinder-volumes + +-----------------+--------------------------------------+ + | lvm_vg_name | cinder-volumes | + | vg_state | adding | + | uuid | 61cb5cd2-171e-4ef7-8228-915d3560cdc3 | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-28T13:45:20.218905+00:00 | + | updated_at | None | + | parameters | {u'lvm_type': u'thin'} | + +-----------------+--------------------------------------+ + +Create a disk partition to add to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-0 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 16237 -t lvm_phys_vol + +-------------+--------------------------------------------------+ + | Property | Value | + +-------------+--------------------------------------------------+ + | device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 | + | device_node | /dev/sdb1 | + | type_guid | ba5eba11-0000-1111-2222-000000000001 | + | type_name | None | + | start_mib | None | + | end_mib | None | + | size_mib | 16237 | + | uuid | 0494615f-bd79-4490-84b9-dcebbe5f377a | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | idisk_uuid | 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 | + | ipv_uuid | None | + | status | Creating | + | created_at | 2018-08-28T13:45:48.512226+00:00 | + | updated_at | None | + +-------------+--------------------------------------------------+ + +Wait for the new partition to be created (i.e. status=Ready): + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-0 --disk 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + | uuid |...| device_nod |...| type_name | size_mib | status | + | |...| e |...| | | | + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + | 0494615f-bd79-4490-84b9-dcebbe5f377a |...| /dev/sdb1 |...| LVM Physical Volume | 16237 | Ready | + | |...| |...| | | | + | |...| |...| | | | + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + +Add the partition to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-0 cinder-volumes 0494615f-bd79-4490-84b9-dcebbe5f377a + +--------------------------+--------------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------------+ + | uuid | 9a0ad568-0ace-4d57-9e03-e7a63f609cf2 | + | pv_state | adding | + | pv_type | partition | + | disk_or_part_uuid | 0494615f-bd79-4490-84b9-dcebbe5f377a | + | disk_or_part_device_node | /dev/sdb1 | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 | + | lvm_pv_name | /dev/sdb1 | + | lvm_vg_name | cinder-volumes | + | lvm_pv_uuid | None | + | lvm_pv_size | 0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | created_at | 2018-08-28T13:47:39.450763+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------------+ + +********************************************* +Adding an LVM storage backend at installation +********************************************* + +Ensure requirements are met to add LVM storage: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add lvm -s cinder + + WARNING : THIS OPERATION IS NOT REVERSIBLE AND CANNOT BE CANCELLED. + + By confirming this operation, the LVM backend will be created. + + Please refer to the system admin guide for minimum spec for LVM + storage. Set the 'confirmed' field to execute this operation + for the lvm backend. + +Add the LVM storage backend: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add lvm -s cinder --confirmed + + System configuration has changed. + Please follow the administrator guide to complete configuring the system. + + +--------------------------------------+------------+---------+-------------+...+----------+--------------+ + | uuid | name | backend | state |...| services | capabilities | + +--------------------------------------+------------+---------+-------------+...+----------+--------------+ + | 6d750a68-115a-4c26-adf4-58d6e358a00d | file-store | file | configured |...| glance | {} | + | e2697426-2d79-4a83-beb7-2eafa9ceaee5 | lvm-store | lvm | configuring |...| cinder | {} | + +--------------------------------------+------------+---------+-------------+...+----------+--------------+ + +Wait for the LVM storage backend to be configured (i.e. state=configured): + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-list + +--------------------------------------+------------+---------+------------+------+----------+--------------+ + | uuid | name | backend | state | task | services | capabilities | + +--------------------------------------+------------+---------+------------+------+----------+--------------+ + | 6d750a68-115a-4c26-adf4-58d6e358a00d | file-store | file | configured | None | glance | {} | + | e2697426-2d79-4a83-beb7-2eafa9ceaee5 | lvm-store | lvm | configured | None | cinder | {} | + +--------------------------------------+------------+---------+------------+------+----------+--------------+ + +*********************************************** +Configuring VM local storage on controller disk +*********************************************** + +Review the available disk space and capacity and obtain the uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 + +--------------------------------------+-----------+---------+---------+---------+------------+... + | uuid | device_no | device_ | device_ | size_mi | available_ |... + | | de | num | type | b | mib |... + +--------------------------------------+-----------+---------+---------+---------+------------+... + | 6b42c9dc-f7c0-42f1-a410-6576f5f069f1 | /dev/sda | 2048 | HDD | 600000 | 434072 |... + | | | | | | |... + | | | | | | |... + | 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 | /dev/sdb | 2064 | HDD | 16240 | 0 |... + | | | | | | |... + | | | | | | |... + | 146195b2-f3d7-42f9-935d-057a53736929 | /dev/sdc | 2080 | HDD | 16240 | 16237 |... + | | | | | | |... + | | | | | | |... + +--------------------------------------+-----------+---------+---------+---------+------------+... + +Create the 'noval-local' volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-0 nova-local + +-----------------+-------------------------------------------------------------------+ + | Property | Value | + +-----------------+-------------------------------------------------------------------+ + | lvm_vg_name | nova-local | + | vg_state | adding | + | uuid | 517d313e-8aa0-4b4d-92e6-774b9085f336 | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-28T14:02:58.486716+00:00 | + | updated_at | None | + | parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} | + +-----------------+-------------------------------------------------------------------+ + +Create a disk partition to add to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-0 146195b2-f3d7-42f9-935d-057a53736929 16237 -t lvm_phys_vol + +-------------+--------------------------------------------------+ + | Property | Value | + +-------------+--------------------------------------------------+ + | device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 | + | device_node | /dev/sdc1 | + | type_guid | ba5eba11-0000-1111-2222-000000000001 | + | type_name | None | + | start_mib | None | + | end_mib | None | + | size_mib | 16237 | + | uuid | 009ce3b1-ed07-46e9-9560-9d2371676748 | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | idisk_uuid | 146195b2-f3d7-42f9-935d-057a53736929 | + | ipv_uuid | None | + | status | Creating | + | created_at | 2018-08-28T14:04:29.714030+00:00 | + | updated_at | None | + +-------------+--------------------------------------------------+ + +Wait for the new partition to be created (i.e. status=Ready): + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-0 --disk 146195b2-f3d7-42f9-935d-057a53736929 + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + | uuid |...| device_nod |...| type_name | size_mib | status | + | |...| e |...| | | | + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + | 009ce3b1-ed07-46e9-9560-9d2371676748 |...| /dev/sdc1 |...| LVM Physical Volume | 16237 | Ready | + | |...| |...| | | | + | |...| |...| | | | + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + +Add the partition to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-0 nova-local 009ce3b1-ed07-46e9-9560-9d2371676748 + +--------------------------+--------------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------------+ + | uuid | 830c9dc8-c71a-4cb2-83be-c4d955ef4f6b | + | pv_state | adding | + | pv_type | partition | + | disk_or_part_uuid | 009ce3b1-ed07-46e9-9560-9d2371676748 | + | disk_or_part_device_node | /dev/sdc1 | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 | + | lvm_pv_name | /dev/sdc1 | + | lvm_vg_name | nova-local | + | lvm_pv_uuid | None | + | lvm_pv_size | 0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | created_at | 2018-08-28T14:06:05.705546+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------------+ + +********************** +Unlocking controller-0 +********************** + +You must unlock controller-0 so that you can use it to install +controller-1. Use the system host-unlock command: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-0 + +The host is rebooted. During the reboot, the command line is +unavailable, and any ssh connections are dropped. To monitor the +progress of the reboot, use the controller-0 console. + +**************************************** +Verifying the controller-0 configuration +**************************************** + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +Verify that the controller-0 services are running: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system service-list + +-----+-------------------------------+--------------+----------------+ + | id | service_name | hostname | state | + +-----+-------------------------------+--------------+----------------+ + ... + | 1 | oam-ip | controller-0 | enabled-active | + | 2 | management-ip | controller-0 | enabled-active | + ... + +-----+-------------------------------+--------------+----------------+ + +Verify that controller-0 has controller and compute subfunctions: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-show 1 | grep subfunctions + | subfunctions | controller,compute | + +Verify that controller-0 is unlocked, enabled, and available: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + +----+--------------+-------------+----------------+-------------+--------------+ + +-------------------------------- +Installing the controller-1 host +-------------------------------- + +After initializing and configuring controller-0, you can add and +configure a backup controller controller-1. + +****************************** +Initializing controller-1 host +****************************** + +Power on controller-1. In controller-1 console you will see: + +:: + + Waiting for this node to be configured. + + Please configure the personality for this node from the + controller node in order to proceed. + +**************************************************** +Updating controller-1 host hostname and personality +**************************************************** + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +Wait for controller-0 to discover new host, list the host until new +UNKNOWN host shows up in table: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +Use the system host-update to update contoller-1 host personality +attribute: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-update 2 personality=controller hostname=controller-1 + +---------------------+--------------------------------------+ + | Property | Value | + +---------------------+--------------------------------------+ + | action | none | + | administrative | locked | + | availability | offline | + | bm_ip | None | + | bm_type | None | + | bm_username | None | + | boot_device | sda | + | capabilities | {} | + | config_applied | None | + | config_status | None | + | config_target | None | + | console | ttyS0,115200 | + | created_at | 2018-08-28T15:09:13.812906+00:00 | + | hostname | controller-1 | + | id | 2 | + | install_output | text | + | install_state | None | + | install_state_info | None | + | invprovision | None | + | location | {} | + | mgmt_ip | 192.168.204.4 | + | mgmt_mac | 08:00:27:3d:e2:66 | + | operational | disabled | + | personality | controller | + | reserved | False | + | rootfs_device | sda | + | serialid | None | + | software_load | 18.03 | + | subfunction_avail | not-installed | + | subfunction_oper | disabled | + | subfunctions | controller,compute | + | task | None | + | tboot | false | + | ttys_dcd | None | + | updated_at | None | + | uptime | 0 | + | uuid | be0f35f7-31d9-4145-bf6a-0c2556cf398c | + | vim_progress_status | None | + +---------------------+--------------------------------------+ + +**************************** +Monitoring controller-1 host +**************************** + +On controller-0, you can monitor the installation progress by running +the system host-show command for the host periodically. Progress is +shown in the install_state field. + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-show controller-1 | grep install + | install_output | text | + | install_state | booting | + | install_state_info | None | + +Wait while the controller-1 is configured and rebooted. Up to 20 minutes +may be required for a reboot, depending on hardware. When the reboot is +complete, the controller-1 is reported as locked, disabled, and online. + +************************* +Listing controller-1 host +************************* + +Once controller-1 has been installed, configured and rebooted, on +controller-0 list the hosts: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +---------------------------------- +Provisioning the controller-1 host +---------------------------------- + +On controller-0, list hosts: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + ... + | 2 | controller-1 | controller | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +*********************************************** +Provisioning network interfaces on controller-1 +*********************************************** + +In order to list out hardware port names, types, PCI addresses that have +been discovered: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list controller-1 + +Provision the controller-1 OAM interface + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -n ens6 -c platform --networks oam controller-1 ens6 + +------------------+--------------------------------------+ + | Property | Value | + +------------------+--------------------------------------+ + | ifname | ens6 | + | iftype | ethernet | + | ports | [u'ens6'] | + | providernetworks | None | + | imac | 08:00:27:ba:3c:38 | + | imtu | 1500 | + | ifclass | oam | + | aemode | None | + | schedpolicy | None | + | txhashpolicy | None | + | uuid | 160bfede-0950-42ba-acef-d0eb31e7fc49 | + | ihost_uuid | be0f35f7-31d9-4145-bf6a-0c2556cf398c | + | vlan_id | None | + | uses | [] | + | used_by | [] | + | created_at | 2018-08-28T15:59:10.701772+00:00 | + | updated_at | 2018-08-29T05:44:38.585642+00:00 | + | sriov_numvfs | 0 | + | ipv4_mode | static | + | ipv6_mode | disabled | + | accelerated | [False] | + +------------------+--------------------------------------+ + +***************************************** +Providing data interfaces on controller-1 +***************************************** + +List all interfaces: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-list -a controller-1 + +--------------------------------------+---------+---------+...+------+--------------+------+------+------------+.. + | uuid | name | network |...| vlan | ports | uses | used | attributes |.. + | | | type |...| id | | i/f | by | |.. + | | | |...| | | | i/f | |.. + +--------------------------------------+---------+---------+...+------+--------------+------+------+------------+.. + | 160bfede-0950-42ba-acef-d0eb31e7fc49 | ens6 | oam |...| None | [u'enp0s3'] | [] | [] | MTU=1500 |.. + | 37b3abcd-bcbe-44d5-b2fb-759c34efec89 | eth1001 | None |...| None | [u'eth1001'] | [] | [] | MTU=1500 |.. + | 81d66feb-b23c-4533-bd4b-129f9b75ecd6 | mgmt0 | mgmt |...| None | [u'enp0s8'] | [] | [] | MTU=1500 |.. + | e78ad9a9-e74d-4c6c-9de8-0e41aad8d7b7 | eth1000 | None |...| None | [u'eth1000'] | [] | [] | MTU=1500 |.. + +--------------------------------------+---------+---------+...+------+--------------+------+------+------------+.. + +Configure the data interfaces: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -p providernet-a -c data controller-1 eth1000 + +------------------+--------------------------------------+ + | Property | Value | + +------------------+--------------------------------------+ + | ifname | eth1000 | + | networktype | data | + | iftype | ethernet | + | ports | [u'eth1000'] | + | providernetworks | providernet-a | + | imac | 08:00:27:39:61:6e | + | imtu | 1500 | + | aemode | None | + | schedpolicy | None | + | txhashpolicy | None | + | uuid | e78ad9a9-e74d-4c6c-9de8-0e41aad8d7b7 | + | ihost_uuid | be0f35f7-31d9-4145-bf6a-0c2556cf398c | + | vlan_id | None | + | uses | [] | + | used_by | [] | + | created_at | 2018-08-28T15:59:17.667899+00:00 | + | updated_at | 2018-08-29T06:04:55.116653+00:00 | + | sriov_numvfs | 0 | + | ipv4_mode | disabled | + | ipv6_mode | disabled | + | accelerated | [True] | + +------------------+--------------------------------------+ + +************************************ +Provisioning storage on controller-1 +************************************ + +Review the available disk space and capacity and obtain the uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 + +--------------------------------------+-------------+------------+-------------+----------+---------------+... + | uuid | device_node | device_num | device_type | size_mib | available_mib |... + +--------------------------------------+-------------+------------+-------------+----------+---------------+... + | 2dcc3d33-ba43-4b5a-b0b0-2b5a2e5737b7 | /dev/sda | 2048 | HDD | 600000 | 434072 |... + | f53437c6-77e3-4185-9453-67eaa8b461b1 | /dev/sdb | 2064 | HDD | 16240 | 16237 |... + | 623bbfc0-2b38-432a-acf4-a28db6066cce | /dev/sdc | 2080 | HDD | 16240 | 16237 |... + +--------------------------------------+-------------+------------+-------------+----------+---------------+... + +Assign Cinder storage to the physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-1 cinder-volumes + +-----------------+--------------------------------------+ + | Property | Value | + +-----------------+--------------------------------------+ + | lvm_vg_name | cinder-volumes | + | vg_state | adding | + | uuid | 3ccadc5c-e4f7-4b04-8403-af5d2ecef96d | + | ihost_uuid | be0f35f7-31d9-4145-bf6a-0c2556cf398c | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-29T05:51:13.965883+00:00 | + | updated_at | None | + | parameters | {u'lvm_type': u'thin'} | + +-----------------+--------------------------------------+ + +Create a disk partition to add to the volume group based on uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-1 f53437c6-77e3-4185-9453-67eaa8b461b1 16237 -t lvm_phys_vol + +-------------+--------------------------------------------------+ + | Property | Value | + +-------------+--------------------------------------------------+ + | device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 | + | device_node | /dev/sdb1 | + | type_guid | ba5eba11-0000-1111-2222-000000000001 | + | type_name | None | + | start_mib | None | + | end_mib | None | + | size_mib | 16237 | + | uuid | 7a41aab0-6695-4d16-9003-73238adda75b | + | ihost_uuid | be0f35f7-31d9-4145-bf6a-0c2556cf398c | + | idisk_uuid | f53437c6-77e3-4185-9453-67eaa8b461b1 | + | ipv_uuid | None | + | status | Creating (on unlock) | + | created_at | 2018-08-29T05:54:23.326612+00:00 | + | updated_at | None | + +-------------+--------------------------------------------------+ + +Wait for the new partition to be created (i.e. status=Ready): + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-1 --disk f53437c6-77e3-4185-9453-67eaa8b461b1 + +--------------------------------------+...+-------------+...+-----------+----------+----------------------+ + | uuid |...| device_node |...| type_name | size_mib | status | + +--------------------------------------+...+-------------+...+-----------+----------+----------------------+ + | 7a41aab0-6695-4d16-9003-73238adda75b |...| /dev/sdb1 |...| None | 16237 | Creating (on unlock) | + +--------------------------------------+...+-------------+...+-----------+----------+----------------------+ + +Add the partition to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-1 cinder-volumes 7a41aab0-6695-4d16-9003-73238adda75b + +--------------------------+--------------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------------+ + | uuid | 11628316-56ab-41b2-b2be-a5d72667a5e9 | + | pv_state | adding | + | pv_type | partition | + | disk_or_part_uuid | 7a41aab0-6695-4d16-9003-73238adda75b | + | disk_or_part_device_node | /dev/sdb1 | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 | + | lvm_pv_name | /dev/sdb1 | + | lvm_vg_name | cinder-volumes | + | lvm_pv_uuid | None | + | lvm_pv_size | 0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | be0f35f7-31d9-4145-bf6a-0c2556cf398c | + | created_at | 2018-08-29T06:25:54.550430+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------------+ + +*********************************************** +Configuring VM local storage on controller disk +*********************************************** + +Review the available disk space and capacity and obtain the uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 + +--------------------------------------+-------------+------------+-------------+----------+---------------+... + | uuid | device_node | device_num | device_type | size_mib | available_mib |... + +--------------------------------------+-------------+------------+-------------+----------+---------------+... + | 2dcc3d33-ba43-4b5a-b0b0-2b5a2e5737b7 | /dev/sda | 2048 | HDD | 600000 | 434072 |... + | f53437c6-77e3-4185-9453-67eaa8b461b1 | /dev/sdb | 2064 | HDD | 16240 | 16237 |... + | 623bbfc0-2b38-432a-acf4-a28db6066cce | /dev/sdc | 2080 | HDD | 16240 | 16237 |... + +--------------------------------------+-------------+------------+-------------+----------+---------------+... + +Create the 'cinder-volumes' local volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-1 nova-local + +-----------------+-------------------------------------------------------------------+ + | Property | Value | + +-----------------+-------------------------------------------------------------------+ + | lvm_vg_name | nova-local | + | vg_state | adding | + | uuid | d205f839-0930-4e77-9f10-97089150f812 | + | ihost_uuid | be0f35f7-31d9-4145-bf6a-0c2556cf398c | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-29T06:28:28.226907+00:00 | + | updated_at | None | + | parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} | + +-----------------+-------------------------------------------------------------------+ + +Create a disk partition to add to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-1 623bbfc0-2b38-432a-acf4-a28db6066cce 16237 -t lvm_phys_vol + +-------------+--------------------------------------------------+ + | Property | Value | + +-------------+--------------------------------------------------+ + | device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 | + | device_node | /dev/sdc1 | + | type_guid | ba5eba11-0000-1111-2222-000000000001 | + | type_name | None | + | start_mib | None | + | end_mib | None | + | size_mib | 16237 | + | uuid | f7bc6095-9375-49fe-83c7-12601c202376 | + | ihost_uuid | be0f35f7-31d9-4145-bf6a-0c2556cf398c | + | idisk_uuid | 623bbfc0-2b38-432a-acf4-a28db6066cce | + | ipv_uuid | None | + | status | Creating (on unlock) | + | created_at | 2018-08-29T06:30:33.176428+00:00 | + | updated_at | None | + +-------------+--------------------------------------------------+ + +Wait for the new partition to be created (i.e. status=Ready): + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-1 + +--------------------------------------+...+-------------+...+-----------+----------+----------------------+ + | uuid |...| device_node |...| type_name | size_mib | status | + +--------------------------------------+...+-------------+...+-----------+----------+----------------------+ + | 7a41aab0-6695-4d16-9003-73238adda75b |...| /dev/sdb1 |...| None | 16237 | Creating (on unlock) | + | f7bc6095-9375-49fe-83c7-12601c202376 |...| /dev/sdc1 |...| None | 16237 | Creating (on unlock) | + +--------------------------------------+...+-------------+...+-----------+----------+----------------------+ + +Add the partition to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-1 nova-local f7bc6095-9375-49fe-83c7-12601c202376 + +--------------------------+--------------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------------+ + | uuid | 94003d64-4e1b-483a-8a6c-0fc4b1e0e272 | + | pv_state | adding | + | pv_type | partition | + | disk_or_part_uuid | f7bc6095-9375-49fe-83c7-12601c202376 | + | disk_or_part_device_node | /dev/sdc1 | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 | + | lvm_pv_name | /dev/sdc1 | + | lvm_vg_name | nova-local | + | lvm_pv_uuid | None | + | lvm_pv_size | 0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | be0f35f7-31d9-4145-bf6a-0c2556cf398c | + | created_at | 2018-08-29T06:33:08.625604+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------------+ + +********************** +Unlocking controller-1 +********************** + +Unlock controller-1: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-1 + +Wait while the controller-1 is rebooted. Up to 10 minutes may be +required for a reboot, depending on hardware. + +REMARK: controller-1 will remain in degraded state until data-syncing +is complete. The duration is dependant on the virtualization host's +configuration - i.e., the number and configuration of physical disks +used to host the nodes' virtual disks. Also, the management network is +expected to have link capacity of 10000 (1000 is not supported due to +excessive data-sync time). Use 'fm alarm-list' to confirm status. + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | unlocked | enabled | available | + +----+--------------+-------------+----------------+-------------+--------------+ + +----------------------------------- +Extending the compute node capacity +----------------------------------- + +You can add up to four compute nodes to the All-in-One Duplex deployment. + +************************** +Compute hosts installation +************************** + +After initializing and configuring the two controllers, you can add up +to four additional compute hosts. To add a host, do the following: + +^^^^^^^^^^^^^^^^^ +Initializing host +^^^^^^^^^^^^^^^^^ + +Power on the host. The following appears in the host console: + +:: + + Waiting for this node to be configured. + + Please configure the personality for this node from the + controller node in order to proceed. + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Updating the hostname and personality +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +Wait for the controller-0 to both discover the new host and to list that host +as UNKNOWN in the table: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | online | + | 2 | controller-1 | controller | unlocked | enabled | online | + | 3 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +Use the system host-update command to update the host personality attribute: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-update 3 personality=compute hostname=compute-0 + +See also: 'system help host-update'. + +Unless it is known that the host's configuration can support the +installation of more than one node, it is recommended that the +installation and configuration of each node be serialized. For example, +if the virtual disks for the entire cluster are hosted on the host's +root disk and that disk happens to be a single rotational type hard disk, +then the host cannot reliably support parallel node installation. + +^^^^^^^^^^^^^^^ +Monitoring host +^^^^^^^^^^^^^^^ + +On controller-0, you can monitor the installation progress by periodically +running the system host-show command for the host. Progress is +shown in the install_state field. + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-show | grep install + | install_output | text | + | install_state | booting | + | install_state_info | None | + +Wait while the host is installed, configured, and rebooted. Depending on +hardware, it could take up to 20 minutes for this process to complete. +When the reboot is complete, the host is reported as locked, disabled, +and online. + +^^^^^^^^^^^^^ +Listing hosts +^^^^^^^^^^^^^ + +You can use the system host-list command to list the hosts once the node +has been installed, configured, and rebooted: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | online | + | 2 | controller-1 | controller | unlocked | enabled | online | + | 3 | compute-0 | compute | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +***************************** +Provisioning the compute host +***************************** + +You must configure the network interfaces and the storage disks on a +host before you can unlock it. For each compute host, do the following: + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Provisioning network interfaces on a compute host +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In order to identify hardware port names, types, and discovered +pci-addresses on controller-0, list the host ports: + +- **Only in virtual environment**: Ensure that the interface used is + one of those attached to the host bridge with model type "virtio" (i.e. + eth1000 and eth1001). The model type "e1000" emulated devices will + not work for provider networks: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-port-list compute-0 + +--------------------------------------+---------+----------+--------------+... + | uuid | name | type | pci address |... + +--------------------------------------+---------+----------+--------------+... + | de9ec830-cf33-4a06-8985-cd3262f6ecd3 | enp2s1 | ethernet | 0000:02:01.0 |... + | 9def88fb-d871-4542-83e2-d4b6e35169a1 | enp2s2 | ethernet | 0000:02:02.0 |... + | b2e38da9-840f-446c-b466-ceb6479266f6 | eth1000 | ethernet | 0000:02:03.0 |... + | c1694675-643d-4ba7-b821-cd147450112e | eth1001 | ethernet | 0000:02:04.0 |... + +--------------------------------------+---------+----------+--------------+... + +Use the following command to provision the data interface for compute: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -p providernet-a -c data compute-0 eth1000 + +^^^^^^^^^^^^^^^^^^^^^^^^^^^ +VSwitch virtual environment +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +**Only in virtual environment**. If the compute node has more than four CPUs, +the system auto-configures the vswitch to use two cores. However, some virtual +environments do not properly support multi-queue, which is required in a +multi-CPU environment. Therefore, run the following command to reduce the +vswitch cores to one: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-cpu-modify compute-0 -f vswitch -p0 1 + +--------------------------------------+-------+-----------+-------+--------+... + | uuid | log_c | processor | phy_c | thread |... + +--------------------------------------+-------+-----------+-------+--------+... + | 9d53c015-8dd5-4cd8-9abb-6f231fa773c2 | 0 | 0 | 0 | 0 |... + | 8620eaa7-19af-4ef7-a5aa-690ed57f01c7 | 1 | 0 | 1 | 0 |... + | f26598a5-8029-4d20-999e-2ec5d532d62e | 2 | 0 | 2 | 0 |... + | f7b3ce4a-10da-485d-873c-4e6988f7e9cf | 3 | 0 | 3 | 0 |... + +--------------------------------------+-------+-----------+-------+--------+... + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Provisioning storage on a compute host +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Review the available disk space and capacity and then obtain the uuid(s) of +the physical disk(s) to be used for nova local: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list compute-0 + +--------------------------------------+-----------+---------+---------+-------+------------+... + | uuid | device_no | device_ | device_ | size_ | available |... + | | de | num | type | gib | gib |... + +--------------------------------------+-----------+---------+---------+-------+------------+... + | 0ae45272-c9f4-4824-8405-f6c8946fda1e | /dev/sda | 2048 | HDD | 200.0 | 120.976 |... + | d751abfe-de57-4b23-b166-1d3d5b4d5ca6 | /dev/sdb | 2064 | HDD | 200.0 | 199.997 |... + +--------------------------------------+-----------+---------+---------+-------+------------+... + +Use the following command to create the 'nova-local' local volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add compute-0 nova-local + +-----------------------+-------------------------------------------------------------------+ + | Property | Value | + +-----------------------+-------------------------------------------------------------------+ + | lvm_vg_name | nova-local | + | vg_state | adding | + | uuid | 78e84a57-c40e-438c-97a4-49ebec6394d1 | + | ihost_uuid | 6f2de9b6-c55c-47b0-b40e-aff47f1e1bda | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size_gib | 0.0 | + | lvm_vg_avail_size_gib | 0.0 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2019-01-15T12:31:45.796538+00:00 | + | updated_at | None | + | parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} | + +-----------------------+-------------------------------------------------------------------+ + +Use the following command to create a disk partition to add to the volume +group based on the uuid of the physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add compute-0 nova-local d751abfe-de57-4b23-b166-1d3d5b4d5ca6 + +--------------------------+--------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------+ + | uuid | bba20f20-81fc-40a2-bf1e-a2fae849625b | + | pv_state | adding | + | pv_type | disk | + | disk_or_part_uuid | d751abfe-de57-4b23-b166-1d3d5b4d5ca6 | + | disk_or_part_device_node | /dev/sdb | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:1f.2-ata-2.0 | + | lvm_pv_name | /dev/sdb | + | lvm_vg_name | nova-local | + | lvm_pv_uuid | None | + | lvm_pv_size_gib | 0.0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | 6f2de9b6-c55c-47b0-b40e-aff47f1e1bda | + | created_at | 2019-01-15T12:32:40.535160+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------+ + +^^^^^^^^^^^^^^^^^^^^^^^^ +Unlocking a compute host +^^^^^^^^^^^^^^^^^^^^^^^^ + +On controller-0, use the system host-unlock command to unlock the +compute node: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock compute-0 + +Wait while the compute node is rebooted and re-configured. Depending on +hardware, it can take up to 10 minutes for the reboot to complete. Once +the reboot is complete, the nodes availability state reports as "in-test" +and is followed by unlocked/enabled. + +------------------- +System health check +------------------- + +*********************** +Listing StarlingX nodes +*********************** + +On controller-0, after a few minutes, all nodes are reported as +unlocked, enabled, and available: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | unlocked | enabled | available | + | 3 | compute-0 | compute | unlocked | enabled | available | + +----+--------------+-------------+----------------+-------------+--------------+ + +***************** +System alarm list +***************** + +When all nodes are unlocked, enabled, and available: check 'fm alarm-list' for +issues. + +Your StarlingX deployment is now up and running with 2x HA controllers with +Cinder storage and all OpenStack services up and running. You can now proceed +with standard OpenStack APIs, CLIs and/or Horizon to load Glance images, +configure Nova Flavors, configure Neutron networks and launch Nova virtual +machines. + +---------------------- +Deployment terminology +---------------------- + +.. include:: deployment_terminology.rst + :start-after: incl-simplex-deployment-terminology: + :end-before: incl-simplex-deployment-terminology-end: + +.. include:: deployment_terminology.rst + :start-after: incl-standard-controller-deployment-terminology: + :end-before: incl-standard-controller-deployment-terminology-end: + +.. include:: deployment_terminology.rst + :start-after: incl-common-deployment-terminology: + :end-before: incl-common-deployment-terminology-end: diff --git a/doc/source/installation_guide/latest/figures/starlingx-deployment-options-controller-storage.png b/doc/source/installation_guide/latest/figures/starlingx-deployment-options-controller-storage.png new file mode 100644 index 000000000..c1d028acf Binary files /dev/null and b/doc/source/installation_guide/latest/figures/starlingx-deployment-options-controller-storage.png differ diff --git a/doc/source/installation_guide/latest/figures/starlingx-deployment-options-dedicated-storage.png b/doc/source/installation_guide/latest/figures/starlingx-deployment-options-dedicated-storage.png new file mode 100644 index 000000000..00e4c3eba Binary files /dev/null and b/doc/source/installation_guide/latest/figures/starlingx-deployment-options-dedicated-storage.png differ diff --git a/doc/source/installation_guide/latest/figures/starlingx-deployment-options-duplex-extended.png b/doc/source/installation_guide/latest/figures/starlingx-deployment-options-duplex-extended.png new file mode 100644 index 000000000..64bb7f5ab Binary files /dev/null and b/doc/source/installation_guide/latest/figures/starlingx-deployment-options-duplex-extended.png differ diff --git a/doc/source/installation_guide/latest/figures/starlingx-deployment-options-duplex.png b/doc/source/installation_guide/latest/figures/starlingx-deployment-options-duplex.png new file mode 100644 index 000000000..3638e4a10 Binary files /dev/null and b/doc/source/installation_guide/latest/figures/starlingx-deployment-options-duplex.png differ diff --git a/doc/source/installation_guide/latest/figures/starlingx-deployment-options-simplex.png b/doc/source/installation_guide/latest/figures/starlingx-deployment-options-simplex.png new file mode 100644 index 000000000..317b7cf49 Binary files /dev/null and b/doc/source/installation_guide/latest/figures/starlingx-deployment-options-simplex.png differ diff --git a/doc/source/installation_guide/latest/index.rst b/doc/source/installation_guide/latest/index.rst new file mode 100644 index 000000000..58d3fd0cd --- /dev/null +++ b/doc/source/installation_guide/latest/index.rst @@ -0,0 +1,288 @@ +============================== +Installation guide stx.2019.05 +============================== + +This is the installation guide for release stx.2019.05. If an installation +guide is needed for a previous release, review the +:doc:`installation guides for previous releases `. + +------------ +Introduction +------------ + +StarlingX may be installed in: + +- **Bare metal**: Real deployments of StarlingX are only supported on + physical servers. +- **Virtual environment**: It should only be used for evaluation or + development purposes. + +StarlingX installed in virtual environments has two options: + +- :doc:`Libvirt/QEMU ` +- VirtualBox + +------------ +Requirements +------------ + +Different use cases require different configurations. + +********** +Bare metal +********** + +The minimum requirements for the physical servers where StarlingX might +be deployed, include: + +- **Controller hosts** + + - Minimum processor is: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 + cores/socket + + - Minimum memory: 64 GB + - Hard drives: + + - Primary hard drive, minimum 500 GB for OS and system databases. + - Secondary hard drive, minimum 500 GB for persistent VM storage. + + - 2 physical Ethernet interfaces: OAM and MGMT network. + - USB boot support. + - PXE boot support. + +- **Storage hosts** + + - Minimum processor is: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 + cores/socket. + + - Minimum memory: 64 GB. + - Hard drives: + + - Primary hard drive, minimum 500 GB for OS. + - 1 or more additional hard drives for CEPH OSD storage, and + - Optionally 1 or more SSD or NVMe drives for CEPH journals. + + - 1 physical Ethernet interface: MGMT network + - PXE boot support. + +- **Compute hosts** + + - Minimum processor is: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 + cores/socket. + + - Minimum memory: 32 GB. + - Hard drives: + + - Primary hard drive, minimum 500 GB for OS. + - 1 or more additional hard drives for ephemeral VM storage. + + - 2 or more physical Ethernet interfaces: MGMT network and 1 or more + provider networks. + - PXE boot support. + +- **All-In-One Simplex or Duplex, controller + compute hosts** + + - Minimum processor is: + + - Typical hardware form factor: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket + - Low cost / low power hardware form factor + + - Single-CPU Intel Xeon D-15xx family, 8 cores + + - Minimum memory: 64 GB. + - Hard drives: + + - Primary hard drive, minimum 500 GB SDD or NVMe. + - 0 or more 500 GB disks (min. 10K RPM). + + - Network ports: + + **NOTE:** Duplex and Simplex configurations require one or more data + ports. + The Duplex configuration requires a management port. + + - Management: 10GE (Duplex only) + - OAM: 10GE + - Data: n x 10GE + +The recommended minimum requirements for the physical servers are +described later in each StarlingX deployment guide. + +^^^^^^^^^^^^^^^^^^^^^^^^ +NVMe drive as boot drive +^^^^^^^^^^^^^^^^^^^^^^^^ + +To use a Non-Volatile Memory Express (NVMe) drive as the boot drive for any of +your nodes, you must configure your host and adjust kernel parameters during +installation: + +- Configure the host to be in UEFI mode. +- Edit the kernel boot parameter. After you are presented with the StarlingX + ISO boot options and after you have selected the preferred installation option + (e.g. Standard Configuration / All-in-One Controller Configuration), press the + TAB key to edit the kernel boot parameters. Modify the **boot_device** and + **rootfs_device** from the default **sda** so that it is the correct device + name for the NVMe drive (e.g. "nvme0n1"). + + :: + + vmlinuz rootwait console=tty0 inst.text inst.stage2=hd:LABEL=oe_iso_boot + inst.ks=hd:LABEL=oe_iso_boot:/smallsystem_ks.cfg boot_device=nvme0n1 + rootfs_device=nvme0n1 biosdevname=0 usbcore.autosuspend=-1 inst.gpt + security_profile=standard user_namespace.enable=1 initrd=initrd.img + + +******************* +Virtual environment +******************* + +The recommended minimum requirements for the workstation, hosting the +virtual machine(s) where StarlingX will be deployed, include: + +^^^^^^^^^^^^^^^^^^^^^ +Hardware requirements +^^^^^^^^^^^^^^^^^^^^^ + +A workstation computer with: + +- Processor: x86_64 only supported architecture with BIOS enabled + hardware virtualization extensions +- Cores: 8 (4 with careful monitoring of cpu load) +- Memory: At least 32GB RAM +- Hard Disk: 500GB HDD +- Network: Two network adapters with active Internet connection + +^^^^^^^^^^^^^^^^^^^^^ +Software requirements +^^^^^^^^^^^^^^^^^^^^^ + +A workstation computer with: + +- Operating System: Freshly installed Ubuntu 16.04 LTS 64-bit +- Proxy settings configured (if applies) +- Git +- KVM/VirtManager +- Libvirt library +- QEMU full-system emulation binaries +- stx-tools project +- StarlingX ISO image + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Deployment environment setup +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This section describes how to set up the workstation computer which will +host the virtual machine(s) where StarlingX will be deployed. + +'''''''''''''''''''''''''''''' +Updating your operating system +'''''''''''''''''''''''''''''' + +Before proceeding with the build, ensure your OS is up to date. You’ll +first need to update the local database list of available packages: + +:: + + $ sudo apt-get update + +''''''''''''''''''''''''' +Install stx-tools project +''''''''''''''''''''''''' + +Clone the stx-tools project. Usually you’ll want to clone it under your +user’s home directory. + +:: + + $ cd $HOME + $ git clone https://git.starlingx.io/stx-tools + + +'''''''''''''''''''''''''''''''''''''''' +Installing requirements and dependencies +'''''''''''''''''''''''''''''''''''''''' + +Navigate to the stx-tools installation libvirt directory: + +:: + + $ cd $HOME/stx-tools/deployment/libvirt/ + + +Install the required packages: + +:: + + $ bash install_packages.sh + + +'''''''''''''''''' +Disabling firewall +'''''''''''''''''' + +Unload firewall and disable firewall on boot: + +:: + + $ sudo ufw disable + Firewall stopped and disabled on system startup + $ sudo ufw status + Status: inactive + + +------------------------------- +Getting the StarlingX ISO image +------------------------------- + +Follow the instructions from the :doc:`/developer_guide/2018_10/index` to build a +StarlingX ISO image. + +********** +Bare metal +********** + +A bootable USB flash drive containing StarlingX ISO image. + + +******************* +Virtual environment +******************* + +Copy the StarlingX ISO Image to the stx-tools deployment libvirt project +directory: + +:: + + $ cp $HOME/stx-tools/deployment/libvirt/ + + +------------------ +Deployment options +------------------ + +- Standard controller + + - :doc:`StarlingX Cloud with Dedicated Storage ` + - :doc:`StarlingX Cloud with Controller Storage ` + +- All-in-one + + - :doc:`StarlingX Cloud Duplex ` + - :doc:`StarlingX Cloud Simplex ` + +.. toctree:: + :hidden: + + installation_libvirt_qemu + controller_storage + dedicated_storage + duplex + simplex diff --git a/doc/source/installation_guide/installation_libvirt_qemu.rst b/doc/source/installation_guide/latest/installation_libvirt_qemu.rst similarity index 72% rename from doc/source/installation_guide/installation_libvirt_qemu.rst rename to doc/source/installation_guide/latest/installation_libvirt_qemu.rst index 9b4553462..84e02c3b9 100644 --- a/doc/source/installation_guide/installation_libvirt_qemu.rst +++ b/doc/source/installation_guide/latest/installation_libvirt_qemu.rst @@ -1,13 +1,11 @@ -.. _Installation-libvirt-qemu: +===================================== +Installation libvirt qemu stx.2019.05 +===================================== -========================= -Installation libvirt qemu -========================= - -Installation for StarlingX using Libvirt/QEMU virtualization. +Installation for StarlingX stx.2019.05 using Libvirt/QEMU virtualization. --------------------- -Hardware Requirements +Hardware requirements --------------------- A workstation computer with: @@ -15,28 +13,27 @@ A workstation computer with: - Processor: x86_64 only supported architecture with BIOS enabled hardware virtualization extensions - Memory: At least 32GB RAM -- Hard Disk: 500GB HDD +- Hard disk: 500GB HDD - Network: One network adapter with active Internet connection --------------------- -Software Requirements +Software requirements --------------------- A workstation computer with: -- Operating System: This process is known to work on Ubuntu 16.04 and - is likely to work on other Linux OS's with some appropriate - adjustments. +- Operating system: This process is known to work on Ubuntu 16.04 and + is likely to work on other Linux OS's with some appropriate adjustments. - Proxy settings configured (if applies) - Git - KVM/VirtManager -- Libvirt Library -- QEMU Full System Emulation Binaries +- Libvirt library +- QEMU full-system emulation binaries - stx-tools project -- StarlingX ISO Image +- StarlingX ISO image ---------------------------- -Deployment Environment Setup +Deployment environment setup ---------------------------- ************* @@ -76,7 +73,7 @@ This rc file shows the defaults baked into the scripts: ************************* -Install stx-tools Project +Install stx-tools project ************************* Clone the stx-tools project into a working directory. @@ -102,7 +99,7 @@ If you created a configuration, load it from stxcloud.rc: **************************************** -Installing Requirements and Dependencies +Installing requirements and dependencies **************************************** Install the required packages and configure QEMU. This only needs to be @@ -115,7 +112,7 @@ time): ****************** -Disabling Firewall +Disabling firewall ****************** Unload firewall and disable firewall on boot: @@ -127,7 +124,7 @@ Unload firewall and disable firewall on boot: ****************** -Configure Networks +Configure networks ****************** Configure the network bridges using setup_network.sh before doing @@ -148,11 +145,11 @@ There is also a script cleanup_network.sh that will remove networking configuration from libvirt. ********************* -Configure Controllers +Configure controllers ********************* -One script exists for building different StarlingX cloud -configurations: setup_configuration.sh. +One script exists for building different StarlingX cloud configurations: +setup_configuration.sh. The script uses the cloud configuration with the -c option: @@ -193,15 +190,15 @@ Tear down the VMs using destroy_configuration.sh. Continue -------- -Pick up the installation in one of the existing guides at the -'Initializing Controller-0 step. +Pick up the installation in one of the existing guides at the initializing +controller-0 step. -- Standard Controller +- Standard controller - - :ref:`StarlingX Cloud with Dedicated Storage Virtual Environment ` - - :ref:`StarlingX Cloud with Controller Storage Virtual Environment ` + - :doc:`StarlingX Cloud with Dedicated Storage Virtual Environment ` + - :doc:`StarlingX Cloud with Controller Storage Virtual Environment ` - All-in-one - - :ref:`StarlingX Cloud Duplex Virtual Environment ` - - :ref:`StarlingX Cloud Simplex Virtual Environment ` + - :doc:`StarlingX Cloud Duplex Virtual Environment ` + - :doc:`StarlingX Cloud Simplex Virtual Environment ` diff --git a/doc/source/installation_guide/latest/simplex.rst b/doc/source/installation_guide/latest/simplex.rst new file mode 100644 index 000000000..d3ea0c826 --- /dev/null +++ b/doc/source/installation_guide/latest/simplex.rst @@ -0,0 +1,729 @@ +=============================================== +All-In-One Simplex deployment guide stx.2019.05 +=============================================== + +.. contents:: + :local: + :depth: 1 + +**NOTE:** The instructions to setup a StarlingX One Node Configuration +(AIO-SX) system with containerized openstack services in this guide +are under development. +For approved instructions, see the +`One Node Configuration wiki page `__. + +---------------------- +Deployment description +---------------------- + +The All-In-One Simplex (AIO-SX) deployment option provides all three cloud +gunctions (controller, compute, and storage) on a single physical server. With +these cloud functions, multiple application types can be deployed and +consolidated onto a single physical server. For example, with a AIO-SX +deployment you can: + +- Consolidate legacy applications that must run standalone on a server by using + multiple virtual machines on a single physical server. +- Consolidate legacy applications that run on different operating systems or + different distributions of operating systems by using multiple virtual + machines on a single physical server. + +Only a small amount of cloud processing / storage power is required with an +All-In-One Simplex deployment. + +.. figure:: figures/starlingx-deployment-options-simplex.png + :scale: 50% + :alt: All-In-One Simplex deployment configuration + + *All-In-One Simplex deployment configuration* + +An All-In-One Simplex deployment provides no protection against an overall +server hardware fault. Protection against overall server hardware fault is +either not required, or done at a higher level. Hardware component protection +could be enabled if, for example, an HW RAID or 2x Port LAG is used in the +deployment. + +-------------------------------------- +Preparing an All-In-One Simplex server +-------------------------------------- + +********** +Bare metal +********** + +Required Server: + +- Combined server (controller + compute): 1 + +^^^^^^^^^^^^^^^^^^^^^ +Hardware requirements +^^^^^^^^^^^^^^^^^^^^^ + +The recommended minimum requirements for the physical servers where +All-In-One Simplex will be deployed are: + +- Minimum processor: + + - Typical hardware form factor: + + - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket + - Low cost / low power hardware form factor + + - Single-CPU Intel Xeon D-15xx family, 8 cores + +- Memory: 64 GB +- BIOS: + + - Hyper-Threading technology: Enabled + - Virtualization technology: Enabled + - VT for directed I/O: Enabled + - CPU power and performance policy: Performance + - CPU C state control: Disabled + - Plug & play BMC detection: Disabled + +- Primary disk: + + - 500 GB SDD or NVMe + +- Additional disks: + + - Zero or more 500 GB disks (min. 10K RPM) + +- Network ports + + **NOTE:** All-In-One Simplex configuration requires one or more data ports. + This configuration does not require a management port. + + - OAM: 10GE + - Data: n x 10GE + +******************* +Virtual environment +******************* + +Run the libvirt qemu setup scripts. Setting up virtualized OAM and +management networks: + +:: + + $ bash setup_network.sh + +Building XML for definition of virtual servers: + +:: + + $ bash setup_configuration.sh -c simplex -i + +The default XML server definition created by the previous script is: + +- simplex-controller-0 + +^^^^^^^^^^^^^^^^^^^^^^^^^ +Power up a virtual server +^^^^^^^^^^^^^^^^^^^^^^^^^ + +To power up the virtual server, run the following command: + +:: + + $ sudo virsh start + +e.g. + +:: + + $ sudo virsh start simplex-controller-0 + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Access a virtual server console +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The XML for virtual servers in stx-tools repo, deployment/libvirt, +provides both graphical and text consoles. + +Access the graphical console in virt-manager by right-click on the +domain (the server) and selecting "Open". + +Access the textual console with the command "virsh console $DOMAIN", +where DOMAIN is the name of the server shown in virsh. + +When booting the controller-0 for the first time, both the serial and +graphical consoles will present the initial configuration menu for the +cluster. One can select serial or graphical console for controller-0. +For the other nodes however only serial is used, regardless of which +option is selected. + +Open the graphic console on all servers before powering them on to +observe the boot device selection and PXI boot progress. Run "virsh +console $DOMAIN" command promptly after power on to see the initial boot +sequence which follows the boot device selection. One has a few seconds +to do this. + +------------------------------ +Installing the controller host +------------------------------ + +Installing controller-0 involves initializing a host with software and +then applying a bootstrap configuration from the command line. The +configured bootstrapped host becomes controller-0. + +Procedure: + +#. Power on the server that will be controller-0 with the StarlingX ISO + on a USB in a bootable USB slot. +#. Configure the controller using the config_controller script. + +************************* +Initializing controller-0 +************************* + +This section describes how to initialize StarlingX in host controller-0. +Except where noted, all the commands must be executed from a console of +the host. + +Power on the host to be configured as controller-0, with the StarlingX +ISO on a USB in a bootable USB slot. Wait for the console to show the +StarlingX ISO booting options: + +- **All-in-one Controller Configuration** + + - When the installer is loaded and the installer welcome screen + appears in the controller-0 host, select the type of installation + "All-in-one Controller Configuration". + +- **Graphical Console** + + - Select the "Graphical Console" as the console to use during + installation. + +- **Standard Security Boot Profile** + + - Select "Standard Security Boot Profile" as the Security Profile. + +Monitor the initialization. When it is complete, a reboot is initiated +on the controller-0 host, briefly displays a GNU GRUB screen, and then +boots automatically into the StarlingX image. + +Log into controller-0 as user wrsroot, with password wrsroot. The +first time you log in as wrsroot, you are required to change your +password. Enter the current password (wrsroot): + +:: + + Changing password for wrsroot. + (current) UNIX Password: + + +Enter a new password for the wrsroot account: + +:: + + New password: + +Enter the new password again to confirm it: + +:: + + Retype new password: + +controller-0 is initialized with StarlingX, and is ready for configuration. + +************************ +Configuring controller-0 +************************ + +This section describes how to perform the controller-0 configuration +interactively just to bootstrap system with minimum critical data. +Except where noted, all the commands must be executed from the console +of the active controller (here assumed to be controller-0). + +When run interactively, the config_controller script presents a series +of prompts for initial configuration of StarlingX: + +- For the virtual environment, you can accept all the default values + immediately after ‘system date and time’. +- For a physical deployment, answer the bootstrap configuration + questions with answers applicable to your particular physical setup. + +The script is used to configure the first controller in the StarlingX +cluster as controller-0. The prompts are grouped by configuration +area. To start the script interactively, use the following command +with no parameters: + +:: + + controller-0:~$ sudo config_controller + System Configuration + ================ + Enter ! at any prompt to abort... + ... + +Select [y] for System date and time: + +:: + + System date and time: + ----------------------------- + + Is the current date and time correct? [y/N]: y + +For System mode choose "simplex": + +:: + + ... + 1) duplex-direct: two node-redundant configuration. Management and + infrastructure networks are directly connected to peer ports + 2) duplex - two node redundant configuration + 3) simplex - single node non-redundant configuration + System mode [duplex-direct]: 3 + +After System date and time and System mode: + +:: + + Applying configuration (this will take several minutes): + + 01/08: Creating bootstrap configuration ... DONE + 02/08: Applying bootstrap manifest ... DONE + 03/08: Persisting local configuration ... DONE + 04/08: Populating initial system inventory ... DONE + 05:08: Creating system configuration ... DONE + 06:08: Applying controller manifest ... DONE + 07:08: Finalize controller configuration ... DONE + 08:08: Waiting for service activation ... DONE + + Configuration was applied + + Please complete any out of service commissioning steps with system + commands and unlock controller to proceed. + +After config_controller bootstrap configuration, REST API, CLI and +Horizon interfaces are enabled on the controller-0 OAM IP address. The +remaining installation instructions will use the CLI. + +-------------------------------- +Provisioning the controller host +-------------------------------- + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +********************************************* +Configuring provider networks at installation +********************************************* + +Set up one provider network of the vlan type, named providernet-a: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-create providernet-a --type=vlan + [wrsroot@controller-0 ~(keystone_admin)]$ neutron providernet-range-create --name providernet-a-range1 --range 100-400 providernet-a + +***************************************** +Providing data interfaces on controller-0 +***************************************** + +List all interfaces: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-list -a controller-0 + +--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+.. + | uuid | name | class |...| vlan | ports | uses | used by | attributes |.. + | | | |...| id | | i/f | i/f | |.. + +--------------------------------------+----------+---------+...+------+--------------+------+---------+------------+.. + | 49fd8938-e76f-49f1-879e-83c431a9f1af | enp0s3 | platform |...| None | [u'enp0s3'] | [] | [] | MTU=1500 |.. + | 8957bb2c-fec3-4e5d-b4ed-78071f9f781c | eth1000 | None |...| None | [u'eth1000'] | [] | [] | MTU=1500 |.. + | bf6f4cad-1022-4dd7-962b-4d7c47d16d54 | eth1001 | None |...| None | [u'eth1001'] | [] | [] | MTU=1500 |.. + | f59b9469-7702-4b46-bad5-683b95f0a1cb | enp0s8 | platform |...| None | [u'enp0s8'] | [] | [] | MTU=1500 |.. + +--------------------------------------+---------+----------+...+------+--------------+------+---------+------------+.. + +Configure the data interfaces: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-if-modify -c data controller-0 eth1000 -p providernet-a + +------------------+--------------------------------------+ + | Property | Value | + +------------------+--------------------------------------+ + | ifname | eth1000 | + | iftype | ethernet | + | ports | [u'eth1000'] | + | providernetworks | providernet-a | + | imac | 08:00:27:c4:ad:3e | + | imtu | 1500 | + | ifclass | data | + | aemode | None | + | schedpolicy | None | + | txhashpolicy | None | + | uuid | 8957bb2c-fec3-4e5d-b4ed-78071f9f781c | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | vlan_id | None | + | uses | [] | + | used_by | [] | + | created_at | 2018-08-28T12:50:51.820151+00:00 | + | updated_at | 2018-08-28T14:46:18.333109+00:00 | + | sriov_numvfs | 0 | + | ipv4_mode | disabled | + | ipv6_mode | disabled | + | accelerated | [True] | + +------------------+--------------------------------------+ + +************************************* +Configuring Cinder on controller disk +************************************* + +Review the available disk space and capacity and obtain the uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 + +--------------------------------------+-----------+---------+---------+---------+------------+... + | uuid | device_no | device_ | device_ | size_mi | available_ |... + | | de | num | type | b | mib |... + +--------------------------------------+-----------+---------+---------+---------+------------+... + | 6b42c9dc-f7c0-42f1-a410-6576f5f069f1 | /dev/sda | 2048 | HDD | 600000 | 434072 |... + | | | | | | |... + | | | | | | |... + | 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 | /dev/sdb | 2064 | HDD | 16240 | 16237 |... + | | | | | | |... + | | | | | | |... + | 146195b2-f3d7-42f9-935d-057a53736929 | /dev/sdc | 2080 | HDD | 16240 | 16237 |... + | | | | | | |... + | | | | | | |... + +--------------------------------------+-----------+---------+---------+---------+------------+... + +Create the 'cinder-volumes' local volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-0 cinder-volumes + +-----------------+--------------------------------------+ + | lvm_vg_name | cinder-volumes | + | vg_state | adding | + | uuid | 61cb5cd2-171e-4ef7-8228-915d3560cdc3 | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-28T13:45:20.218905+00:00 | + | updated_at | None | + | parameters | {u'lvm_type': u'thin'} | + +-----------------+--------------------------------------+ + +Create a disk partition to add to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-0 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 16237 -t lvm_phys_vol + +-------------+--------------------------------------------------+ + | Property | Value | + +-------------+--------------------------------------------------+ + | device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 | + | device_node | /dev/sdb1 | + | type_guid | ba5eba11-0000-1111-2222-000000000001 | + | type_name | None | + | start_mib | None | + | end_mib | None | + | size_mib | 16237 | + | uuid | 0494615f-bd79-4490-84b9-dcebbe5f377a | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | idisk_uuid | 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 | + | ipv_uuid | None | + | status | Creating | + | created_at | 2018-08-28T13:45:48.512226+00:00 | + | updated_at | None | + +-------------+--------------------------------------------------+ + +Wait for the new partition to be created (i.e. status=Ready): + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-0 --disk 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + | uuid |...| device_nod |...| type_name | size_mib | status | + | |...| e |...| | | | + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + | 0494615f-bd79-4490-84b9-dcebbe5f377a |...| /dev/sdb1 |...| LVM Physical Volume | 16237 | Ready | + | |...| |...| | | | + | |...| |...| | | | + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + +Add the partition to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-0 cinder-volumes 0494615f-bd79-4490-84b9-dcebbe5f377a + +--------------------------+--------------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------------+ + | uuid | 9a0ad568-0ace-4d57-9e03-e7a63f609cf2 | + | pv_state | adding | + | pv_type | partition | + | disk_or_part_uuid | 0494615f-bd79-4490-84b9-dcebbe5f377a | + | disk_or_part_device_node | /dev/sdb1 | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0-part1 | + | lvm_pv_name | /dev/sdb1 | + | lvm_vg_name | cinder-volumes | + | lvm_pv_uuid | None | + | lvm_pv_size | 0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | created_at | 2018-08-28T13:47:39.450763+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------------+ + +********************************************* +Adding an LVM storage backend at installation +********************************************* + +Ensure requirements are met to add LVM storage: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add lvm -s cinder + + WARNING : THIS OPERATION IS NOT REVERSIBLE AND CANNOT BE CANCELLED. + + By confirming this operation, the LVM backend will be created. + + Please refer to the system admin guide for minimum spec for LVM + storage. Set the 'confirmed' field to execute this operation + for the lvm backend. + +Add the LVM storage backend: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-add lvm -s cinder --confirmed + + System configuration has changed. + Please follow the administrator guide to complete configuring the system. + + +--------------------------------------+------------+---------+-------------+...+----------+--------------+ + | uuid | name | backend | state |...| services | capabilities | + +--------------------------------------+------------+---------+-------------+...+----------+--------------+ + | 6d750a68-115a-4c26-adf4-58d6e358a00d | file-store | file | configured |...| glance | {} | + | e2697426-2d79-4a83-beb7-2eafa9ceaee5 | lvm-store | lvm | configuring |...| cinder | {} | + +--------------------------------------+------------+---------+-------------+...+----------+--------------+ + +Wait for the LVM storage backend to be configured (i.e. state=configured): + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system storage-backend-list + +--------------------------------------+------------+---------+------------+------+----------+--------------+ + | uuid | name | backend | state | task | services | capabilities | + +--------------------------------------+------------+---------+------------+------+----------+--------------+ + | 6d750a68-115a-4c26-adf4-58d6e358a00d | file-store | file | configured | None | glance | {} | + | e2697426-2d79-4a83-beb7-2eafa9ceaee5 | lvm-store | lvm | configured | None | cinder | {} | + +--------------------------------------+------------+---------+------------+------+----------+--------------+ + +*********************************************** +Configuring VM local storage on controller disk +*********************************************** + +Review the available disk space and capacity and obtain the uuid of the +physical disk: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 + +--------------------------------------+-----------+---------+---------+---------+------------+... + | uuid | device_no | device_ | device_ | size_mi | available_ |... + | | de | num | type | b | mib |... + +--------------------------------------+-----------+---------+---------+---------+------------+... + | 6b42c9dc-f7c0-42f1-a410-6576f5f069f1 | /dev/sda | 2048 | HDD | 600000 | 434072 |... + | | | | | | |... + | | | | | | |... + | 534352d8-fec2-4ca5-bda7-0e0abe5a8e17 | /dev/sdb | 2064 | HDD | 16240 | 0 |... + | | | | | | |... + | | | | | | |... + | 146195b2-f3d7-42f9-935d-057a53736929 | /dev/sdc | 2080 | HDD | 16240 | 16237 |... + | | | | | | |... + | | | | | | |... + +--------------------------------------+-----------+---------+---------+---------+------------+... + +Create the 'nova-local' volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-add controller-0 nova-local + +-----------------+-------------------------------------------------------------------+ + | Property | Value | + +-----------------+-------------------------------------------------------------------+ + | lvm_vg_name | nova-local | + | vg_state | adding | + | uuid | 517d313e-8aa0-4b4d-92e6-774b9085f336 | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | lvm_vg_access | None | + | lvm_max_lv | 0 | + | lvm_cur_lv | 0 | + | lvm_max_pv | 0 | + | lvm_cur_pv | 0 | + | lvm_vg_size | 0.00 | + | lvm_vg_total_pe | 0 | + | lvm_vg_free_pe | 0 | + | created_at | 2018-08-28T14:02:58.486716+00:00 | + | updated_at | None | + | parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'image'} | + +-----------------+-------------------------------------------------------------------+ + +Create a disk partition to add to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-add controller-0 146195b2-f3d7-42f9-935d-057a53736929 16237 -t lvm_phys_vol + +-------------+--------------------------------------------------+ + | Property | Value | + +-------------+--------------------------------------------------+ + | device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 | + | device_node | /dev/sdc1 | + | type_guid | ba5eba11-0000-1111-2222-000000000001 | + | type_name | None | + | start_mib | None | + | end_mib | None | + | size_mib | 16237 | + | uuid | 009ce3b1-ed07-46e9-9560-9d2371676748 | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | idisk_uuid | 146195b2-f3d7-42f9-935d-057a53736929 | + | ipv_uuid | None | + | status | Creating | + | created_at | 2018-08-28T14:04:29.714030+00:00 | + | updated_at | None | + +-------------+--------------------------------------------------+ + +Wait for the new partition to be created (i.e. status=Ready): + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-disk-partition-list controller-0 --disk 146195b2-f3d7-42f9-935d-057a53736929 + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + | uuid |...| device_nod |...| type_name | size_mib | status | + | |...| e |...| | | | + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + | 009ce3b1-ed07-46e9-9560-9d2371676748 |...| /dev/sdc1 |...| LVM Physical Volume | 16237 | Ready | + | |...| |...| | | | + | |...| |...| | | | + +--------------------------------------+...+------------+...+---------------------+----------+--------+ + +Add the partition to the volume group: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-pv-add controller-0 nova-local 009ce3b1-ed07-46e9-9560-9d2371676748 + +--------------------------+--------------------------------------------------+ + | Property | Value | + +--------------------------+--------------------------------------------------+ + | uuid | 830c9dc8-c71a-4cb2-83be-c4d955ef4f6b | + | pv_state | adding | + | pv_type | partition | + | disk_or_part_uuid | 009ce3b1-ed07-46e9-9560-9d2371676748 | + | disk_or_part_device_node | /dev/sdc1 | + | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0-part1 | + | lvm_pv_name | /dev/sdc1 | + | lvm_vg_name | nova-local | + | lvm_pv_uuid | None | + | lvm_pv_size | 0 | + | lvm_pe_total | 0 | + | lvm_pe_alloced | 0 | + | ihost_uuid | 9c332b27-6f22-433b-bf51-396371ac4608 | + | created_at | 2018-08-28T14:06:05.705546+00:00 | + | updated_at | None | + +--------------------------+--------------------------------------------------+ + +********************** +Unlocking controller-0 +********************** + +You must unlock controller-0 so that you can use it to install +controller-1. Use the system host-unlock command: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-unlock controller-0 + +The host is rebooted. During the reboot, the command line is +unavailable, and any ssh connections are dropped. To monitor the +progress of the reboot, use the controller-0 console. + +**************************************** +Verifying the controller-0 configuration +**************************************** + +On controller-0, acquire Keystone administrative privileges: + +:: + + controller-0:~$ source /etc/nova/openrc + +Verify that the controller-0 services are running: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system service-list + +-----+-------------------------------+--------------+----------------+ + | id | service_name | hostname | state | + +-----+-------------------------------+--------------+----------------+ + ... + | 1 | oam-ip | controller-0 | enabled-active | + | 2 | management-ip | controller-0 | enabled-active | + ... + +-----+-------------------------------+--------------+----------------+ + +Verify that controller-0 has controller and compute subfunctions: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-show 1 | grep subfunctions + | subfunctions | controller,compute | + +Verify that controller-0 is unlocked, enabled, and available: + +:: + + [wrsroot@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + +----+--------------+-------------+----------------+-------------+--------------+ + +***************** +System alarm list +***************** + +When all nodes are unlocked, enabled, and available: check 'fm alarm-list' for +issues. + +Your StarlingX deployment is now up and running with one controller with Cinder +storage and all OpenStack services up and running. You can now proceed with +standard OpenStack APIs, CLIs and/or Horizon to load Glance images, configure +Nova Flavors, configure Neutron networks and launch Nova virtual machines. + +---------------------- +Deployment terminology +---------------------- + +.. include:: deployment_terminology.rst + :start-after: incl-simplex-deployment-terminology: + :end-before: incl-simplex-deployment-terminology-end: + +.. include:: deployment_terminology.rst + :start-after: incl-standard-controller-deployment-terminology: + :end-before: incl-standard-controller-deployment-terminology-end: + +.. include:: deployment_terminology.rst + :start-after: incl-common-deployment-terminology: + :end-before: incl-common-deployment-terminology-end: