Build guide for Layered build
Added step by step guidelines for users to do layered build. Story: 2007223 Change-Id: I25326cab06dd1fab743a946bfd92c9240f93bb25 Signed-off-by: Poornima <poornima.y.n@intel.com>
This commit is contained in:
Poornima
Mary Camp
parent
8bfe26449b
commit
79a84abac3
@ -1,6 +1,8 @@
|
|||||||
=============
|
.. _Layered-Build:
|
||||||
Layered Build
|
|
||||||
=============
|
=======================
|
||||||
|
Layered Build Reference
|
||||||
|
=======================
|
||||||
|
|
||||||
What is build layering?
|
What is build layering?
|
||||||
-----------------------
|
-----------------------
|
||||||
|
|||||||
@ -9,6 +9,8 @@ R3.0 and earlier release.
|
|||||||
:local:
|
:local:
|
||||||
:depth: 1
|
:depth: 1
|
||||||
|
|
||||||
|
.. _Requirements:
|
||||||
|
|
||||||
------------
|
------------
|
||||||
Requirements
|
Requirements
|
||||||
------------
|
------------
|
||||||
@ -37,6 +39,8 @@ A workstation computer with:
|
|||||||
http://lists.starlingx.io/pipermail/starlingx-discuss/2018-July/000136.html for more details)
|
http://lists.starlingx.io/pipermail/starlingx-discuss/2018-July/000136.html for more details)
|
||||||
* Public SSH key
|
* Public SSH key
|
||||||
|
|
||||||
|
.. _Development-environment-setup:
|
||||||
|
|
||||||
-----------------------------
|
-----------------------------
|
||||||
Development environment setup
|
Development environment setup
|
||||||
-----------------------------
|
-----------------------------
|
||||||
@ -139,6 +143,8 @@ Install tools project
|
|||||||
|
|
||||||
cd $HOME/tools/
|
cd $HOME/tools/
|
||||||
|
|
||||||
|
.. _Prepare_the_base_Docker_image:
|
||||||
|
|
||||||
-----------------------------
|
-----------------------------
|
||||||
Prepare the base Docker image
|
Prepare the base Docker image
|
||||||
-----------------------------
|
-----------------------------
|
||||||
@ -208,6 +214,8 @@ The creation of the StarlingX ISO relies on a repository of RPM binaries,
|
|||||||
RPM sources, and tar compressed files. This section describes how to build
|
RPM sources, and tar compressed files. This section describes how to build
|
||||||
this CentOS mirror repository.
|
this CentOS mirror repository.
|
||||||
|
|
||||||
|
.. _Run_building_Docker_container:
|
||||||
|
|
||||||
*****************************
|
*****************************
|
||||||
Run building Docker container
|
Run building Docker container
|
||||||
*****************************
|
*****************************
|
||||||
@ -458,6 +466,8 @@ Build the image:
|
|||||||
|
|
||||||
build-iso
|
build-iso
|
||||||
|
|
||||||
|
.. _Build-installer:
|
||||||
|
|
||||||
---------------
|
---------------
|
||||||
Build installer
|
Build installer
|
||||||
---------------
|
---------------
|
||||||
|
|||||||
@ -9,6 +9,7 @@ Developer Resources
|
|||||||
:maxdepth: 1
|
:maxdepth: 1
|
||||||
|
|
||||||
build_guide
|
build_guide
|
||||||
|
layered_build_guide
|
||||||
Layered_Build
|
Layered_Build
|
||||||
code-submission-guide
|
code-submission-guide
|
||||||
navigate_source_code
|
navigate_source_code
|
||||||
|
|||||||
859
doc/source/developer_resources/layered_build_guide.rst
Normal file
859
doc/source/developer_resources/layered_build_guide.rst
Normal file
@ -0,0 +1,859 @@
|
|||||||
|
===================
|
||||||
|
Layered Build Guide
|
||||||
|
===================
|
||||||
|
This section describes the steps for layered build and building an ISO image
|
||||||
|
from a StarlingX R4.0 release onwards.
|
||||||
|
|
||||||
|
For more detailed explanation of layered build, please refer :ref:`Layered-Build`.
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:local:
|
||||||
|
:depth: 1
|
||||||
|
|
||||||
|
-------------------
|
||||||
|
:ref:`Requirements`
|
||||||
|
-------------------
|
||||||
|
Layered build has the same requirements as StarlingX R3.0 build.
|
||||||
|
Click the heading above for details.
|
||||||
|
|
||||||
|
------------------------------------
|
||||||
|
:ref:`Development-environment-setup`
|
||||||
|
------------------------------------
|
||||||
|
Layered build uses the same development environment setup as StarlingX R3.0 build.
|
||||||
|
Click the heading above for details.
|
||||||
|
|
||||||
|
--------------------
|
||||||
|
Build compiler layer
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
*****************************
|
||||||
|
Prepare the base Docker image
|
||||||
|
*****************************
|
||||||
|
|
||||||
|
Follow :ref:`Prepare_the_base_Docker_image` for creating the base StarlingX
|
||||||
|
image with changes only in localrc file as below.
|
||||||
|
Since, we are building compiler layer, create and modify only localrc
|
||||||
|
file. Make sure to set the project and layer as compiler and user name as
|
||||||
|
desired. For example:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
# tbuilder localrc
|
||||||
|
MYUNAME=<your user name>
|
||||||
|
PROJECT=compiler
|
||||||
|
HOST_PREFIX=$HOME/starlingx/workspace
|
||||||
|
HOST_MIRROR_DIR=$HOME/starlingx/mirror
|
||||||
|
LAYER=compiler
|
||||||
|
|
||||||
|
.. _Build-the-CentOS-mirror-repository-compiler:
|
||||||
|
|
||||||
|
**********************************
|
||||||
|
Build the CentOS mirror repository
|
||||||
|
**********************************
|
||||||
|
|
||||||
|
The creation of the StarlingX ISO relies on a repository of RPM binaries,
|
||||||
|
RPM sources, and tar compressed files. This section describes how to build
|
||||||
|
this CentOS mirror repository.
|
||||||
|
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
Run building Docker container
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Follow the :ref:`building docker container <Run_building_Docker_container>` for
|
||||||
|
more details.
|
||||||
|
At the end of this procedure, you will be inside the Starlingx container.
|
||||||
|
|
||||||
|
Quick command summary is as below:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/tools/
|
||||||
|
bash tb.sh env
|
||||||
|
bash tb.sh run
|
||||||
|
bash tb.sh exec
|
||||||
|
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
Download source code repositories
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
#. Inside the building Docker container, start the internal environment:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
eval $(ssh-agent)
|
||||||
|
ssh-add
|
||||||
|
|
||||||
|
|
||||||
|
#. Use the repo tool to create a local clone of the manifest git repository
|
||||||
|
based on the “master” branch for compiler layer.
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $MY_REPO_ROOT_DIR
|
||||||
|
repo init -u https://opendev.org/starlingx/manifest -b master -m compiler.xml
|
||||||
|
|
||||||
|
#. Synchronize code for compiler layer using command below:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
repo sync -j`nproc`
|
||||||
|
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
Download packages
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
#. Inside the Docker container, enter the following commands to download
|
||||||
|
the required packages to populate the CentOS mirror repository:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $MY_REPO_ROOT_DIR/stx-tools/centos-mirror-tools && bash download_mirror.sh
|
||||||
|
|
||||||
|
#. Monitor the download of packages until it is complete. When the download
|
||||||
|
is complete successfully with no missing packages, the following message appears:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
step #5: done successfully
|
||||||
|
sudo rm -rf /tmp/stx_mirror_4d2URF
|
||||||
|
IMPORTANT: The following 3 files are just bootstrap versions. Based on them, the workable images
|
||||||
|
for StarlingX could be generated by running "update-pxe-network-installer" command after "build-iso"
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz
|
||||||
|
|
||||||
|
Success
|
||||||
|
|
||||||
|
***************
|
||||||
|
Verify packages
|
||||||
|
***************
|
||||||
|
|
||||||
|
#. Verify no missing or failed packages exist:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cat logs/*_missing_*.log
|
||||||
|
cat logs/*_failmoved_*.log
|
||||||
|
|
||||||
|
#. In case missing or failed packages do exist, which is usually caused by
|
||||||
|
network instability (or timeout), you need to download the packages
|
||||||
|
manually or re-run the download command given in previous step.
|
||||||
|
Doing so assures you get all RPMs listed in *centos_srpms_3rdparties.lst,
|
||||||
|
centos_srpms_centos.lst, centos_srpms_centos3rdparties.lst*.
|
||||||
|
|
||||||
|
******************
|
||||||
|
Packages structure
|
||||||
|
******************
|
||||||
|
|
||||||
|
The following is a general overview of the packages structure resulting from
|
||||||
|
downloading the packages for compiler layer:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
/localdisk/designer/<user>/<project>/stx-tools/centos-mirror-tools/output
|
||||||
|
.
|
||||||
|
└── stx-r1
|
||||||
|
└── CentOS
|
||||||
|
└── pike
|
||||||
|
├── Binary
|
||||||
|
│ ├── noarch
|
||||||
|
│ └── x86_64
|
||||||
|
├── downloads
|
||||||
|
│ └── puppet
|
||||||
|
├── layer_build_info
|
||||||
|
├── layer_image_inc
|
||||||
|
├── layer_pkg_lists
|
||||||
|
├── layer_wheels_inc
|
||||||
|
└── Source
|
||||||
|
|
||||||
|
*******************************
|
||||||
|
Copy CentOS mirror repository
|
||||||
|
*******************************
|
||||||
|
Exit from the building Docker container. Run the following commands:
|
||||||
|
|
||||||
|
#. Navigate to CentOS mirror directory *mirror/CentOS* under your *starlingx*
|
||||||
|
workspace directory:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/starlingx/mirror/CentOS/
|
||||||
|
|
||||||
|
#. Copy the built CentOS mirror repository *$HOME/starlingx/mirror/* workspace
|
||||||
|
directory. Make sure to insert the <user>:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cp -r $HOME/starlingx/workspace/localdisk/designer/<user>/compiler/stx-tools/centos-mirror-tools/output/stx-r1 .
|
||||||
|
|
||||||
|
***************************
|
||||||
|
Create a tarball repository
|
||||||
|
***************************
|
||||||
|
|
||||||
|
#. Enter the StarlingX container using below command:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/tools/
|
||||||
|
./tb.sh exec
|
||||||
|
|
||||||
|
#. Copy downloaded CentOS tarballs into StarlingX repo
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
|
||||||
|
|
||||||
|
Alternatively, you can run the "populate_downloads.sh" script to copy the
|
||||||
|
tarballs instead of using a symlink:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
bash populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
|
||||||
|
|
||||||
|
**************
|
||||||
|
Build packages
|
||||||
|
**************
|
||||||
|
|
||||||
|
#. **Temporal!** Build-Pkgs Errors. Be prepared to have some missing /
|
||||||
|
corrupted rpm and tarball packages generated during
|
||||||
|
:ref:`Build-the-CentOS-mirror-repository-compiler`, which will cause the next step
|
||||||
|
to fail. If that step does fail, manually download those missing /
|
||||||
|
corrupted packages.
|
||||||
|
|
||||||
|
#. Update the symbolic links:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
bash generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
|
||||||
|
|
||||||
|
#. Build the packages:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
build-pkgs
|
||||||
|
build-pkgs --installer
|
||||||
|
|
||||||
|
------------------
|
||||||
|
Build distro layer
|
||||||
|
------------------
|
||||||
|
|
||||||
|
.. _Build-the-CentOS-mirror-repository-distro:
|
||||||
|
|
||||||
|
**********************************
|
||||||
|
Build the CentOS mirror repository
|
||||||
|
**********************************
|
||||||
|
|
||||||
|
The creation of the StarlingX ISO relies on a repository of RPM binaries,
|
||||||
|
RPM sources, and tar compressed files. This section describes how to build
|
||||||
|
this CentOS mirror repository.
|
||||||
|
|
||||||
|
If you were building compiler layer, exit from container and enter the tools
|
||||||
|
directory using below commands:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/tools/
|
||||||
|
|
||||||
|
Since, we are building distro layer, modify localrc file. Make sure to set the
|
||||||
|
project and layer as distro. For example:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
# tbuilder localrc
|
||||||
|
MYUNAME=<your user name>
|
||||||
|
PROJECT=distro
|
||||||
|
HOST_PREFIX=$HOME/starlingx/workspace
|
||||||
|
HOST_MIRROR_DIR=$HOME/starlingx/mirror
|
||||||
|
LAYER=distro
|
||||||
|
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
Run building Docker container
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
Follow the :ref:`building docker container <Run_building_Docker_container>` for
|
||||||
|
more details.
|
||||||
|
At the end of this procedure, you will be inside the Starlingx container.
|
||||||
|
|
||||||
|
Quick command summary is as below:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/tools/
|
||||||
|
bash tb.sh env
|
||||||
|
bash tb.sh run #Need not give, If the container was created earlier.
|
||||||
|
bash tb.sh exec
|
||||||
|
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
Download source code repositories
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
#. Inside the building Docker container, start the internal environment:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
eval $(ssh-agent)
|
||||||
|
ssh-add
|
||||||
|
|
||||||
|
|
||||||
|
#. Use the repo tool to create a local clone of the manifest git repository
|
||||||
|
based on the “master” branch for compiler layer.
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $MY_REPO_ROOT_DIR
|
||||||
|
repo init -u https://opendev.org/starlingx/manifest -b master -m distro.xml
|
||||||
|
|
||||||
|
#. Synchronize code for compiler layer using command below:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
repo sync -j`nproc`
|
||||||
|
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
Download packages
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
#. Inside the Docker container, enter the following commands to download
|
||||||
|
the required packages to populate the CentOS mirror repository:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $MY_REPO_ROOT_DIR/stx-tools/centos-mirror-tools && bash download_mirror.sh
|
||||||
|
|
||||||
|
#. Monitor the download of packages until it is complete. When the download
|
||||||
|
is complete, the following message appears:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
step #5: done successfully
|
||||||
|
sudo rm -rf /tmp/stx_mirror_UIQ675
|
||||||
|
IMPORTANT: The following 3 files are just bootstrap versions. Based on them, the workable images
|
||||||
|
for StarlingX could be generated by running "update-pxe-network-installer" command after "build-iso"
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz
|
||||||
|
|
||||||
|
Success
|
||||||
|
|
||||||
|
***************
|
||||||
|
Verify packages
|
||||||
|
***************
|
||||||
|
|
||||||
|
#. Verify no missing or failed packages exist:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cat logs/*_missing_*.log
|
||||||
|
cat logs/*_failmoved_*.log
|
||||||
|
|
||||||
|
#. In case missing or failed packages do exist, which is usually caused by
|
||||||
|
network instability (or timeout), you need to download the packages
|
||||||
|
manually.
|
||||||
|
Doing so assures you get all RPMs listed in *centos_srpms_3rdparties.lst,
|
||||||
|
centos_srpms_centos.lst, centos_srpms_centos3rdparties.lst*.
|
||||||
|
|
||||||
|
******************
|
||||||
|
Packages structure
|
||||||
|
******************
|
||||||
|
|
||||||
|
The following is a general overview of the packages structure resulting from
|
||||||
|
downloading the packages for distro layer:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
/localdisk/designer/<user>/distro/stx-tools/centos-mirror-tools/output
|
||||||
|
.
|
||||||
|
└── stx-r1
|
||||||
|
└── CentOS
|
||||||
|
└── pike
|
||||||
|
├── Binary
|
||||||
|
│ ├── EFI
|
||||||
|
│ │ └── BOOT
|
||||||
|
│ │ └── fonts
|
||||||
|
│ ├── images
|
||||||
|
│ │ └── pxeboot
|
||||||
|
│ ├── isolinux
|
||||||
|
│ ├── LiveOS
|
||||||
|
│ ├── noarch
|
||||||
|
│ └── x86_64
|
||||||
|
├── downloads
|
||||||
|
│ └── puppet
|
||||||
|
│ └── packstack
|
||||||
|
│ └── puppet
|
||||||
|
│ └── modules
|
||||||
|
├── layer_build_info
|
||||||
|
├── layer_image_inc
|
||||||
|
├── layer_pkg_lists
|
||||||
|
├── layer_repos
|
||||||
|
│ └── compiler
|
||||||
|
│ └── std
|
||||||
|
│ ├── repodata
|
||||||
|
│ └── repodata.upstream
|
||||||
|
├── layer_wheels_inc
|
||||||
|
└── Source
|
||||||
|
|
||||||
|
*******************************
|
||||||
|
Copy CentOS mirror repository
|
||||||
|
*******************************
|
||||||
|
Exit from the building Docker container. Run the following commands:
|
||||||
|
|
||||||
|
#. Navigate to CentOS mirror directory *mirror/CentOS* under your *starlingx*
|
||||||
|
workspace directory:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/starlingx/mirror/CentOS/
|
||||||
|
|
||||||
|
#. Copy the built CentOS mirror repository *$HOME/starlingx/mirror/* workspace
|
||||||
|
directory. Make sure to insert the <user>:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cp -r $HOME/starlingx/workspace/localdisk/designer/<user>/distro/stx-tools/centos-mirror-tools/output/stx-r1 .
|
||||||
|
|
||||||
|
***************************
|
||||||
|
Create a tarball repository
|
||||||
|
***************************
|
||||||
|
|
||||||
|
#. Enter the StarlingX container using below command:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/tools/
|
||||||
|
./tb.sh exec
|
||||||
|
|
||||||
|
#. Copy downloaded CentOS tarballs into StarlingX repo
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
|
||||||
|
|
||||||
|
Alternatively, you can run the "populate_downloads.sh" script to copy the
|
||||||
|
tarballs instead of using a symlink:
|
||||||
|
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
|
||||||
|
|
||||||
|
|
||||||
|
**************
|
||||||
|
Build packages
|
||||||
|
**************
|
||||||
|
|
||||||
|
#. **Temporal!** Build-Pkgs Errors. Be prepared to have some missing /
|
||||||
|
corrupted rpm and tarball packages generated during
|
||||||
|
:ref:`Build-the-CentOS-mirror-repository-distro`, which will cause the next step
|
||||||
|
to fail. If that step does fail, manually download those missing /
|
||||||
|
corrupted packages.
|
||||||
|
|
||||||
|
#. Update the symbolic links:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
bash generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
|
||||||
|
|
||||||
|
This step creates the repo directory, following is the output on the console
|
||||||
|
for a successful repo directory creation for the <user> stx:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
Copying comps.xml file.
|
||||||
|
Createing yum repodata.
|
||||||
|
Directory walk started
|
||||||
|
Directory walk done - 51 packages
|
||||||
|
Temporary output repo path: /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Source/.repodata/
|
||||||
|
Preparing sqlite DBs
|
||||||
|
Pool started (with 5 workers)
|
||||||
|
Pool finished
|
||||||
|
Directory walk started
|
||||||
|
Directory walk done - 0 packages
|
||||||
|
Temporary output repo path: /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/rt/Source/.repodata/
|
||||||
|
Preparing sqlite DBs
|
||||||
|
Pool started (with 5 workers)
|
||||||
|
Pool finished
|
||||||
|
Directory walk started
|
||||||
|
Directory walk done - 1450 packages
|
||||||
|
Temporary output repo path: /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/.repodata/
|
||||||
|
Preparing sqlite DBs
|
||||||
|
Pool started (with 5 workers)
|
||||||
|
Pool finished
|
||||||
|
Directory walk started
|
||||||
|
Directory walk done - 0 packages
|
||||||
|
Temporary output repo path: /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/rt/Binary/.repodata/
|
||||||
|
Preparing sqlite DBs
|
||||||
|
Pool started (with 5 workers)
|
||||||
|
Pool finished
|
||||||
|
Copying mock.cfg.proto file.
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/EFI/BOOT/BOOTX64.EFI
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/EFI/BOOT/fonts/unicode.pf2
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/EFI/BOOT/grub.cfg
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/EFI/BOOT/grubx64.efi
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/images/efiboot.img
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/images/pxeboot/initrd.img
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/images/pxeboot/vmlinuz
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/isolinux/boot.msg
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/isolinux/grub.conf
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/isolinux/initrd.img
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/isolinux/isolinux.bin
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/isolinux/isolinux.cfg
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/isolinux/memtest
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/isolinux/splash.png
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/isolinux/vesamenu.c32
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/isolinux/vmlinuz
|
||||||
|
Creating symlink for /localdisk/designer/stx/distro/cgcs-root/cgcs-centos-repo/Binary/LiveOS/squashfs.img
|
||||||
|
Creating folder EFI
|
||||||
|
Creating folder EFI/BOOT
|
||||||
|
Creating folder EFI/BOOT/fonts
|
||||||
|
Creating folder images
|
||||||
|
Creating folder images/pxeboot
|
||||||
|
Creating folder isolinux
|
||||||
|
Creating folder LiveOS
|
||||||
|
Done creating repo directory
|
||||||
|
|
||||||
|
#. Build the packages:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
build-pkgs
|
||||||
|
build-pkgs --installer
|
||||||
|
|
||||||
|
|
||||||
|
-----------------
|
||||||
|
Build flock layer
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
.. _Build-the-CentOS-mirror-repository-flock:
|
||||||
|
|
||||||
|
**********************************
|
||||||
|
Build the CentOS mirror repository
|
||||||
|
**********************************
|
||||||
|
|
||||||
|
The creation of the StarlingX ISO relies on a repository of RPM binaries,
|
||||||
|
RPM sources, and tar compressed files. This section describes how to build
|
||||||
|
this CentOS mirror repository.
|
||||||
|
|
||||||
|
If you were building distro layer, exit from container and enter the tools
|
||||||
|
directory using below commands:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/tools/
|
||||||
|
|
||||||
|
Since, we are building flock layer, modify localrc file. Make sure to set the
|
||||||
|
project and layer as flock. For example:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
# tbuilder localrc
|
||||||
|
MYUNAME=<your user name>
|
||||||
|
PROJECT=flock
|
||||||
|
HOST_PREFIX=$HOME/starlingx/workspace
|
||||||
|
HOST_MIRROR_DIR=$HOME/starlingx/mirror
|
||||||
|
LAYER=flock
|
||||||
|
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
Run building Docker container
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
Follow the :ref:`building docker container <Run_building_Docker_container>` for
|
||||||
|
more details.
|
||||||
|
At the end of this procedure, you will be inside the Starlingx container.
|
||||||
|
|
||||||
|
Quick command summary is as below:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/tools/
|
||||||
|
bash tb.sh env
|
||||||
|
bash tb.sh run #Need not give, If the container was created earlier.
|
||||||
|
bash tb.sh exec
|
||||||
|
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
Download source code repositories
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
|
||||||
|
#. Inside the building Docker container, start the internal environment:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
eval $(ssh-agent)
|
||||||
|
ssh-add
|
||||||
|
|
||||||
|
|
||||||
|
#. Use the repo tool to create a local clone of the manifest git repository
|
||||||
|
based on the “master” branch for flock layer.
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $MY_REPO_ROOT_DIR
|
||||||
|
repo init -u https://opendev.org/starlingx/manifest -b master -m flock.xml
|
||||||
|
|
||||||
|
#. Synchronize code for compiler layer using command below:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
repo sync -j`nproc`
|
||||||
|
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
Download packages
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
#. Inside the Docker container, enter the following commands to download
|
||||||
|
the required packages to populate the CentOS mirror repository:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $MY_REPO_ROOT_DIR/stx-tools/centos-mirror-tools && bash download_mirror.sh
|
||||||
|
|
||||||
|
#. Monitor the download of packages until it is complete. When the download
|
||||||
|
is complete, the following message appears:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
step #5: done successfully
|
||||||
|
sudo rm -rf /tmp/stx_mirror_ievdiA
|
||||||
|
IMPORTANT: The following 3 files are just bootstrap versions. Based
|
||||||
|
on them, the workable images for StarlingX could be generated by
|
||||||
|
running "update-pxe-network-installer" command after "build-iso"
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz
|
||||||
|
|
||||||
|
***************
|
||||||
|
Verify packages
|
||||||
|
***************
|
||||||
|
|
||||||
|
When the download is not successful, the following message appears:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
IMPORTANT: The following 3 files are just bootstrap versions. Based
|
||||||
|
on them, the workable images for StarlingX could be generated by
|
||||||
|
running "update-pxe-network-installer" command after "build-iso"
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img
|
||||||
|
- ./output/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz
|
||||||
|
|
||||||
|
Warning: Not all download steps succeeded. You are likely missing files.
|
||||||
|
|
||||||
|
|
||||||
|
You can verify and list missing or failed packages using below command:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cat logs/*_missing_*.log
|
||||||
|
cat logs/*_failmoved_*.log
|
||||||
|
|
||||||
|
In case missing or failed packages do exist, which is usually caused by
|
||||||
|
network instability (or timeout), you need to download the packages
|
||||||
|
manually.
|
||||||
|
Doing so assures you get all RPMs listed in *centos_srpms_3rdparties.lst,
|
||||||
|
centos_srpms_centos.lst, centos_srpms_centos3rdparties.lst*.
|
||||||
|
|
||||||
|
******************
|
||||||
|
Packages structure
|
||||||
|
******************
|
||||||
|
|
||||||
|
The following is a general overview of the packages structure resulting from
|
||||||
|
downloading the packages for flock layer:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
/localdisk/designer/<user>/flock/stx-tools/centos-mirror-tools/output
|
||||||
|
|
||||||
|
└── stx-r1
|
||||||
|
└── CentOS
|
||||||
|
└── pike
|
||||||
|
├── Binary
|
||||||
|
│ ├── EFI
|
||||||
|
│ │ └── BOOT
|
||||||
|
│ │ └── fonts
|
||||||
|
│ ├── images
|
||||||
|
│ │ └── pxeboot
|
||||||
|
│ ├── isolinux
|
||||||
|
│ ├── LiveOS
|
||||||
|
│ ├── noarch
|
||||||
|
│ └── x86_64
|
||||||
|
├── downloads
|
||||||
|
│ └── puppet
|
||||||
|
├── layer_build_info
|
||||||
|
├── layer_image_inc
|
||||||
|
├── layer_pkg_lists
|
||||||
|
├── layer_repos
|
||||||
|
│ ├── compiler
|
||||||
|
│ │ └── std
|
||||||
|
│ │ ├── repodata
|
||||||
|
│ │ └── repodata.upstream
|
||||||
|
│ └── distro
|
||||||
|
│ ├── installer
|
||||||
|
│ │ ├── repodata
|
||||||
|
│ │ └── repodata.upstream
|
||||||
|
│ ├── rt
|
||||||
|
│ │ ├── repodata
|
||||||
|
│ │ └── repodata.upstream
|
||||||
|
│ └── std
|
||||||
|
│ ├── repodata
|
||||||
|
│ └── repodata.upstream
|
||||||
|
├── layer_wheels_inc
|
||||||
|
└── Source
|
||||||
|
|
||||||
|
|
||||||
|
*******************************
|
||||||
|
Copy CentOS mirror repository
|
||||||
|
*******************************
|
||||||
|
Exit from the building Docker container. Run the following commands:
|
||||||
|
|
||||||
|
#. Navigate to CentOS mirror directory *mirror/CentOS* under your *starlingx*
|
||||||
|
workspace directory:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/starlingx/mirror/CentOS/
|
||||||
|
|
||||||
|
#. Copy the built CentOS mirror repository *$HOME/starlingx/mirror/*
|
||||||
|
workspace directory:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cp -r $HOME/starlingx/workspace/localdisk/designer/<user>/flock/stx-tools/centos-mirror-tools/output/stx-r1 .
|
||||||
|
|
||||||
|
***************************
|
||||||
|
Create a tarball repository
|
||||||
|
***************************
|
||||||
|
|
||||||
|
#. Enter the StarlingX container using below command:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/tools/
|
||||||
|
./tb.sh exec
|
||||||
|
|
||||||
|
#. Copy downloaded CentOS tarballs into StarlingX repo
|
||||||
|
::
|
||||||
|
|
||||||
|
ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
|
||||||
|
|
||||||
|
Alternatively, you can run the "populate_downloads.sh" script to copy the
|
||||||
|
tarballs instead of using a symlink:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
|
||||||
|
|
||||||
|
#. Exit from the container. On the host machine, create mirror binaries:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
mkdir -p $HOME/starlingx/mirror/CentOS/stx-installer
|
||||||
|
cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img $HOME/starlingx/mirror/CentOS/stx-installer/initrd.img
|
||||||
|
cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz $HOME/starlingx/mirror/CentOS/stx-installer/vmlinuz
|
||||||
|
cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img $HOME/starlingx/mirror/CentOS/stx-installer/squashfs.img
|
||||||
|
|
||||||
|
**************
|
||||||
|
Build packages
|
||||||
|
**************
|
||||||
|
|
||||||
|
#. Enter the StarlingX container using below command:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
cd $HOME/tools/
|
||||||
|
./tb.sh exec
|
||||||
|
|
||||||
|
#. **Temporal!** Build-Pkgs Errors. Be prepared to have some missing /
|
||||||
|
corrupted rpm and tarball packages generated during
|
||||||
|
:ref:`Build-the-CentOS-mirror-repository-flock`, which will cause the next step
|
||||||
|
to fail. If that step does fail, manually download those missing /
|
||||||
|
corrupted packages.
|
||||||
|
|
||||||
|
#. Update the symbolic links:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
bash generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
|
||||||
|
|
||||||
|
Following is the output:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
Copying comps.xml file.
|
||||||
|
Createing yum repodata.
|
||||||
|
Directory walk started
|
||||||
|
Directory walk done - 1 packages
|
||||||
|
Temporary output repo path: /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Source/.repodata/
|
||||||
|
Preparing sqlite DBs
|
||||||
|
Pool started (with 5 workers)
|
||||||
|
Pool finished
|
||||||
|
Directory walk started
|
||||||
|
Directory walk done - 0 packages
|
||||||
|
Temporary output repo path: /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/rt/Source/.repodata/
|
||||||
|
Preparing sqlite DBs
|
||||||
|
Pool started (with 5 workers)
|
||||||
|
Pool finished
|
||||||
|
Directory walk started
|
||||||
|
Directory walk done - 1892 packages
|
||||||
|
Temporary output repo path: /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/.repodata/
|
||||||
|
Preparing sqlite DBs
|
||||||
|
Pool started (with 5 workers)
|
||||||
|
Pool finished
|
||||||
|
Directory walk started
|
||||||
|
Directory walk done - 40 packages
|
||||||
|
Temporary output repo path: /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/rt/Binary/.repodata/
|
||||||
|
Preparing sqlite DBs
|
||||||
|
Pool started (with 5 workers)
|
||||||
|
Pool finished
|
||||||
|
Copying mock.cfg.proto file.
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/EFI/BOOT/BOOTX64.EFI
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/EFI/BOOT/fonts/unicode.pf2
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/EFI/BOOT/grub.cfg
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/EFI/BOOT/grubx64.efi
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/images/efiboot.img
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/images/pxeboot/initrd.img
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/images/pxeboot/vmlinuz
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/isolinux/boot.msg
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/isolinux/grub.conf
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/isolinux/initrd.img
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/isolinux/isolinux.bin
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/isolinux/isolinux.cfg
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/isolinux/memtest
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/isolinux/splash.png
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/isolinux/vesamenu.c32
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/isolinux/vmlinuz
|
||||||
|
Creating symlink for /localdisk/designer/stx/flock/cgcs-root/cgcs-centos-repo/Binary/LiveOS/squashfs.img
|
||||||
|
Creating folder EFI
|
||||||
|
Creating folder EFI/BOOT
|
||||||
|
Creating folder EFI/BOOT/fonts
|
||||||
|
Creating folder images
|
||||||
|
Creating folder images/pxeboot
|
||||||
|
Creating folder isolinux
|
||||||
|
Creating folder LiveOS
|
||||||
|
Done creating repo directory
|
||||||
|
|
||||||
|
|
||||||
|
#. Build the packages:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
build-pkgs
|
||||||
|
|
||||||
|
-------------------
|
||||||
|
Build StarlingX ISO
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
Build the image:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
build-iso
|
||||||
|
|
||||||
|
----------------------
|
||||||
|
:ref:`Build-installer`
|
||||||
|
----------------------
|
||||||
|
Layered build has the same procedure for build installer as StarlingX R3.0 build
|
||||||
|
except for the changes in path of files as below. Click the heading above for details.
|
||||||
|
|
||||||
|
#. The steps covered by the script **update-pxe-network-installer** is detailed in
|
||||||
|
$MY_REPO/stx/stx-metal/installer/initrd/README. This script creates three files on
|
||||||
|
/localdisk/loadbuild/stx/flock/pxe-network-installer/output.
|
||||||
|
|
||||||
|
#. The path for **build_srpm.data** is $MY_REPO/stx/metal/installer/pxe-network-installer/centos/.
|
||||||
|
|
||||||
Loading…
x
Reference in New Issue
Block a user