root/build-tools/build-srpms-common.sh
Scott Little b2286535c9 Update to build tools to support Build layering.
1: Building rpms now produces an rpm.lst file.  This file serves to
list all rpms produced by the build.  A seperate rpm.lst file
is producede for each build type (std, rt, installer). The file is
co-resident with the matching repodata directory.

The rpm.lst files will need to be published by cengn for each layer build.

The download tools in stx-tools will need to find the rpm.lst files
of lower layer builds, and use it to direct the download of rpms
from the lower layer.

2: Building rpms now produces an image.inc file.  This file serves
to list all rpms that the layer recommends be built into an iso.
The file is stored under the $MY_WORKSPACE/<build-type> subdirectory,
although it has identical content for any build-type.

The image.inc file will need to be published by cengn for each layer
build.

The download tools in stx-tools will need to download the per-layer
image.inc files to the $MY_REPO/cgcs-centos-repo/layer_image_inc/
sub-directory, renaming the file in some layer specific way.

The build-iso tool will process any *.inc files found under
$MY_REPO/cgcs-centos-repo/layer_image_inc/ ,
adding those rpms to the iso.

3) Add a mechanism for layer specific mock.cfg prototypes.
This will allow special handling of the real-time (aka 'rt'),
repositories for layer builds that need to do a 'rt' build.

4) Add support for a $MY_REPO/cgcs-centos-repo/rt subdirectory,
a place to build a repo of real-time rpms originating from
lower layer builds.

The download tools in stx-tools will need to populate the new
rt repos.

As of this writing, non-rt rpms remain in $MY_REPO/cgcs-centos-repo/.
i.e. there is not a new $MY_REPO/cgcs-centos-repo/std/ directory.

5) Some changes to make us more flexible about where we find
realease and bsp files.

6) Found that kernel mudules were not reliably building against
the hearnel-headers of our modified kernel.  Found that adding '--update'
to our mock build command was not working.  Does mock expect '--update'
to only be used independently of a build command?  It does work when
used in that manner, so that's what we will do.

7) The build-pkgs, build-srpms, build-rpms family of commands
can take a layer argument and/or will read the LAYER environment
variable.  Current the only use of this variable is to modify
the build-info.  It does NOT limit the compile to packages
for a specific layer.

Story: 2006166
Task: 37094

Depends-On: https://review.opendev.org/698756
Depends-On: https://review.opendev.org/700819
Change-Id: I817e08a19cdabe08b3fcc47dee63a36b461c13c0
Co-Authored-by: Martin Chen <haochuan.z.chen@intel.com>
Signed-off-by: Scott Little <scott.little@windriver.com>
2020-02-07 16:36:40 -05:00

203 lines
6.4 KiB
Bash

#
# Copyright (c) 2018 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
#
# Functions common to build-srpm-serial and build-srpm-parallel.
#
SRC_BUILD_TYPE_SRPM="srpm"
SRC_BUILD_TYPE_SPEC="spec"
SRC_BUILD_TYPES="$SRC_BUILD_TYPE_SRPM $SRC_BUILD_TYPE_SPEC"
set_build_info () {
local info_file="$MY_WORKSPACE/BUILD_INFO"
local layer_prefix="${LAYER^^}_"
if [ "${LAYER}" == "" ]; then
layer_prefix=""
fi
mkdir -p "$(dirname ${info_file})"
echo "${layer_prefix}OS=\"centos\"" > "${info_file}"
echo "${layer_prefix}JOB=\"n/a\"" >> "${info_file}"
echo "${layer_prefix}BUILD_BY=\"${USER}\"" >> "${info_file}"
echo "${layer_prefix}BUILD_NUMBER=\"n/a\"" >> "${info_file}"
echo "${layer_prefix}BUILD_HOST=\"$(hostname)\"" >> "${info_file}"
echo "${layer_prefix}BUILD_DATE=\"$(date '+%Y-%m-%d %H:%M:%S %z')\"" >> "${info_file}"
}
str_lst_contains() {
TARGET="$1"
LST="$2"
if [[ $LST =~ (^|[[:space:]])$TARGET($|[[:space:]]) ]] ; then
return 0
else
return 1
fi
}
#
# capture_md5sum_from_input_vars <src-build-type> <srpm-or-spec-path> <work-dir>
#
# Returns md5 data for all input files of a src.rpm.
# Assumes PKG_BASE, ORIG_SRPM_PATH have been defined and the
# build_srpm.data file has already been sourced.
#
# Arguments:
# src-build-type: Any single value from $SRC_BUILD_TYPES.
# e.g. 'srpm' or 'spec'
# srpm-or-spec-path: Absolute path to an src.rpm, or to a
# spec file.
# work-dir: Optional working directory. If a path is
# specified but does not exist, it will be created.
#
# Returns: output of md5sum command with canonical path names
#
md5sums_from_input_vars () {
local SRC_BUILD_TYPE="$1"
local SRPM_OR_SPEC_PATH="$2"
local WORK_DIR="$3"
local TMP_FLAG=0
local LINK_FILTER='[/]stx[/]downloads[/]'
if ! str_lst_contains "$SRC_BUILD_TYPE" "$SRC_BUILD_TYPES" ; then
>&2 echo "ERROR: $FUNCNAME (${LINENO}): invalid arg: SRC_BUILD_TYPE='$SRC_BUILD_TYPE'"
return 1
fi
if [ -z $WORK_DIR ]; then
WORK_DIR=$(mktemp -d /tmp/${FUNCNAME}_XXXXXX)
if [ $? -ne 0 ]; then
>&2 echo "ERROR: $FUNCNAME (${LINENO}): mktemp -d /tmp/${FUNCNAME}_XXXXXX"
return 1
fi
TMP_FLAG=1
else
mkdir -p "$WORK_DIR"
if [ $? -ne 0 ]; then
>&2 echo "ERROR: $FUNCNAME (${LINENO}): mkdir -p '$WORK_DIR'"
return 1
fi
fi
local INPUT_FILES="$WORK_DIR/srpm_input.files"
local INPUT_LINKS="$WORK_DIR/srpm_input.links"
local INPUT_FILES_SORTED="$WORK_DIR/srpm_sorted_input.files"
if [ -f "$INPUT_LINKS" ]; then
\rm -f "$INPUT_LINKS"
fi
# Create lists of input files (INPUT_FILES) and symlinks (INPUT_LINKS).
# First elements are absolute paths...
find "$PKG_BASE" -type f > $INPUT_FILES
if [ $? -ne 0 ]; then
>&2 echo "ERROR: $FUNCNAME (${LINENO}): find '$PKG_BASE' -type f"
return 1
fi
if [ "$SRC_BUILD_TYPE" == "$SRC_BUILD_TYPE_SRPM" ]; then
find "$SRPM_OR_SPEC_PATH" -type f >> $INPUT_FILES
if [ $? -ne 0 ]; then
>&2 echo "ERROR: $FUNCNAME (${LINENO}): find '$SRPM_OR_SPEC_PATH' -type f"
return 1
fi
fi
# ...additional elements are based on values already sourced from
# build_srpm.data (COPY_LIST, SRC_DIR, COPY_LIST_TO_TAR, OPT_DEP_LIST)
# and may be relative to $PKG_BASE
#
# Use a subshell so any directory changes have no lastin effect.
(
cd $PKG_BASE
if [ "x$COPY_LIST" != "x" ]; then
ABS_COPY_LIST=$(readlink -f $COPY_LIST)
if [ $? -ne 0 ]; then
>&2 echo "ERROR: $FUNCNAME (${LINENO}): readlink -f '$COPY_LIST' -type f"
return 1
fi
find $ABS_COPY_LIST -type f >> $INPUT_FILES
if [ $? -ne 0 ]; then
>&2 echo "ERROR: $FUNCNAME (${LINENO}): find '$ABS_COPY_LIST' -type f"
return 1
fi
# Treat most links normally
find $ABS_COPY_LIST -type l | grep -v "$LINK_FILTER" >> $INPUT_FILES
# Links in the downloads directory likely point outside of $MY_REPO
# and might not be 'portable' from a build avoidance prespective.
# We'll treat these specially.
find $ABS_COPY_LIST -type l | grep "$LINK_FILTER" >> $INPUT_LINKS
fi
if [ "$SRC_BUILD_TYPE" == "$SRC_BUILD_TYPE_SPEC" ]; then
if [ "x$SRC_DIR" != "x" ]; then
if [ -d "$SRC_DIR" ]; then
find $(readlink -f "$SRC_DIR") -type f | grep -v '[/][.]git$' | grep -v '[/][.]git[/]' >> $INPUT_FILES
if [ $? -ne 0 ]; then
>&2 echo "ERROR: $FUNCNAME (${LINENO}): find '$SRC_DIR' -type f"
return 1
fi
fi
fi
if [ "x$COPY_LIST_TO_TAR" != "x" ]; then
for x in $COPY_LIST_TO_TAR; do
find $(readlink -f $x) -type f >> $INPUT_FILES
if [ $? -ne 0 ]; then
>&2 echo "ERROR: $FUNCNAME (${LINENO}): find '$x' -type f"
return 1
fi
done
fi
fi
if [ "x$OPT_DEP_LIST" != "x" ]; then
for x in $OPT_DEP_LIST; do
find $(readlink -f $x) -type f >> $INPUT_FILES 2> /dev/null || true
done
fi
)
if [ $? -eq 1 ]; then
return 1
fi
# Create sorted, unique list of cononical paths
(
# Regular files, get canonical path
cat $INPUT_FILES | xargs readlink -f
# A Symlink that likely points outside of $MY_REPO.
# get canonical path to the symlink itself, and not
# to what the symlink points to.
if [ -f $INPUT_LINKS ]; then
while IFS= read -r f; do
echo "$(readlink -f $(dirname $f))/$(basename $f)"
done < "$INPUT_LINKS"
fi
) | sort --unique > $INPUT_FILES_SORTED
# Remove $MY_REPO prefix from paths
cat $INPUT_FILES_SORTED | xargs md5sum | sed "s# $(readlink -f $MY_REPO)/# #"
if [ $TMP_FLAG -eq 0 ]; then
\rm -f $INPUT_FILES_SORTED
\rm -f $INPUT_FILES
if [ -f $INPUT_LINKS ]; then
\rm -f $INPUT_LINKS
fi
else
\rm -rf $WORK_DIR
fi
return 0
}