starlingx/tools
Code Issues Proposed changes

Mirror failed to download 7.6 installer

Browse Source 
The starlingx mirror failed to capture the centos 7.6 installer.
Although the installer files are stored under
mirror.centos.org/<VER>/os/x86_64/, it is not captured by
reposync.  No options that I have tried will make it do so.

We can either demand that folks create .lst file entries
for any installer files they want to use, or modify
the mirror scripts with a special case to capture the
installer of all future centos versions via other means.

This update takes the later approach.  It does make
some assumptions about the names of subdirectories we
need to capture, e.g. EFI, images, isolinux, LiveOS,
but they hold true for Centos releases to date.

Change-Id: Id90b22fbb93c8143ed112be33ac168f1ee84f2f3
Signed-off-by: Scott Little <scott.little@windriver.com>
This commit is contained in:
Scott Little 2019-01-07 15:45:48 -05:00
parent 46bb9f9a3b
commit a02bca0923
1 changed files with 33 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
centos-mirror-tools/stx_mirror_scripts/daily_repo_sync.sh
View File

@ -24,6 +24,13 @@ YUM_REPOS_DIR="$YUM_CONF_DIR/yum.repos.d"
DOWNLOAD_PATH_ROOT="/export/mirror/centos"
URL_UTILS="url_utils.sh"
# These variables drive the download of the centos installer
# and other non-repo files found under the os/x86_64 subdirectory.
OS_PATH_PREFIX=/export/mirror/centos/centos
OS_PATH_SUFFIX=os/x86_64
OS_FILES="EULA GPL"
OS_DIRS="EFI LiveOS images isolinux"
DAILY_REPO_SYNC_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}" )" )"
if [ -f "$DAILY_REPO_SYNC_DIR/$URL_UTILS" ]; then
@ -118,6 +125,32 @@ for REPO in $(find $YUM_REPOS_DIR -name '*.repo'); do
continue
fi
# The following will download the centos installer and other non-repo
# files and directories found under the os/x86_64 subdirectory.
if [[ "$DOWNLOAD_PATH" == "$OS_PATH_PREFIX"/*/"$OS_PATH_SUFFIX" ]]; then
for f in $OS_FILES; do
CMD="wget '$REPO_URL/$f' --output-document='$DOWNLOAD_PATH/$f'"
echo "$CMD"
eval $CMD
if [ $? -ne 0 ]; then
echo "Error: $CMD"
ERR_COUNT=$((ERR_COUNT+1))
continue
fi
done
for d in $OS_DIRS; do
CMD="wget -r -N -l 3 -nv -np -e robots=off --reject-regex '.*[?].*' --reject index.html '$REPO_URL/$d/' -P '$OS_PATH_PREFIX/'"
echo "$CMD"
eval $CMD
if [ $? -ne 0 ]; then
echo "Error: $CMD"
ERR_COUNT=$((ERR_COUNT+1))
continue
fi
done
fi
popd
done
done | tee $LOGFILE