Several updates because the world is a dark place

A few things have changed and we need to fix them in one go.

Use mirror for installing docker for buildset-registry

While, we need to make this more systemic, that's hanging off of the
mirror rework. For now, since we know all of these jobs are debian
based, just set the mirror location.

Replace use of zuul cloner with git clones

You can never be a prophet in your own hometown. This is now broken
because of the git cache rework, so just replace it.

Update libjemalloc library

python:slim is based on buster now, which has libjemalloc2 not
libjemalloc1.

Remove gerrit repo remote for submodules

A recent change to the base jobs to use prepare-workspace-git
broke the gerrit image builds by actually having the origin
remote by /dev/null as intended. This breaks submodules because
for a few of them where we don't have matching stable branches
the submodule relative path behavior is actually exactly what
we want.

Since we don't care about the remote otherwise, remove the
origin remote before doing the submodule update --init so that
the submodule will clone the refs from the zuul prepared repo.

Change-Id: Ieb5b6bc8711fe971ed3445c7c267306ac4616464

.zuul.yaml

@@ -26,6 +26,8 @@
     parent: opendev-build-docker-image
     dependencies: opendev-buildset-registry
     abstract: true
+    vars:
+      docker_mirror_base_url: "http://{{ zuul_site_mirror_fqdn }}/deb-docker"
 
 - job:
     name: system-config-upload-image
@@ -961,7 +963,9 @@
       - publish-tox-docs-infra
     check:
       jobs:
-        - opendev-buildset-registry
+        - opendev-buildset-registry:
+            vars:
+              docker_mirror_base_url: "http://{{ zuul_site_mirror_fqdn }}/deb-docker"
         - tox-linters
         - puppet-beaker-rspec-puppet-4-infra-system-config
         - system-config-run-base
@@ -1015,7 +1019,9 @@
         - system-config-build-image-python-builder
     gate:
       jobs:
-        - opendev-buildset-registry
+        - opendev-buildset-registry:
+            vars:
+              docker_mirror_base_url: "http://{{ zuul_site_mirror_fqdn }}/deb-docker"
         - tox-linters
         - puppet-beaker-rspec-puppet-4-infra-system-config
         - system-config-run-base

docker/python-base/Dockerfile

@@ -16,8 +16,8 @@
 FROM python:slim
 
 RUN apt-get update \
-  && apt-get install -y dumb-init libjemalloc1 \
+  && apt-get install -y dumb-init libjemalloc2 \
   && apt-get clean \
   && rm -rf /var/lib/apt/lists/*
-ENV LD_PRELOAD /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
+ENV LD_PRELOAD /usr/lib/x86_64-linux-gnu/libjemalloc.so.2
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]

playbooks/zuul/gerrit/repos.yaml

@@ -16,9 +16,14 @@
 
     # There is no stable branch for download-commands and no 3.0 branch of
     # singleusergroup so we need to check out the actual ref defined in the
-    # parent repo. Luckily, since we don't set a remote and this submodule
+    # parent repo. The default zuul remote, file:///dev/null, doesn't work
-    # uses relative paths, it should just clone the right thing and do it
+    # here because relative paths cause it to be
-    # locally.
+    # file:///dev/plugins/download-commands, which isn't a thing. Removing
+    # the origin causes git to use relative local filesystem paths.
+    - name: Remove origin remote
+      command: "git remote rm origin"
+      args:
+        chdir: /home/zuul/src/gerrit.googlesource.com/gerrit
     - name: Checkout core plugin submodules
       command: "git submodule update --init plugins/{{ item }}"
       args:

tools/install_modules_acceptance.sh

@@ -23,7 +23,7 @@
 # library modules.
 
 # This script uses system-config/modules.env as the source of truth for modules
-# to install. It detects the presence of zuul-cloner to decide if we are running
+# to install. It detects the presence of /home/zuul to decide if we are running
 # in a zuul environment or not.
 
 ROOT=$(readlink -fn $(dirname $0)/..)
@@ -53,32 +53,16 @@ install_openstack() {
         modulepath='/etc/puppetlabs/code/modules'
     fi
 
-    cat > clonemap.yaml <<EOF
+    sudo -E git clone /home/zuul/src/opendev.org/openstack/project-config /etc/project-config
-clonemap:
-  - name: openstack/project-config
-    dest: /etc/project-config
-  - name: '(.*?)/puppet-(.*)'
-    dest: '$modulepath/\2'
-EOF
 
     project_names=""
     source ${ROOT}/modules.env
     for MOD in ${!INTEGRATION_MODULES[*]}; do
-        project_scope=$(basename `dirname $MOD`)
+        project_scope=$(basename $(dirname $MOD))
-        repo_name=`basename $MOD`
+        repo_name=$(basename $MOD)
-        project_names+=" $project_scope/$repo_name"
+        short_name=$(echo $repo_name | cut -f2- -d-)
+        sudo -E git clone /home/zuul/src/opendev.org/$project_scope/$repo_name $modulepath/$short_name
     done
-
-    sudo -E /usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git \
-        https://git.openstack.org \
-        openstack/project-config \
-        $project_names
-
-    # Zuul v3 doesn't add remotes like origin but our use of puppet vcsrepo
-    # errors if it can't update itself against origin (because vcsrepo manages
-    # repos that it will initially clone). Address this by adding an origin
-    # for the vcsrepo managed repo here.
-    sudo git -C /etc/project-config remote add origin 'https://opendev.org/openstack/project-config'
 }
 
 install_all() {
@@ -86,7 +70,7 @@ install_all() {
 
 }
 
-if [ -e /usr/zuul-env/bin/zuul-cloner ] ; then
+if [ -d /home/zuul/src/opendev.org ] ; then
     install_external
     install_openstack
 else