opendev/system-config
Code Issues Proposed changes

Merge "gerrit docs: cleanup and use shell-session"

Browse Source
This commit is contained in:
Zuul 2022-06-24 05:56:55 +00:00 committed by Gerrit Code Review
commit 2599cdccb8
2 changed files with 118 additions and 131 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

212
doc/source/gerrit.rst
View File

@ -45,26 +45,6 @@ The Gerrit installation at /home/gerrit2 is located on a Cinder
volume. See :ref:`cinder` for details on volume management. Note volume. See :ref:`cinder` for details on volume management. Note
that SSD volumes are used (and they have a minimum size of 100G). that SSD volumes are used (and they have a minimum size of 100G).
Gerrit Configuration
--------------------
Most of Gerrit's configuration is in configuration files or Git
repositories (and in our case, managed by Puppet), but a few items
must be configured in the database. The following is a record of
these changes:
Add information about the CLA:
.. code-block:: mysql
sudo -u root
mysql
use reviewdb;
insert into contributor_agreements values (
'Y', 'Y', 'Y', 'ICLA',
'OpenStack Individual Contributor License Agreement',
'static/cla.html', 2);
Groups Groups
------ ------
@ -99,14 +79,16 @@ Once you've created your groups you should create the
``openstack-project-creator`` account by hand (the account name is ``openstack-project-creator`` account by hand (the account name is
referenced from referenced from
:git_file:`playbooks/roles/gerrit/templates/projects.ini.j2`) :git_file:`playbooks/roles/gerrit/templates/projects.ini.j2`)
using:: using
cat $pubkey | ssh -p 29418 $USER@$HOST gerrit create-account \ .. code-block:: shell-session
--group "'Project Bootstrappers'" \
--group Administrators \ $ cat $pubkey | ssh -p 29418 $USER@$HOST gerrit create-account \
--full-name "'Project Creator'" \ --group "'Project Bootstrappers'" \
--email openstack-infra@lists.openstack.org \ --group Administrators \
--ssh-key - openstack-project-creator --full-name "'Project Creator'" \
--email openstack-infra@lists.openstack.org \
--ssh-key - openstack-project-creator
.. _gerrit_github_integration: .. _gerrit_github_integration:
@ -123,10 +105,12 @@ with instructions to use Gerrit.
These are both handled automatically by :ref:`jeepyb`. These are both handled automatically by :ref:`jeepyb`.
Note that the user running Gerrit will need to accept the GitHub host Note that the user running Gerrit will need to accept the GitHub host
keys. e.g.:: keys. e.g.
sudo su - gerrit2 .. code-block:: shell-session
ssh github.com
$ sudo su - gerrit2
$ ssh github.com
Troubleshooting Troubleshooting
--------------- ---------------
@ -138,9 +122,9 @@ be created, but will appear empty. When trying replication from Gerrit,
it will show a `Permission denied` error when trying to push content. it will show a `Permission denied` error when trying to push content.
To solve that, following steps are needed: To solve that, following steps are needed:
#. Login into github.com, using openstack-project-creator user. #. Login into ``github.com``, using ``openstack-project-creator`` user.
#. Navigate to the failed repository, and enter on Settings > Collaborators #. Navigate to the failed repository, and enter on ``Settings > Collaborators
& teams option. & teams`` option.
#. Add Gerrit as Team member to that project. #. Add Gerrit as Team member to that project.
After the team has been added, project will start replicating successfully After the team has been added, project will start replicating successfully
@ -151,17 +135,16 @@ Gerrit IRC Bot
============== ==============
Gerritbot consumes the Gerrit event stream and announces relevant Gerritbot consumes the Gerrit event stream and announces relevant
events on IRC. :ref:`gerritbot` is an openstack-infra project and is events on IRC. :ref:`gerritbot` is an ``OpenDev`` project and is also
also available on Pypi. available on Pypi.
Launchpad Bug Integration Launchpad Bug Integration
========================= =========================
In addition to the hyperlinks provided by the regex in gerrit.config, In addition to the hyperlinks provided by the regex in ``gerrit.config``,
we use a Gerrit hook to update Launchpad bugs when changes referencing we use a Gerrit hook to update Launchpad bugs when changes referencing
them are applied. This is managed by the :ref:`jeepyb` them are applied. This is managed by the :ref:`jeepyb` project.
openstack-infra project.
Storyboard Integration Storyboard Integration
====================== ======================
@ -175,17 +158,10 @@ New Project Creation
==================== ====================
Gerrit project creation is now managed through changes to the Gerrit project creation is now managed through changes to the
openstack/project-config repository. :ref:`jeepyb` handles ``openstack/project-config`` repository. :ref:`jeepyb` handles
automatically creating any new projects defined in the configuration automatically creating any new projects defined in the configuration
files. files.
Local Git Replica
=================
Gerrit replicates all repos to a local directory so that Apache can
serve the anonymous http requests out directly. This is automatically
configured by :ref:`jeepyb`.
.. _acl: .. _acl:
Access Controls Access Controls
@ -199,7 +175,7 @@ High level goals:
#. Zuul can perform verification (blocking or approving: +/-2). #. Zuul can perform verification (blocking or approving: +/-2).
#. Third Party CI systems can perform informational verification (+/-1). #. Third Party CI systems can perform informational verification (+/-1).
#. All registered users can create changes. #. All registered users can create changes.
#. Members of $PROJECT-core group can perform full code review #. Members of ``$PROJECT-core`` group can perform full code review
(blocking or approving: +/- 2), and submit changes to be merged. (blocking or approving: +/- 2), and submit changes to be merged.
#. Drivers (PTL and delegates) of client library projects should be #. Drivers (PTL and delegates) of client library projects should be
able to add tags (which are automatically used to trigger able to add tags (which are automatically used to trigger
@ -215,8 +191,8 @@ SSH because it's fast, and it gets syntax checked).
#. Visit ``https://$HOST/#/admin/projects/All-Projects,access`` and click on Edit. #. Visit ``https://$HOST/#/admin/projects/All-Projects,access`` and click on Edit.
#. Look for the reference to 'refs/meta/config', click on the drop-box #. Look for the reference to ``refs/meta/config``, click on the drop-box
for 'add permission' and choose 'PUSH'. for ``add permission`` and choose ``PUSH``.
#. Type in Administrators as the group name #. Type in Administrators as the group name
@ -225,22 +201,24 @@ SSH because it's fast, and it gets syntax checked).
#. Click on Save Changes #. Click on Save Changes
Then... we need to fetch the All-Projects ACLs, update them, then push the Then... we need to fetch the All-Projects ACLs, update them, then push the
updates back into Gerrit:: updates back into Gerrit
export USER=$your_gerrit_user .. code-block:: shell-session
export HOST=$your_gerrit_host
cd $anywhereyoulike
mkdir All-Projects-ACLs
cd All-Projects-ACLs
git init
git remote add gerrit ssh://$USER@$HOST:29418/All-Projects.git
git fetch gerrit +refs/meta/*:refs/remotes/gerrit-meta/*
git checkout -b config remotes/gerrit-meta/config
There will be two interesting files, `groups` and `project.config`. $ export USER=$your_gerrit_user
`groups` contains UUIDs and names of groups that will be referenced $ export HOST=$your_gerrit_host
in `project.config`. UUIDs can be found on the group page in Gerrit. $ cd $anywhereyoulike
Next, edit `project.config` to look like:: $ mkdir All-Projects-ACLs
$ cd All-Projects-ACLs
$ git init
$ git remote add gerrit ssh://$USER@$HOST:29418/All-Projects.git
$ git fetch gerrit +refs/meta/*:refs/remotes/gerrit-meta/*
$ git checkout -b config remotes/gerrit-meta/config
There will be two interesting files, ``groups`` and ``project.config``.
``groups`` contains UUIDs and names of groups that will be referenced
in ``project.config``. UUIDs can be found on the group page in Gerrit.
Next, edit ``project.config`` to look like::
[access "refs/*"] [access "refs/*"]
create = group Project Bootstrappers create = group Project Bootstrappers
@ -355,10 +333,12 @@ the built in groups such as 'Service Users' which may or may not be
present in the initial groups file. You can find the UUID values by navigating present in the initial groups file. You can find the UUID values by navigating
to Admin -> Groups -> Group Name -> General in the Web UI. to Admin -> Groups -> Group Name -> General in the Web UI.
Finally, commit the changes and push the config back up to Gerrit:: Finally, commit the changes and push the config back up to Gerrit
git commit -am "Initial All-Projects config" .. code-block:: shell-session
git push gerrit HEAD:refs/meta/config
$ git commit -am "Initial All-Projects config"
$ git push gerrit HEAD:refs/meta/config
Manual Administrative Tasks Manual Administrative Tasks
@ -408,9 +388,11 @@ To rename a project:
responding to ssh to ensure the next step can run successfully. responding to ssh to ensure the next step can run successfully.
#. Run the ansible rename repos playbook, passing in the path to your yaml #. Run the ansible rename repos playbook, passing in the path to your yaml
file:: file
sudo ansible-playbook -f 10 /home/zuul/src/opendev.org/opendev/system-config/playbooks/rename_repos.yaml -e repolist=ABSOLUTE_PATH_TO_VARS_FILE .. code-block:: shell-session
$ sudo ansible-playbook -f 10 /home/zuul/src/opendev.org/opendev/system-config/playbooks/rename_repos.yaml -e repolist=ABSOLUTE_PATH_TO_VARS_FILE
#. :ref:`Force-merge <force-merging-a-change>` the prepared configuration #. :ref:`Force-merge <force-merging-a-change>` the prepared configuration
changes. changes.
@ -429,9 +411,11 @@ To rename a project:
or review servers. It should be a noop. or review servers. It should be a noop.
Developers will either need to re-clone a new copy of the repository, Developers will either need to re-clone a new copy of the repository,
or manually update their remotes with something like:: or manually update their remotes with something like
git remote set-url origin https://opendev.org/$ORG/$PROJECT .. code-block:: shell-session
$ git remote set-url origin https://opendev.org/$ORG/$PROJECT
Third-Party Testing Access Third-Party Testing Access
@ -441,14 +425,9 @@ The command to add an account for an automated system which gets -1/+1
code verify voting rights (as outlined in :ref:`third-party-testing`) code verify voting rights (as outlined in :ref:`third-party-testing`)
looks like: looks like:
.. code-block:: bash .. code-block:: shell-session
ssh -p 29418 review.opendev.org "gerrit create-account \ $ ssh -p 29418 review.opendev.org "gerrit create-account --group 'Third-Party CI' --full-name 'Some CI Bot' --email ci-bot@third-party.org --ssh-key 'ssh-rsa AAAAB3Nz...zaUCse1P ci-bot@third-party.org' some-ci-bot"
--group 'Third-Party CI' \
--full-name 'Some CI Bot' \
--email ci-bot@third-party.org \
--ssh-key 'ssh-rsa AAAAB3Nz...zaUCse1P ci-bot@third-party.org' \
some-ci-bot"
Details on the create-account_ command can be found in the Gerrit Details on the create-account_ command can be found in the Gerrit
API documentation. API documentation.
@ -476,32 +455,32 @@ The first step to resolve this is to confirm the ID of the unwanted
account, . As an admin user with a HTTP password set, search for the account, . As an admin user with a HTTP password set, search for the
new account: new account:
.. .. code-block:: shell-session
curl -u you.admin -i -H "Accept: application/json" 'https://review.opendev.org/a/accounts/foo@new.com' $ curl -u you.admin -i -H "Accept: application/json" 'https://review.opendev.org/a/accounts/foo@new.com'
That will return an ``_acount_id``. For this example, assume it is That will return an ``_acount_id``. For this example, assume it is
``12345``. The user should check in their settings they are *not* ``12345``. The user should check in their settings they are *not*
using this account. using this account.
Clone ``All-Users`` to modify the account, and checkout the account Clone ``All-Users`` to modify the account, and checkout the account
config, which is sharded by the last two digits of the account-id. config, which is sharded by the last two digits of the ``_account_id``.
.. .. code-block:: shell-session
git clone ssh://you.admin@review.opendev.org:29418/All-Users $ git clone ssh://you.admin@review.opendev.org:29418/All-Users
git fetch refs/users/45/12345 $ git fetch origin refs/users/45/12345
git checkout FETCH_HEAD $ git checkout FETCH_HEAD
Edit the ``[account]`` section of ``account.conf`` to remove Edit the ``[account]`` section of ``account.conf`` to remove
``preferredEmail`` and have a line ``active = false``. Put your admin ``preferredEmail`` and have a line ``active = false``. Put your admin
account into ``Project Bootstrappers`` (see :ref:`sysadmin`) and account into ``Project Bootstrappers`` (see :ref:`sysadmin`) and
commit this commit this
.. .. code-block:: shell-session
git commit -m "Make duplicate account inactive" --author <your@email.com> $ git commit -m "Make duplicate account inactive" --author <your@email.com>
git push origin HEAD:refs/users/45/12345 $ git push origin HEAD:refs/users/45/12345
There will still be an OpenID external ID associated with this now There will still be an OpenID external ID associated with this now
inactive account. This will prevent adding ``foo@new.com`` to another inactive account. This will prevent adding ``foo@new.com`` to another
@ -509,17 +488,17 @@ account until this is removed.
Check this via the API with Check this via the API with
.. .. code-block:: shell-session
curl -u you.admin -i -H "Accept: application/json" https://review.opendev.org/a/accounts/12345/external.ids $ curl -u you.admin -i -H "Accept: application/json" https://review.opendev.org/a/accounts/12345/external.ids
This will give a json result with an ``identity`` URL like This will give a json result with an ``identity`` URL like
``"identity":"https://login.ubuntu.com/+id/RaND0m``. Use this to ``"identity":"https://login.ubuntu.com/+id/RaND0m``. Use this to
delete the record with another call delete the record with another call
.. .. code-block:: shell-session
curl -XPOST -u you.admin -i -H "Content-Type: application/json" -d '["https://login.ubuntu.com/+id/RaND0m"]' https://review.opendev.org/a/accounts/12345/external.ids:delete $ curl -XPOST -u you.admin -i -H "Content-Type: application/json" -d '["https://login.ubuntu.com/+id/RaND0m"]' https://review.opendev.org/a/accounts/12345/external.ids:delete
The user should now be able to add ``foo@new.com`` to their old The user should now be able to add ``foo@new.com`` to their old
account. account.
@ -534,40 +513,47 @@ must follow that steps:
members can be found on: https://review.opendev.org/#/admin/groups/270,members members can be found on: https://review.opendev.org/#/admin/groups/270,members
That will give you the name and email of all members. Then you can get the matching That will give you the name and email of all members. Then you can get the matching
numerical account ID with the help of REST API:: numerical account ID with the help of REST API
curl -i -H "Accept: application/json" --digest --user <<gerrit_user>>:<<http_pass>> -X GET https://review.opendev.org/a/accounts/{email} .. code-block:: shell-session
$ curl -i -H "Accept: application/json" --digest --user <<gerrit_user>>:<<http_pass>> -X GET https://review.opendev.org/a/accounts/{email}
This will return a JSON dictionary, that will contain _account_id field. This will return a JSON dictionary, that will contain _account_id field.
2. Mark the account as inactive using gerrit ssh api, with:: 2. Mark the account as inactive using gerrit ssh api, with
ssh -p 29418 review.opendev.org gerrit set-account --inactive {account-id} .. code-block:: shell-session
Alternatively you can use REST API, sending a DELETE for:: $ ssh -p 29418 review.opendev.org gerrit set-account --inactive {account-id}
curl -i -H "Accept: application/json" --digest --user <<gerrit_user>>:<<http_pass>> -X DELETE https://review.opendev.org/a/accounts/{account-id}/active Alternatively you can use REST API, sending a DELETE for
3. Check if there are active gerrit ssh connections:: .. code-block:: shell-session
ssh -p 29418 review.opendev.org gerrit show-connections -n | grep {account-id} $ curl -i -H "Accept: application/json" --digest --user <<gerrit_user>>:<<http_pass>> -X DELETE https://review.opendev.org/a/accounts/{account-id}/active
And kill all of them with subsequent:: 3. Check if there are active gerrit ssh connections
ssh -p 29418 review.opendev.org gerrit close-connection {connection-id} .. code-block:: shell-session
$ ssh -p 29418 review.opendev.org gerrit show-connections -n | grep {account-id}
And kill all of them with subsequent
.. code-block:: shell-session
$ ssh -p 29418 review.opendev.org gerrit close-connection {connection-id}
4. You can check if the account is properly marked as inactive using REST API, 4. You can check if the account is properly marked as inactive using REST API,
sending a GET for:: sending a GET for
curl -i -H "Accept: application/json" --digest --user <<gerrit_user>>:<<http_pass>> -X GET https://review.opendev.org/a/accounts/{account-id}/active .. code-block:: shell-session
$ curl -i -H "Accept: application/json" --digest --user <<gerrit_user>>:<<http_pass>> -X GET https://review.opendev.org/a/accounts/{account-id}/active
A 200 return code means the account is active, and 204 means account inactive. A 200 return code means the account is active, and 204 means account inactive.
4. In the case of a failing Third Party CI, if the account caused a loop of comments in
a change, you can delete them with following query::
delete from change_messages where author_id={account-id} and change_id={change-id};
Generating a Thread Dump for Debugging Generating a Thread Dump for Debugging
-------------------------------------- --------------------------------------
@ -577,8 +563,10 @@ still possible with java command line tools. You may find yourself wanting
to do this if Gerrit is suffering from poor performance or you are trying to to do this if Gerrit is suffering from poor performance or you are trying to
debug odd Gerrit behavior. debug odd Gerrit behavior.
To run jstack and produce a thread dump do:: To run ``jstack`` and produce a thread dump do
review02# docker exec -it gerrit-compose_gerrit_1 bash .. code-block:: shell-session
gerrit@review02:/$ ps -ef | grep java # find the Gerrit java process PID
gerrit@review02:/$ jstack $PID > /tmp/dump.yearmonthday root@review02 # docker exec -it gerrit-compose_gerrit_1 bash
gerrit@review02 $ ps -ef | grep java # find the Gerrit java process PID
gerrit@review02 $ jstack ${PID} > /tmp/dump.yearmonthday

37
doc/source/sysadmin.rst
View File

@ -62,39 +62,36 @@ have no OpenID so that it is only accessable by API/CLI methods so they
cannot be compromised at the third-party ID provider. cannot be compromised at the third-party ID provider.
To create a personal Gerrit admin account from a shell on the server, run To create a personal Gerrit admin account from a shell on the server, run
the following command:: the following command
sudo -u gerrit2 ssh -i ~gerrit2/review_site/etc/ssh_host_rsa_key \ .. code-block:: shell-session
-p 29418 -l 'Gerrit Code Review' localhost \
"suexec --as openstack-project-creator -- \ $ sudo -u gerrit2 ssh -i ~gerrit2/review_site/etc/ssh_host_rsa_key -p 29418 -l 'Gerrit Code Review' localhost "suexec --as openstack-project-creator -- gerrit create-account --group Administrators --full-name myname.admin --ssh-key 'ssh-rsa AAAA...BCDE myname@computer' myname.admin"
gerrit create-account --group Administrators --full-name myname.admin \
--ssh-key 'ssh-rsa AAAA...BCDE myname@computer' myname.admin"
We ``suexec`` as the ``openstack-project-creator`` account because the We ``suexec`` as the ``openstack-project-creator`` account because the
magic ``Gerrit Code Review`` pseudoaccount can't set group memberships so magic ``Gerrit Code Review`` pseudoaccount can't set group memberships so
we need to run that command as a user which is already in the we need to run that command as a user which is already in the
``Administrators`` group. With an account like this, routine actions like ``Administrators`` group. With an account like this, routine actions like
populating new groups with initial members is still quite simple:: populating new groups with initial members is still quite simple
ssh -p 29418 myname.admin@review.opendev.org \ .. code-block:: shell-session
"gerrit set-members some-new-group --add somebody@example.org"
$ ssh -p 29418 myname.admin@review.opendev.org "gerrit set-members some-new-group --add somebody@example.org"
Another common example is bypassing Zuul to submit a change for merging Another common example is bypassing Zuul to submit a change for merging
directly to a project. In this case we must first add our account to directly to a project. In this case we must first add our account to
another group which has permission to set the relevant labels (it doesn't another group which has permission to set the relevant labels (it doesn't
get that simply by being an administrator), and then do the get that simply by being an administrator), and then do the
commenting/voting/submitting, followed by cleaning up the extra group commenting/voting/submitting, followed by cleaning up the extra group
membership again at the end:: membership again at the end
ssh -p 29418 myname.admin@review.opendev.org \ .. code-block:: shell-session
"gerrit set-members 'Project Bootstrappers' --add myname.admin"
ssh -p 29418 myname.admin@review.opendev.org \ $ ssh -p 29418 myname.admin@review.opendev.org "gerrit set-members 'Project Bootstrappers' --add myname.admin"
"gerrit review 12345,6 --message 'Bypassing Zuul to merge this.' \
--code-review=2 --verified=2 --label workflow=1 --submit"
ssh -p 29418 myname.admin@review.opendev.org \ $ ssh -p 29418 myname.admin@review.opendev.org "gerrit review 12345,6 --message 'Bypassing Zuul to merge this.' --code-review=2 --verified=2 --label workflow=1 --submit"
"gerrit set-members 'Project Bootstrappers' --remove myname.admin"
$ ssh -p 29418 myname.admin@review.opendev.org "gerrit set-members 'Project Bootstrappers' --remove myname.admin"
Note that it's possible to temporarily add your normal OpenID-associated Note that it's possible to temporarily add your normal OpenID-associated
WebUI account to the ``Administrators`` group or other groups with similar WebUI account to the ``Administrators`` group or other groups with similar
@ -141,9 +138,11 @@ Accessing Clouds
================ ================
As an unprivileged user who is a member of the `sudo` group on bridge, As an unprivileged user who is a member of the `sudo` group on bridge,
you can inspect any of the clouds with:: you can inspect any of the clouds with
sudo openstack --os-cloud <cloud name> --os-cloud-region <region name> .. code-block:: shell-session
$ sudo openstack --os-cloud <cloud name> --os-cloud-region <region name>
Backups Backups
======= =======