opendev/system-config
Code Issues Proposed changes

Add doc on using jenkins restart playbook.

Browse Source 
Add documentation on how to run the jenkins restart playbook against a
specific jenkins master. This is useful if a jenkins master starts to
leak threads before its weekly restart.

Change-Id: Ib5163589c1c83e4fcb7493daa387f42cda02bc9d
This commit is contained in:
Clark Boylan 2015-12-10 11:04:09 -08:00
parent 93f1ea9bb8
commit 7a2c00ddd8
1 changed files with 23 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
doc/source/jenkins.rst
View File

@ -165,3 +165,26 @@ slave is then:
#. Set appropriate labels on it
#. Profit!
Safe Master Restarts
====================
Jenkins masters periodically leak threads reducing their job
throughput and eventually leading to crashes. We work around this
by performing weekly rolling restarts of the Jenkins masters with
an ansible playbook.
If you need to perform a safe restart against a single master you
can do this by running the same playbook and limiting it to a
specific jenkins master
To do this::
root@puppetmaster# ansible-playbook -f1 --limit $server_fqdn \
/opt/system-config/production/playbooks/restart_jenkins_masters.yaml \
--extra-vars "user=hudson-openstack \
password=$(/opt/system-config/production/tools/hieraedit.py \
--yaml /etc/puppet/hieradata/production/fqdn/nodepool.openstack.org.yaml jenkins_api_key)"
Consider running this in screen as the worst case run time is as
long as our longest running job.