opendev/system-config
Code Issues Proposed changes

Merge "Document Zuul's SIGUSR2 handler"

Browse Source
This commit is contained in:
Zuul 2021-10-25 16:03:57 +00:00 committed by Gerrit Code Review
commit ba66fa0c5c
1 changed files with 25 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
doc/source/zuul.rst
View File

@ -275,6 +275,31 @@ the problem. Note that ``Traceback`` messages are not prefixed with the event
ID which means you'll have to grep with additional context, for example using
``grep -B20 -A20``.
Another useful debugging tool is Zuul's SIGUSR2 handler. This signal handler
produces a thread dump in the debug log and toggles the yappi python profiler.
Each Zuul service supports the signal handler and it can be triggered via::
you@zuul02$ sudo kill -USR2 $ZUUL_PID
To determine ``$ZUUL_PID`` you can run ``ps`` against the ``zuul-*`` service
that you are interested in getting information from. For example::
you@zuul02$ ps -ef | grep zuul-scheduler
zuuld 1893030 1893010 0 08:33 ? 00:00:00 /usr/bin/dumb-init -- /usr/local/bin/zuul-scheduler -f
zuuld 1893052 1893030 69 08:33 ? 07:57:42 /usr/local/bin/python /usr/local/bin/zuul-scheduler -f
zuuld 1893198 1893052 0 08:33 ? 00:03:22 /usr/local/bin/python /usr/local/bin/zuul-scheduler -f
All of the zuul services are run under ``dumb-init``. The process to send
SIGUSR2 to is the child of the ``dumb-init`` process. In the example above
``$ZUUL_PID`` would be ``1893052``.
The first time you run it you will turn on the yappi profiler. This profiler
does incur a runtime cost which can significantly slow down Zuul's processing
of pipelines. Be sure to resend the signal once you have let Zuul run long
enough to collect a representative set of profiler data. In most cases a minute
or two should be sufficient. Slow memory leaks may require hours, but running
Zuul under yappi for hours isn't practical.
.. _zuul_github_projects:
GitHub Projects