Default developer use case to systemd
This moves the developer use case over to systemd, and updates all the relevant docs to discuss the systemd workflow instead of screen. It does so by defaulting USE_SCREEN=False, so will not impact people that set it explicitly. Change-Id: I6d664612bc2b850eb7f56852afbc841867223ab7
This commit is contained in:
Sean Dague
parent
803acffcf9
commit
8b8441f3be
@ -278,43 +278,22 @@ number of days of old log files to keep.
|
||||
|
||||
LOGDAYS=1
|
||||
|
||||
The some of the project logs (Nova, Cinder, etc) will be colorized by
|
||||
default (if ``SYSLOG`` is not set below); this can be turned off by
|
||||
setting ``LOG_COLOR`` to ``False``.
|
||||
|
||||
::
|
||||
Some coloring is used during the DevStack runs to make it easier to
|
||||
see what is going on. This can be disabled with::
|
||||
|
||||
LOG_COLOR=False
|
||||
|
||||
Logging the Service Output
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
DevStack will log the ``stdout`` output of the services it starts.
|
||||
When using ``screen`` this logs the output in the screen windows to a
|
||||
file. Without ``screen`` this simply redirects stdout of the service
|
||||
process to a file in ``LOGDIR``.
|
||||
By default, services run under ``systemd`` and are natively logging to
|
||||
the systemd journal.
|
||||
|
||||
::
|
||||
To query the logs use the ``journalctl`` command, such as::
|
||||
|
||||
LOGDIR=$DEST/logs
|
||||
|
||||
Note the use of ``DEST`` to locate the main install directory; this
|
||||
is why we suggest setting it in ``local.conf``.
|
||||
|
||||
Enabling Syslog
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
Logging all services to a single syslog can be convenient. Enable
|
||||
syslogging by setting ``SYSLOG`` to ``True``. If the destination log
|
||||
host is not localhost ``SYSLOG_HOST`` and ``SYSLOG_PORT`` can be used
|
||||
to direct the message stream to the log host.
|
||||
|
||||
::
|
||||
|
||||
SYSLOG=True
|
||||
SYSLOG_HOST=$HOST_IP
|
||||
SYSLOG_PORT=516
|
||||
journalctl --unit devstack@*
|
||||
|
||||
More examples can be found in :ref:`journalctl-examples`.
|
||||
|
||||
Example Logging Configuration
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
@ -326,7 +305,6 @@ a file, keep service logs and disable color in the stored files.
|
||||
|
||||
[[local|localrc]]
|
||||
DEST=/opt/stack/
|
||||
LOGDIR=$DEST/logs
|
||||
LOGFILE=$LOGDIR/stack.sh.log
|
||||
LOG_COLOR=False
|
||||
|
||||
@ -587,9 +565,7 @@ Swift
|
||||
|
||||
Swift is disabled by default. When enabled, it is configured with
|
||||
only one replica to avoid being IO/memory intensive on a small
|
||||
VM. When running with only one replica the account, container and
|
||||
object services will run directly in screen. The others services like
|
||||
replicator, updaters or auditor runs in background.
|
||||
VM.
|
||||
|
||||
If you would like to enable Swift you can add this to your ``localrc``
|
||||
section:
|
||||
@ -630,7 +606,7 @@ install the swift3 middleware emulation. Swift will be configured to
|
||||
act as a S3 endpoint for Keystone so effectively replacing the
|
||||
``nova-objectstore``.
|
||||
|
||||
Only Swift proxy server is launched in the screen session all other
|
||||
Only Swift proxy server is launched in the systemd system all other
|
||||
services are started in background and managed by ``swift-init`` tool.
|
||||
|
||||
Heat
|
||||
|
||||
@ -8,56 +8,33 @@ with it?
|
||||
Inspecting Services
|
||||
===================
|
||||
|
||||
By default most services in DevStack are running in a `screen
|
||||
<https://www.gnu.org/software/screen/manual/screen.html>`_
|
||||
session.
|
||||
By default most services in DevStack are running as `systemd` units
|
||||
named `devstack@$servicename.service`. You can see running services
|
||||
with.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
os3:~> screen -list
|
||||
There is a screen on:
|
||||
28994.stack (08/10/2016 09:01:33 PM) (Detached)
|
||||
1 Socket in /var/run/screen/S-sdague.
|
||||
sudo systemctl status --unit="devstack@*"
|
||||
|
||||
You can attach to this screen session using ``screen -r`` which gives
|
||||
you a view of the services in action.
|
||||
|
||||
.. image:: assets/images/screen_session_1.png
|
||||
:width: 100%
|
||||
|
||||
Basic Screen Commands
|
||||
---------------------
|
||||
|
||||
The following minimal commands will be useful to using screen:
|
||||
|
||||
* ``ctrl-a n`` - go to next window. Next is assumed to be right of
|
||||
current window.
|
||||
* ``ctrl-a p`` - go to previous window. Previous is assumed to be left
|
||||
of current window.
|
||||
* ``ctrl-a [`` - entry copy/scrollback mode. This allows you to
|
||||
navigate back through the logs with the up arrow.
|
||||
* ``ctrl-a d`` - detach from screen. Gets you back to a normal
|
||||
terminal, while leaving everything running.
|
||||
|
||||
For more about using screen, see the excellent `screen manual
|
||||
<https://www.gnu.org/software/screen/manual/screen.html>`_.
|
||||
To learn more about the basics of systemd, see :doc:`/systemd`
|
||||
|
||||
Patching a Service
|
||||
==================
|
||||
|
||||
If you want to make a quick change to a running service the easiest
|
||||
way to do this is:
|
||||
way to do that is to change the code directly in /opt/stack/$service
|
||||
and then restart the affected daemons.
|
||||
|
||||
* attach to screen
|
||||
* navigate to the window in question
|
||||
* ``ctrl-c`` to kill the service
|
||||
* make appropriate changes to the code
|
||||
* ``up arrow`` in the screen window to display the command used to run
|
||||
that service
|
||||
* ``enter`` to restart the service
|
||||
.. code-block:: bash
|
||||
|
||||
This works for services, except those running under Apache (currently
|
||||
just ``keystone`` by default).
|
||||
sudo systemctl restart --unit=devstack@n-cpu.service
|
||||
|
||||
If your change impacts more than one daemon you can restart by
|
||||
wildcard as well.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo systemctl restart --unit="devstack@n-*"
|
||||
|
||||
.. warning::
|
||||
|
||||
@ -102,14 +79,6 @@ in gerrit by using the ref name that gerrit assigns to each change.
|
||||
NOVA_BRANCH=refs/changes/10/353710/1
|
||||
|
||||
|
||||
Testing Changes to Apache Based Services
|
||||
========================================
|
||||
|
||||
When testing changes to Apache based services, such as ``keystone``,
|
||||
you can either use the Testing a Patch Series approach above, or make
|
||||
changes in the code tree and issue an apache restart.
|
||||
|
||||
|
||||
Testing Changes to Libraries
|
||||
============================
|
||||
|
||||
@ -132,9 +101,17 @@ your changes instead of just upstream master.
|
||||
OSLOPOLICY_REPO=/home/sdague/oslo.policy
|
||||
OSLOPOLICY_BRANCH=better_exception
|
||||
|
||||
Because libraries are used by many services, library changes really
|
||||
need to go through a full ``./unstack.sh && ./stack.sh`` to see your
|
||||
changes in action.
|
||||
As libraries are not installed `editable` by pip, after you make any
|
||||
local changes you will need to:
|
||||
|
||||
To figure out the repo / branch names for every library that's
|
||||
supported, you'll need to read the devstack source.
|
||||
* cd to top of library path
|
||||
* sudo pip install -U .
|
||||
* restart all services you want to use the new library
|
||||
|
||||
You can do that with wildcards such as
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo systemctl restart --unit="devstack@n-*"
|
||||
|
||||
which will restart all nova services.
|
||||
|
||||
@ -41,8 +41,9 @@ Why not use packages?
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Unlike packages, DevStack leaves your cloud ready to develop -
|
||||
checkouts of the code and services running in screen. However, many
|
||||
people are doing the hard work of packaging and recipes for production
|
||||
checkouts of the code and services running locally under systemd,
|
||||
making it easy to hack on and test new patches. However, many people
|
||||
are doing the hard work of packaging and recipes for production
|
||||
deployments.
|
||||
|
||||
Why isn't $MY\_FAVORITE\_DISTRO supported?
|
||||
|
||||
@ -21,3 +21,4 @@
|
||||
development
|
||||
hacking
|
||||
guides
|
||||
systemd
|
||||
|
||||
@ -2,12 +2,7 @@
|
||||
Using Systemd in DevStack
|
||||
===========================
|
||||
|
||||
.. note::
|
||||
|
||||
This is an in progress document as we work out the way forward here
|
||||
with DevStack and systemd.
|
||||
|
||||
DevStack can be run with all the services as systemd unit
|
||||
By default DevStack is run with all the services as systemd unit
|
||||
files. Systemd is now the default init system for nearly every Linux
|
||||
distro, and systemd encodes and solves many of the problems related to
|
||||
poorly running processes.
|
||||
@ -25,12 +20,6 @@ There is also a common developer workflow of changing code in more
|
||||
than one service, and needing to restart a bunch of services for that
|
||||
to take effect.
|
||||
|
||||
To enable this add the following to your local.conf::
|
||||
|
||||
USE_SYSTEMD=True
|
||||
|
||||
|
||||
|
||||
Unit Structure
|
||||
==============
|
||||
|
||||
@ -42,8 +31,9 @@ Unit Structure
|
||||
code is left in place in case we can switch back later.
|
||||
|
||||
All DevStack user units are created as a part of the DevStack slice
|
||||
given the name ``devstack@$servicename.service``. This lets us do
|
||||
certain operations at the slice level.
|
||||
given the name ``devstack@$servicename.service``. This makes it easy
|
||||
to understand which services are part of the devstack run, and lets us
|
||||
disable / stop them in a single command.
|
||||
|
||||
Manipulating Units
|
||||
==================
|
||||
@ -89,6 +79,8 @@ Or to see the status of all Nova processes you can do::
|
||||
We'll eventually make the unit names a bit more meaningful so that
|
||||
it's easier to understand what you are restarting.
|
||||
|
||||
.. _journalctl-examples:
|
||||
|
||||
Querying Logs
|
||||
=============
|
||||
|
||||
@ -126,7 +118,7 @@ pypi, and they are all very different. They unfortunately all install
|
||||
into the ``systemd`` namespace, which can cause some issues.
|
||||
|
||||
- ``systemd-python`` - this is the upstream maintained library, it has
|
||||
a version number like systemd itself (currently ``233``). This is
|
||||
a version number like systemd itself (currently ``234``). This is
|
||||
the one you want.
|
||||
- ``systemd`` - a python 3 only library, not what you want.
|
||||
- ``python-systemd`` - another library you don't want. Installing it
|
||||
@ -145,20 +137,6 @@ system units.
|
||||
Future Work
|
||||
===========
|
||||
|
||||
oslo.log journald
|
||||
-----------------
|
||||
|
||||
Journald has an extremely rich mechanism for direct logging including
|
||||
structured metadata. We should enhance oslo.log to take advantage of
|
||||
that. It would let us do things like::
|
||||
|
||||
journalctl REQUEST_ID=......
|
||||
|
||||
journalctl INSTANCE_ID=......
|
||||
|
||||
And get all lines related to the request id or instance id. (Note:
|
||||
this work has been started at https://review.openstack.org/#/c/451525/)
|
||||
|
||||
log colorizing
|
||||
--------------
|
||||
|
||||
7
stack.sh
7
stack.sh
@ -1474,6 +1474,13 @@ if [[ -n "$DEPRECATED_TEXT" ]]; then
|
||||
echo_summary "WARNING: $DEPRECATED_TEXT"
|
||||
fi
|
||||
|
||||
# If USE_SYSTEMD is enabled, tell the user about using it.
|
||||
if [[ "$USE_SYSTEMD" == "True" ]]; then
|
||||
echo "Services are running under systemd unit files."
|
||||
echo "For more information see: "
|
||||
echo "https://docs.openstack.org/developer/devstack/systemd.html"
|
||||
fi
|
||||
|
||||
# Indicate how long this took to run (bash maintained variable ``SECONDS``)
|
||||
echo_summary "stack.sh completed in $SECONDS seconds."
|
||||
|
||||
|
||||
16
stackrc
16
stackrc
@ -80,12 +80,19 @@ NOVA_ENABLED_APIS=osapi_compute,metadata
|
||||
# Set the root URL for Horizon
|
||||
HORIZON_APACHE_ROOT="/dashboard"
|
||||
|
||||
# TODO(sdague): Queens
|
||||
#
|
||||
# All the non systemd paths should be removed in queens, they only
|
||||
# exist in Pike to support testing from grenade. Ensure that all this
|
||||
# is cleaned up and purged, which should dramatically simplify the
|
||||
# devstack codebase.
|
||||
|
||||
# Whether to use 'dev mode' for screen windows. Dev mode works by
|
||||
# stuffing text into the screen windows so that a developer can use
|
||||
# ctrl-c, up-arrow, enter to restart the service. Starting services
|
||||
# this way is slightly unreliable, and a bit slower, so this can
|
||||
# be disabled for automated testing by setting this value to False.
|
||||
USE_SCREEN=$(trueorfalse True USE_SCREEN)
|
||||
USE_SCREEN=$(trueorfalse False USE_SCREEN)
|
||||
|
||||
# Whether to use SYSTEMD to manage services
|
||||
USE_SYSTEMD=$(trueorfalse False USE_SYSTEMD)
|
||||
@ -100,9 +107,6 @@ else
|
||||
JOURNALCTL_F="journalctl -f -o short-precise --unit"
|
||||
fi
|
||||
|
||||
if [[ "$USE_SYSTEMD" == "True" ]]; then
|
||||
USE_SCREEN=False
|
||||
fi
|
||||
|
||||
# Whether or not to enable Kernel Samepage Merging (KSM) if available.
|
||||
# This allows programs that mark their memory as mergeable to share
|
||||
@ -157,6 +161,10 @@ elif [[ -f $RC_DIR/.localrc.auto ]]; then
|
||||
source $RC_DIR/.localrc.auto
|
||||
fi
|
||||
|
||||
# TODO(sdague): Delete all this in Queens.
|
||||
if [[ "$USE_SYSTEMD" == "True" ]]; then
|
||||
USE_SCREEN=False
|
||||
fi
|
||||
# if we are forcing off USE_SCREEN (as we do in the gate), force on
|
||||
# systemd. This allows us to drop one of 3 paths through the code.
|
||||
if [[ "$USE_SCREEN" == "False" ]]; then
|
||||
|
||||
Loading…
Reference in New Issue
Block a user