5016627f04
Spec repos do not have code to unit test. The gate job definition for the py27 and py35 jobs skip if there doc only changes, which is all we will ever have in the specs repo. Therefore the one "test" we had will never be run. We were using this unit test as a check for formatting issues with the rst files. This was done before doc8 was available. Now that we can use doc8, we should just switch to running that as part of the pep8 jobs. Also fixes all the errors caught by doc8 that were not caught by our unit test check. Change-Id: Ida20764edde3a07c89703d82b41958c96548b239
5.7 KiB
Rootwrap daemon mode
https://blueprints.launchpad.net/cinder/+spec/rootwrap-daemon-mode
Cinder is one of projects that require root privileges. Currently this is achieved with oslo.rootwrap that has to be run with sudo. Both sudo and rootwrap produce significant performance overhead. This blueprint is one of the series of blueprints that would cover mitigating rootwrap part of the overhead using new mode of operations for rootwrap - daemon mode. These blueprints will be created in several projects starting with oslo.rootwrap [#rw_bp].
Problem description
As you can see in [#ne_ml] rootwrap presents big performance overhead for Neutron. Impact on Cinder is not as significant but it is still there. Details of the overhead are covered in [#rw_bp].
Use Cases
This will eliminate bottleneck in large number of concurrent executed operations.
Proposed change
This blueprint proposes implement changes that allow to run oslo.rootwrap daemon. The daemon works just as a usual rootwrap but accepts commands to be run over authenticated UNIX domain socket instead of command line and run continuously in background.
Note that this is not usual RPC over some message queue. It uses UNIX socket, so no remote connections are available. It also uses digest authentication with key shared over stdout (pipe) with parent process, so no other processes will have access to the daemon. Further details of rootwrap daemon are covered in [#rw_bp].
use_rootwrap_daemon configuration option should be added
that will make utils.execute use daemon instead of usual
rootwrap.
Alternatives
Alternative approaches have been discussed in [#rw_eth].
Data model impact
None
REST API impact
None
Security impact
This change requires additional endpoint to be available to run as
root -cinder-rootwrap-daemon.
All security issues with using client+daemon instead of plain rootwrap are covered in [#rw_bp].
Notifications impact
None
Other end user impact
None
Performance Impact
This change introduces performance boost for disk operations that are
required to be run with root privileges. Current state of rootwrap
daemon in Neutron shows over 10x speedup comparing to usual
sudo rootwrap call. Total speedup for Cinder shows
impressive results too [#rw_perf]: test scenario
CinderVolumes.create_and_delete_volume Current performance :
| action | min (sec) | avg (sec) | max (sec) | count |
| cinder.create_volume cinder.delete_volume total | 2.779 13.535 16.314 | 5.76 24.958 30.718 | 14.375 32.959 35.96 | 8 8 8 |
Load duration: 131.423681974 Full duration: 135.794852018
With use_rootwrap_daemon enabled:
| action | min (sec) | avg (sec) | max (sec) | count |
| cinder.create_volume cinder.delete_volume total | 2.49 2.183 4.673 | 2.619 2.226 4.845 | 3.086 2.353 5.3 | 8 8 8 |
Load duration: 19.7548749447 Full duration: 22.2729279995
Other deployer impact
This change introduces new config variable
use_rootwrap_daemon that switches on new behavior. Note
that by default use_rootwrap_daemon will be turned off so
to get the speedup one will have to turn it on. With it turned on
cinder-rootwrap-daemon is used to run commands that require
root privileges.
This change also introduces new binary
cinder-rootwrap-daemon that should be deployed beside
cinder-rootwrap.
Developer impact
None
Implementation
Assignee(s)
- Primary assignee:
-
Anton Arefiev(aarefiev)
Work Items
The only work item here is to implement new config variable and run rootwrap in daemon mode with it.
Dependencies
- rootwrap-daemon-mode blueprint in oslo.rootwrap1.
Testing
Rootwrap has it's own functional testing for the rootwrap client/daemon pieces2, Cinder part will be covered by unit tests.
Cinder has an unusual usecase where tens/hundreds of mb are passed over stdin/out (sheepdog backup) - that test case should be covered in the functional tests.
Also we can add new Tempest job with turned on use rootwrap daemon flag.
Documentation Impact
Set use_rootwrap_daemon=True configuration option in
cinder.conf to make utils.execute use daemon instead of
usual rootwrap.
References
oslo.rootwrap blueprint: https://blueprints.launchpad.net/oslo.rootwrap/+spec/rootwrap-daemon-mode↩︎
Rootwrap daemon functional testing https://github.com/openstack/oslo.rootwrap/blob/master/tests/test_functional.py↩︎