openstack/cinder-specs
Code Issues Proposed changes
cinder-specs/specs/liberty/rootwrap-daemon-mode.rst
Rajat Dhasmana 6e6af443fb Fix: docs job 
The warnings for footnotes not being referenced are now treated
as errors and we see gate is failing with the following errors.

2024-11-04 08:55:48.092592 | ubuntu-jammy | /home/zuul/src/opendev.org/openstack/cinder-specs/doc/source/specs/liberty/rootwrap-daemon-mode.rst:180: WARNING: Footnote [#] is not referenced. [ref.footnote]
2024-11-04 08:55:48.093125 | ubuntu-jammy | /home/zuul/src/opendev.org/openstack/cinder-specs/doc/source/specs/liberty/rootwrap-daemon-mode.rst:186: WARNING: Footnote [#] is not referenced. [ref.footnote]
2024-11-04 08:55:48.093154 | ubuntu-jammy | /home/zuul/src/opendev.org/openstack/cinder-specs/doc/source/specs/liberty/rootwrap-daemon-mode.rst:189: WARNING: Footnote [#] is not referenced. [ref.footnote]

This is easy to fix by adding references to the footnotes as done
in this patch.

Change-Id: I9991ad83abac99b10d8d7421c0934e1eb84b26cc
2024-11-04 14:28:26 +05:30

5.7 KiB
Raw Blame History

Rootwrap daemon mode

https://blueprints.launchpad.net/cinder/+spec/rootwrap-daemon-mode

Cinder is one of projects that require root privileges. Currently this is achieved with oslo.rootwrap that has to be run with sudo. Both sudo and rootwrap produce significant performance overhead. This blueprint is one of the series of blueprints that would cover mitigating rootwrap part of the overhead using new mode of operations for rootwrap - daemon mode. These blueprints will be created in several projects starting with oslo.rootwrap1.

Problem description

As you can see in2 rootwrap presents big performance overhead for Neutron. Impact on Cinder is not as significant but it is still there. Details of the overhead are covered in3.

Use Cases

This will eliminate bottleneck in large number of concurrent executed operations.

Proposed change

This blueprint proposes implement changes that allow to run oslo.rootwrap daemon. The daemon works just as a usual rootwrap but accepts commands to be run over authenticated UNIX domain socket instead of command line and run continuously in background.

Note that this is not usual RPC over some message queue. It uses UNIX socket, so no remote connections are available. It also uses digest authentication with key shared over stdout (pipe) with parent process, so no other processes will have access to the daemon. Further details of rootwrap daemon are covered in4.

use_rootwrap_daemon configuration option should be added that will make utils.execute use daemon instead of usual rootwrap.

Alternatives

Alternative approaches have been discussed in5.

Data model impact

None

REST API impact

None

Security impact

This change requires additional endpoint to be available to run as root -cinder-rootwrap-daemon.

All security issues with using client+daemon instead of plain rootwrap are covered in6.

Notifications impact

None

Other end user impact

None

Performance Impact

This change introduces performance boost for disk operations that are required to be run with root privileges. Current state of rootwrap daemon in Neutron shows over 10x speedup comparing to usual sudo rootwrap call. Total speedup for Cinder shows impressive results too7: test scenario CinderVolumes.create_and_delete_volume Current performance :

action min (sec) avg (sec) max (sec) count
cinder.create_volume cinder.delete_volume total 2.779 13.535 16.314 5.76 24.958 30.718 14.375 32.959 35.96 8 8 8

Load duration: 131.423681974 Full duration: 135.794852018

With use_rootwrap_daemon enabled:

action min (sec) avg (sec) max (sec) count
cinder.create_volume cinder.delete_volume total 2.49 2.183 4.673 2.619 2.226 4.845 3.086 2.353 5.3 8 8 8

Load duration: 19.7548749447 Full duration: 22.2729279995

Other deployer impact

This change introduces new config variable use_rootwrap_daemon that switches on new behavior. Note that by default use_rootwrap_daemon will be turned off so to get the speedup one will have to turn it on. With it turned on cinder-rootwrap-daemon is used to run commands that require root privileges.

This change also introduces new binary cinder-rootwrap-daemon that should be deployed beside cinder-rootwrap.

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

Anton Arefiev(aarefiev)

Work Items

The only work item here is to implement new config variable and run rootwrap in daemon mode with it.

Dependencies

  • rootwrap-daemon-mode blueprint in oslo.rootwrap8.

Testing

Rootwrap has it's own functional testing for the rootwrap client/daemon pieces9, Cinder part will be covered by unit tests.

Cinder has an unusual usecase where tens/hundreds of mb are passed over stdin/out (sheepdog backup) - that test case should be covered in the functional tests.

Also we can add new Tempest job with turned on use rootwrap daemon flag.

Documentation Impact

Set use_rootwrap_daemon=True configuration option in cinder.conf to make utils.execute use daemon instead of usual rootwrap.

References

  1. oslo.rootwrap blueprint: https://blueprints.launchpad.net/oslo.rootwrap/+spec/rootwrap-daemon-mode↩︎

  2. Original mailing list thread: http://lists.openstack.org/pipermail/openstack-dev/2014-March/029017.html↩︎

  3. oslo.rootwrap blueprint: https://blueprints.launchpad.net/oslo.rootwrap/+spec/rootwrap-daemon-mode↩︎

  4. oslo.rootwrap blueprint: https://blueprints.launchpad.net/oslo.rootwrap/+spec/rootwrap-daemon-mode↩︎

  5. Alternative approaches https://etherpad.openstack.org/p/neutron-agent-exec-performance↩︎

  6. oslo.rootwrap blueprint: https://blueprints.launchpad.net/oslo.rootwrap/+spec/rootwrap-daemon-mode↩︎

  7. Cinder performance testing results http://paste.openstack.org/show/160890/↩︎

  8. oslo.rootwrap blueprint: https://blueprints.launchpad.net/oslo.rootwrap/+spec/rootwrap-daemon-mode↩︎

  9. Rootwrap daemon functional testing https://github.com/openstack/oslo.rootwrap/blob/master/tests/test_functional.py↩︎