From 89f59062864e5cbfc839a6084c323ce35438aa57 Mon Sep 17 00:00:00 2001 From: Ben Martin Date: Mon, 27 Jul 2015 14:19:09 -0500 Subject: [PATCH] +Document method to avoid rsync filling root drive When rsync pushes to a remote node with an unmounted drive and if certain steps are not taken, rsync may attempt to write files to the local drive at the location where the drive was mounted. There are two suggested solutions for this issue: 1) Set the permissions for all mount points in /srv/node/ to root:root 755 2) Mount the drives elsewhere and symlink the drives to /srv/.../ The first method ensures that only root and not the swift user can write in the /srv/.../ directories. The second method will prompt a broken link issue if rsync attempts to write to an unmounted drive. Change-Id: I60ce4ed9ef8401768d5f78b6806cbb2e2a65303e Closes-Bug: #1470576 --- doc/source/admin_guide.rst | 4 ++++ doc/source/deployment_guide.rst | 4 ++++ doc/source/development_saio.rst | 19 +++++++++++++++++++ 3 files changed, 27 insertions(+) diff --git a/doc/source/admin_guide.rst b/doc/source/admin_guide.rst index f27c20741e..aca4bbca05 100644 --- a/doc/source/admin_guide.rst +++ b/doc/source/admin_guide.rst @@ -154,6 +154,10 @@ until it has been resolved. If the drive is going to be replaced immediately, then it is just best to replace the drive, format it, remount it, and let replication fill it up. +After the drive is unmounted, make sure the mount point is owned by root +(root:root 755). This ensures that rsync will not try to replicate into the +root drive once the failed drive is unmounted. + If the drive can't be replaced immediately, then it is best to leave it unmounted, and set the device weight to 0. This will allow all the replicas that were on that drive to be replicated elsewhere until the drive diff --git a/doc/source/deployment_guide.rst b/doc/source/deployment_guide.rst index 6969a331c1..3d86746260 100644 --- a/doc/source/deployment_guide.rst +++ b/doc/source/deployment_guide.rst @@ -1229,6 +1229,10 @@ For a standard swift install, all data drives are mounted directly under be sure to set the `devices` config option in all of the server configs to point to the correct directory. +The mount points for each drive in /srv/node/ should be owned by the root user +almost exclusively (root:root 755). This is required to prevent rsync from +syncing files into the root drive in the event a drive is unmounted. + Swift uses system calls to reserve space for new objects being written into the system. If your filesystem does not support `fallocate()` or `posix_fallocate()`, be sure to set the `disable_fallocate = true` config diff --git a/doc/source/development_saio.rst b/doc/source/development_saio.rst index 1e6825f661..e406fb02ef 100644 --- a/doc/source/development_saio.rst +++ b/doc/source/development_saio.rst @@ -95,6 +95,16 @@ another device when creating the VM, and follow these instructions: # **Make sure to include the trailing slash after /srv/$x/** for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done + Note: We create the mount points and mount the storage disk under + /mnt/sdb1. This disk will contain one directory per simulated swift node, + each owned by the current swift user. + + We then create symlinks to these directories under /srv. + If the disk sdb is unmounted, files will not be written under + /srv/\*, because the symbolic link destination /mnt/sdb1/* will not + exist. This prevents disk sync operations from writing to the root + partition in the event a drive is unmounted. + #. Next, skip to :ref:`common-dev-section`. @@ -135,6 +145,15 @@ these instructions: # **Make sure to include the trailing slash after /srv/$x/** for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done + Note: We create the mount points and mount the loopback file under + /mnt/sdb1. This file will contain one directory per simulated swift node, + each owned by the current swift user. + + We then create symlinks to these directories under /srv. + If the loopback file is unmounted, files will not be written under + /srv/\*, because the symbolic link destination /mnt/sdb1/* will not + exist. This prevents disk sync operations from writing to the root + partition in the event a drive is unmounted. .. _common-dev-section: