openstack/swift
Code Issues Proposed changes

Add container sharding documentation

Browse Source 
Co-Authored-By: Matthew Oliver <matt@oliver.net.au>
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: John Dickinson <me@not.mn>

Change-Id: I0693e54c1d7f3b77f53c3df5c616a16f74723b97
This commit is contained in:
Alistair Coles 2018-05-02 09:33:17 +01:00
parent 2641814010
commit 5c5b08d047
16 changed files with 14524 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
doc/source/container.rst
View File

@ -24,6 +24,16 @@ Container Backend
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
.. _container-replicator:
Container Replicator
====================
.. automodule:: swift.container.replicator
:members:
:undoc-members:
:show-inheritance:
.. _container-server: .. _container-server:
Container Server Container Server
@ -44,12 +54,12 @@ Container Reconciler
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
.. _container-replicator: .. _container-sharder:
Container Replicator Container Sharder
==================== =================
.. automodule:: swift.container.replicator .. automodule:: swift.container.sharder
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:

2019
doc/source/images/sharded_GET.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 80 KiB

2112
doc/source/images/sharding_GET.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 86 KiB

1694
doc/source/images/sharding_cleave1_load.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 67 KiB

1754
doc/source/images/sharding_cleave2_load.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 71 KiB

649
doc/source/images/sharding_cleave_basic.svg Normal file
View File

@ -0,0 +1,649 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="630"
height="235"
version="1.1"
id="svg161"
sodipodi:docname="sharding_snip5.svg"
inkscape:version="0.92.2 (5c3e80d, 2017-08-06)">
<metadata
id="metadata167">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title></dc:title>
</cc:Work>
</rdf:RDF>
</metadata>
<defs
id="defs165">
<defs
id="defs157">
<path
id="f"
d="M 0,20 411,-1484 H 569 L 162,20 H 0"
inkscape:connector-curvature="0" />
<path
id="g"
d="M 797,-207 C 713,-60 620,16 414,20 203,24 88,-98 87,-302 c 0,-112 37,-198 111,-258 74,-60 192,-93 356,-96 l 243,-4 c 10,-201 -43,-307 -232,-305 -154,2 -223,43 -242,172 l -188,-17 c 31,-195 175,-292 434,-292 259,0 410,116 410,364 v 466 c 3,95 12,159 101,161 17,0 37,-2 59,-7 V -6 C 1094,5 1047,10 1000,10 857,8 812,-66 803,-207 Z m -525,-92 c -1,116 66,187 183,184 230,-7 361,-164 342,-419 -124,5 -305,-2 -389,30 -83,32 -136,92 -136,205"
inkscape:connector-curvature="0" />
<path
id="h"
d="m 779,-765 c -19,-119 -93,-196 -233,-196 -95,0 -164,32 -207,95 -43,63 -64,170 -64,320 0,144 23,251 68,320 45,69 114,104 205,104 139,0 224,-79 240,-212 l 182,12 C 946,-119 781,22 553,20 230,17 87,-201 87,-542 c 0,-338 145,-557 464,-560 225,-1 380,129 413,323"
inkscape:connector-curvature="0" />
<path
id="i"
d="m 336,-268 c 2,98 22,139 114,141 24,0 59,-5 104,-14 V -8 C 495,8 434,16 372,16 228,16 156,-66 156,-229 V -951 H 31 v -131 h 132 l 53,-242 h 120 v 242 h 200 v 131 H 336 v 683"
inkscape:connector-curvature="0" />
<g
id="a">
<use
id="use42"
xlink:href="#f"
transform="scale(0.01736111)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use44"
xlink:href="#g"
transform="matrix(0.01736111,0,0,0.01736111,9.8784722,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use46"
xlink:href="#h"
transform="matrix(0.01736111,0,0,0.01736111,29.652778,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use48"
xlink:href="#h"
transform="matrix(0.01736111,0,0,0.01736111,47.430556,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use50"
xlink:href="#i"
transform="matrix(0.01736111,0,0,0.01736111,65.208333,0)"
x="0"
y="0"
width="100%"
height="100%" />
</g>
<path
id="j"
d="M 187,0 V -219 H 382 V 0 H 187"
inkscape:connector-curvature="0" />
<path
id="k"
d="m 513,-963 c -139,1 -238,29 -238,149 0,72 41,100 95,127 55,27 348,94 404,125 101,56 176,118 176,263 C 950,-73 758,21 511,20 254,19 107,-55 57,-254 l 159,-31 c 34,123 133,168 295,168 156,0 264,-34 264,-168 0,-156 -183,-165 -315,-204 -171,-51 -245,-68 -323,-172 -26,-35 -37,-82 -37,-135 0,-220 172,-304 413,-303 232,1 379,74 418,265 l -162,20 C 746,-918 647,-964 513,-963"
inkscape:connector-curvature="0" />
<path
id="l"
d="m 322,-1484 c -2,195 6,405 -8,587 h 3 c 73,-129 159,-205 346,-205 250,0 342,118 343,381 V 0 H 825 v -686 c 3,-190 -43,-277 -223,-277 -176,-1 -280,140 -280,325 V 0 H 142 v -1484 h 180"
inkscape:connector-curvature="0" />
<path
id="m"
d="m 318,-861 c 55,-157 83,-241 257,-241 24,0 48,3 73,10 v 165 c -24,-7 -56,-10 -96,-10 -75,0 -132,33 -171,97 -39,64 -59,156 -59,276 V 0 H 142 c -3,-364 6,-725 -6,-1082 h 170 c 5,123 8,196 8,221 h 4"
inkscape:connector-curvature="0" />
<path
id="n"
d="m 835,0 c -5,-29 -11,-137 -10,-174 h -4 C 759,-45 663,20 484,20 177,20 86,-201 86,-536 c 0,-377 133,-566 398,-566 178,1 273,67 339,188 -4,-187 -1,-380 -2,-570 h 180 v 1261 c 0,113 2,187 6,223 z m -14,-554 c 0,-255 -58,-415 -289,-415 -91,0 -151,37 -196,101 -76,109 -77,543 -1,651 44,63 105,98 195,98 235,0 291,-173 291,-435"
inkscape:connector-curvature="0" />
<path
id="o"
d="m 87,-548 c 0,-338 159,-553 484,-554 325,-2 484,204 477,599 H 276 c -1,227 88,385 302,388 146,2 246,-65 283,-166 l 158,45 C 954,-65 807,20 578,20 240,20 87,-193 87,-548 Z m 775,-93 c -19,-206 -90,-328 -294,-328 -185,0 -285,140 -290,328 h 584"
inkscape:connector-curvature="0" />
<path
id="p"
d="M -31,407 V 277 H 1162 V 407 H -31"
inkscape:connector-curvature="0" />
<g
id="b">
<use
id="use60"
xlink:href="#f"
transform="scale(0.01736111)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use62"
xlink:href="#j"
transform="matrix(0.01736111,0,0,0.01736111,9.8784722,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use64"
xlink:href="#k"
transform="matrix(0.01736111,0,0,0.01736111,19.756944,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use66"
xlink:href="#l"
transform="matrix(0.01736111,0,0,0.01736111,37.534722,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use68"
xlink:href="#g"
transform="matrix(0.01736111,0,0,0.01736111,57.309028,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use70"
xlink:href="#m"
transform="matrix(0.01736111,0,0,0.01736111,77.083333,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use72"
xlink:href="#n"
transform="matrix(0.01736111,0,0,0.01736111,88.923611,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use74"
xlink:href="#o"
transform="matrix(0.01736111,0,0,0.01736111,108.69792,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use76"
xlink:href="#n"
transform="matrix(0.01736111,0,0,0.01736111,128.47222,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use78"
xlink:href="#p"
transform="matrix(0.01736111,0,0,0.01736111,148.24653,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use80"
xlink:href="#g"
transform="matrix(0.01736111,0,0,0.01736111,168.02083,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use82"
xlink:href="#h"
transform="matrix(0.01736111,0,0,0.01736111,187.79514,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use84"
xlink:href="#h"
transform="matrix(0.01736111,0,0,0.01736111,205.57292,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use86"
xlink:href="#i"
transform="matrix(0.01736111,0,0,0.01736111,223.35069,0)"
x="0"
y="0"
width="100%"
height="100%" />
</g>
<path
id="q"
d="m 571,-1102 c 350,0 480,196 482,560 2,357 -151,562 -488,562 -332,0 -479,-218 -479,-562 0,-373 162,-560 485,-560 z m -8,989 c 244,0 301,-164 301,-429 0,-266 -49,-427 -290,-427 -239,0 -299,165 -299,427 0,252 61,429 288,429"
inkscape:connector-curvature="0" />
<path
id="r"
d="m 663,-1102 c 251,0 343,119 343,381 V 0 H 825 v -686 c 0,-183 -40,-279 -223,-277 -184,2 -280,141 -280,336 V 0 H 142 c -3,-345 6,-754 -6,-1082 h 170 c 5,68 6,94 8,185 h 3 c 76,-134 157,-205 346,-205"
inkscape:connector-curvature="0" />
<path
id="s"
d="m 156,0 v -153 h 359 v -1084 l -318,227 v -170 l 333,-229 h 166 v 1256 h 343 V 0 H 156"
inkscape:connector-curvature="0" />
<path
id="t"
d="m 655,-1102 c 307,0 398,221 398,556 0,377 -133,566 -398,566 -180,0 -272,-66 -339,-188 1,22 -7,151 -10,168 H 132 c 4,-36 6,-110 6,-223 v -1261 h 180 c -2,196 4,384 -4,576 h 4 c 62,-129 158,-194 337,-194 z m -337,573 c 0,254 57,416 289,416 91,0 152,-37 197,-101 76,-109 76,-543 0,-651 -44,-63 -105,-98 -195,-98 -236,0 -291,169 -291,434"
inkscape:connector-curvature="0" />
<path
id="u"
d="m 572,-1430 c 269,0 442,128 442,386 0,144 -72,232 -149,325 -108,130 -487,366 -564,566 h 735 V 0 H 103 v -127 c 119,-285 378,-432 583,-627 76,-72 141,-150 143,-284 1,-156 -100,-244 -257,-244 -156,0 -263,93 -277,238 l -184,-17 c 24,-224 208,-369 461,-369"
inkscape:connector-curvature="0" />
<path
id="v"
d="m 715,-719 c 191,17 334,134 334,330 C 1049,-115 858,20 571,20 288,20 108,-110 78,-362 l 186,-17 c 24,167 126,250 307,250 177,0 294,-88 291,-266 -3,-174 -149,-246 -344,-244 H 416 v -156 h 98 c 179,2 311,-77 311,-243 0,-157 -98,-244 -264,-244 -159,0 -264,88 -278,233 l -181,-14 c 23,-229 206,-367 461,-367 262,0 447,124 447,373 0,197 -120,298 -295,334 v 4"
inkscape:connector-curvature="0" />
<path
id="w"
d="M 1036,-1263 C 892,-1043 790,-871 731,-746 626,-522 556,-302 553,0 H 365 c 0,-180 39,-369 115,-568 76,-199 203,-429 382,-688 H 105 v -153 h 931 v 146"
inkscape:connector-curvature="0" />
<g
id="c">
<use
id="use96"
xlink:href="#h"
transform="scale(0.01302083)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use98"
xlink:href="#q"
transform="matrix(0.01302083,0,0,0.01302083,13.333333,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use100"
xlink:href="#r"
transform="matrix(0.01302083,0,0,0.01302083,28.164062,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use102"
xlink:href="#i"
transform="matrix(0.01302083,0,0,0.01302083,42.994792,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use104"
xlink:href="#p"
transform="matrix(0.01302083,0,0,0.01302083,50.403646,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use106"
xlink:href="#s"
transform="matrix(0.01302083,0,0,0.01302083,65.234375,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use108"
xlink:href="#o"
transform="matrix(0.01302083,0,0,0.01302083,80.065104,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use110"
xlink:href="#o"
transform="matrix(0.01302083,0,0,0.01302083,94.895833,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use112"
xlink:href="#t"
transform="matrix(0.01302083,0,0,0.01302083,109.72656,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use114"
xlink:href="#u"
transform="matrix(0.01302083,0,0,0.01302083,124.55729,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use116"
xlink:href="#v"
transform="matrix(0.01302083,0,0,0.01302083,139.38802,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use118"
xlink:href="#w"
transform="matrix(0.01302083,0,0,0.01302083,154.21875,0)"
x="0"
y="0"
width="100%"
height="100%" />
</g>
<g
id="d">
<use
id="use121"
xlink:href="#h"
transform="scale(0.01302083)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use123"
xlink:href="#q"
transform="matrix(0.01302083,0,0,0.01302083,13.333333,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use125"
xlink:href="#r"
transform="matrix(0.01302083,0,0,0.01302083,28.164062,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use127"
xlink:href="#i"
transform="matrix(0.01302083,0,0,0.01302083,42.994792,0)"
x="0"
y="0"
width="100%"
height="100%" />
</g>
<path
id="x"
d="m 765,-739 c 171,25 285,155 285,346 C 1049,-119 859,20 570,20 286,20 89,-117 89,-391 89,-574 212,-714 370,-737 v -4 c -143,-30 -248,-160 -248,-328 1,-229 198,-361 444,-361 254,0 448,125 449,363 1,166 -104,300 -250,324 z m -197,-70 c 171,-2 261,-74 260,-248 0,-159 -87,-239 -262,-239 -165,0 -260,77 -260,239 0,163 99,249 262,248 z m 4,694 c 200,0 291,-92 291,-295 1,-179 -116,-264 -297,-264 -175,0 -292,98 -291,268 0,194 99,291 297,291"
inkscape:connector-curvature="0" />
<path
id="y"
d="m 492,-1341 c -103,3 -130,49 -131,162 v 97 h 211 v 131 H 361 V 0 H 181 V -951 H 29 v -131 h 152 v -122 c 0,-192 78,-276 264,-278 50,0 92,4 127,12 v 137 c -30,-5 -57,-8 -80,-8"
inkscape:connector-curvature="0" />
<g
id="e">
<use
id="use132"
xlink:href="#h"
transform="scale(0.01302083)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use134"
xlink:href="#q"
transform="matrix(0.01302083,0,0,0.01302083,13.333333,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use136"
xlink:href="#r"
transform="matrix(0.01302083,0,0,0.01302083,28.164062,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use138"
xlink:href="#i"
transform="matrix(0.01302083,0,0,0.01302083,42.994792,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use140"
xlink:href="#p"
transform="matrix(0.01302083,0,0,0.01302083,50.403646,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use142"
xlink:href="#n"
transform="matrix(0.01302083,0,0,0.01302083,65.234375,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use144"
xlink:href="#n"
transform="matrix(0.01302083,0,0,0.01302083,80.065104,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use146"
xlink:href="#x"
transform="matrix(0.01302083,0,0,0.01302083,94.895833,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use148"
xlink:href="#v"
transform="matrix(0.01302083,0,0,0.01302083,109.72656,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use150"
xlink:href="#u"
transform="matrix(0.01302083,0,0,0.01302083,124.55729,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use152"
xlink:href="#x"
transform="matrix(0.01302083,0,0,0.01302083,139.38802,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use154"
xlink:href="#y"
transform="matrix(0.01302083,0,0,0.01302083,154.21875,0)"
x="0"
y="0"
width="100%"
height="100%" />
</g>
</defs>
</defs>
<sodipodi:namedview
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1"
objecttolerance="10"
gridtolerance="10"
guidetolerance="10"
inkscape:pageopacity="0"
inkscape:pageshadow="2"
inkscape:window-width="3840"
inkscape:window-height="2031"
id="namedview163"
showgrid="false"
inkscape:zoom="2.8284271"
inkscape:cx="431.66392"
inkscape:cy="182.41243"
inkscape:window-x="0"
inkscape:window-y="55"
inkscape:window-maximized="1"
inkscape:current-layer="svg161" />
<g
id="g264">
<path
d="m 0.80639,2.9507 h 628.09436 v 115.928 H 0.80639 Z"
id="path2"
inkscape:connector-curvature="0"
style="fill:#ffffff;fill-opacity:0;stroke-width:0.45284379" />
<path
d="m 9.90855,2.9507 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.11121 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 H 196.53 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.54655,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.54655,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.0931 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.11121 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55105,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5556,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5511,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5511,0 h 9.1022 m 4.5556,0 h 9.1021 m 4.5511,0 h 9.1022 m 4.5465,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5466,0 h 9.1021 m 4.5556,0 h 9.1022 m 4.5511,0 h 4.5511 v 4.293 m 0,4.2929 v 8.5905 m 0,4.2929 v 8.5905 m 0,4.2929 v 8.5859 m 0,4.3021 v 8.5859 m 0,4.2929 v 8.5859 m 0,4.3021 v 8.5814 m 0,4.2929 v 8.5859 m 0,4.2885 v 8.5904 m 0,4.293 v 4.2884 h -4.5511 m -4.5511,0 h -9.1022 m -4.551,0 h -9.1022 m -4.5556,0 h -9.1022 m -4.5556,0 h -9.1021 m -4.5511,0 h -9.1022 m -4.5556,0 h -9.1021 m -4.5511,0 h -9.1112 m -4.5511,0 h -9.1022 m -4.5511,0 h -9.1021 m -4.5511,0 h -9.1022 m -4.551,0 h -9.1022 m -4.5556,0 h -9.1022 m -4.5511,0 h -9.1021 m -4.55109,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.09763 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.11121 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55561,0 H 9.90855 m -4.55108,0 H 0.80639 v -4.293 m 0,-4.2929 v -8.5905 m 0,-4.2929 v -8.5905 m 0,-4.2929 v -8.5769 m 0,-4.302 v -8.5859 m 0,-4.293 v -8.5859 m 0,-4.302 v -8.5814 m 0,-4.293 v -8.5904 m 0,-4.2884 v -8.5905 m 0,-4.2929 v -4.293 h 4.55108"
id="path4"
inkscape:connector-curvature="0"
style="fill:none;stroke:#000000;stroke-width:0.90568757" />
<path
d="M 4.88199,2.9507 H 51.97774 V 33.2912 H 4.88199 Z"
id="path6"
inkscape:connector-curvature="0"
style="fill:#ffffff;fill-opacity:0;stroke:#000000;stroke-width:0.90568757;stroke-opacity:0" />
<path
d="m 0.80639,118.8787 h 628.09436 v 115.928 H 0.80639 Z"
id="path10"
inkscape:connector-curvature="0"
style="fill:#d2fff2;stroke-width:0.45284379" />
<path
d="m 9.90855,118.8787 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.11121 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 H 196.53 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.54655,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.54655,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.0931 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.11121 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55105,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5556,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5511,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5511,0 h 9.1022 m 4.5556,0 h 9.1021 m 4.5511,0 h 9.1022 m 4.5465,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5466,0 h 9.1021 m 4.5556,0 h 9.1022 m 4.5511,0 h 4.5511 v 4.293 m 0,4.2929 v 8.5905 m 0,4.2929 v 8.5905 m 0,4.2929 v 8.5859 m 0,4.3021 v 8.5859 m 0,4.2929 v 8.586 m 0,4.302 v 8.5814 m 0,4.2929 v 8.5859 m 0,4.2885 v 8.5904 m 0,4.293 v 4.2884 h -4.5511 m -4.5511,0 h -9.1022 m -4.551,0 h -9.1022 m -4.5556,0 h -9.1022 m -4.5556,0 h -9.1021 m -4.5511,0 h -9.1022 m -4.5556,0 h -9.1021 m -4.5511,0 h -9.1112 m -4.5511,0 h -9.1022 m -4.5511,0 h -9.1021 m -4.5511,0 h -9.1022 m -4.551,0 h -9.1022 m -4.5556,0 h -9.1022 m -4.5511,0 h -9.1021 m -4.55109,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.09763 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.11121 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55561,0 H 9.90855 m -4.55108,0 H 0.80639 v -4.293 m 0,-4.2929 v -8.5905 m 0,-4.2929 v -8.5905 m 0,-4.2929 v -8.5769 m 0,-4.302 v -8.5859 m 0,-4.293 v -8.5859 m 0,-4.302 v -8.5814 m 0,-4.293 v -8.5904 m 0,-4.2884 v -8.5905 m 0,-4.2929 v -4.293 h 4.55108"
id="path12"
inkscape:connector-curvature="0"
style="fill:none;stroke:#000000;stroke-width:0.90568757" />
<path
d="m 4.88199,118.8787 h 117.73938 v 30.3405 H 4.88199 Z"
id="path14"
inkscape:connector-curvature="0"
style="fill:#ffffff;fill-opacity:0;stroke:#000000;stroke-width:0.90568757;stroke-opacity:0" />
<path
d="m 43.49452,161.537 c 0,-2.7368 3.59697,-4.9515 8.042,-4.9515 h 159.37776 c 4.44503,0 8.042,2.2147 8.042,4.9515 v 30.1586 c 0,2.7368 -3.59697,4.9514 -8.042,4.9514 H 51.53652 c -4.44503,0 -8.042,-2.2146 -8.042,-4.9514 z"
id="path18"
inkscape:connector-curvature="0"
style="fill:#ffffff;stroke:#000000;stroke-width:1.1473186" />
<path
d="M 154.20269,158.1403 380.23061,81.7455"
id="path22"
inkscape:connector-curvature="0"
style="fill:none;stroke:#000000;stroke-width:0.90568757" />
<path
style="stroke-width:0.45284379"
d="m 160.45647,151.574 -5.41149,6.2764 8.11496,1.7208 -0.18113,0.8786 -9.61841,-2.0288 6.40774,-7.4447 z m 221.20965,-69.8285 -1.2906,0.4302 -0.29435,-0.8604 0.0725,-0.023 h 1.35853 z"
id="path24"
inkscape:connector-curvature="0" />
<path
d="m 321.87263,45.9708 c 0,-2.7532 2.22799,-4.9812 4.98128,-4.9812 h 228.68614 c 2.7533,0 4.9812,2.228 4.9812,4.9812 v 30.3406 c 0,2.7533 -2.2279,4.9813 -4.9812,4.9813 H 326.85391 c -2.75329,0 -4.98128,-2.228 -4.98128,-4.9813 z"
id="path26"
inkscape:connector-curvature="0"
style="fill:#ffffff;stroke:#000000;stroke-width:0.90568757" />
<path
d="m 239.12085,161.5353 c 0,-2.7371 3.5692,-4.9519 7.97993,-4.9519 H 405.2485 c 4.41075,0 7.97997,2.2148 7.97997,4.9519 v 30.1619 c 0,2.7371 -3.56922,4.952 -7.97997,4.952 H 247.10078 c -4.41073,0 -7.97993,-2.2149 -7.97993,-4.952 z"
id="path30"
inkscape:connector-curvature="0"
style="fill:#ffffff;stroke:#000000;stroke-width:1.14294505" />
<path
d="M 273.62212,155.5772 404.62983,81.7455"
id="path34"
inkscape:connector-curvature="0"
style="fill:none;stroke:#000000;stroke-width:0.90568757" />
<path
style="stroke-width:0.45284379"
d="m 278.54454,147.9513 -4.14805,7.1866 8.28704,0.1721 -0.0136,0.9057 -9.82671,-0.2038 4.91336,-8.5135 z m 127.49363,-66.4775 -1.17739,0.6657 -0.45285,-0.788 0.10869,-0.059 h 1.42193 z"
id="path36"
inkscape:connector-curvature="0" />
<text
id="text277"
y="135.63394"
x="8.0519142"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:14.49100113px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
xml:space="preserve"><tspan
style="stroke-width:0.45284379"
y="135.63394"
x="8.0519142"
id="tspan275"
sodipodi:role="line">/.shards_acct</tspan></text>
<text
id="text281"
y="20.61161"
x="8.9575539"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:14.49100113px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
xml:space="preserve"><tspan
style="stroke-width:0.45284379"
y="20.61161"
x="8.9575539"
id="tspan279"
sodipodi:role="line">/acct</tspan></text>
<text
id="text101"
y="182.27689"
x="61.487461"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:13.2834177px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
xml:space="preserve"><tspan
style="stroke-width:0.45284379"
y="182.27689"
x="61.487461"
id="tspan99"
sodipodi:role="line">cont-568d8e-&lt;ts&gt;-0</tspan></text>
<text
id="text101-7"
y="183.15335"
x="260.96158"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:13.2834177px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
xml:space="preserve"><tspan
style="stroke-width:0.45284379"
y="183.15335"
x="260.96158"
id="tspan99-8"
sodipodi:role="line">cont-750ed3-&lt;ts&gt;-1</tspan></text>
<text
id="text101-6"
y="65.650284"
x="427.37085"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:13.2834177px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
xml:space="preserve"><tspan
style="stroke-width:0.45284379"
y="65.650284"
x="427.37085"
id="tspan99-88"
sodipodi:role="line">cont</tspan></text>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

1502
doc/source/images/sharding_db_states.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 72 KiB

259
doc/source/images/sharding_scan_basic.svg Normal file
View File

@ -0,0 +1,259 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="630.00006"
height="120"
version="1.1"
id="svg54"
sodipodi:docname="sharding_snip2.svg"
inkscape:version="0.92.2 (5c3e80d, 2017-08-06)">
<metadata
id="metadata60">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title />
</cc:Work>
</rdf:RDF>
</metadata>
<defs
id="defs58">
<defs
id="defs50">
<path
id="d"
d="m 779,-765 c -19,-119 -93,-196 -233,-196 -95,0 -164,32 -207,95 -43,63 -64,170 -64,320 0,144 23,251 68,320 45,69 114,104 205,104 139,0 224,-79 240,-212 l 182,12 C 946,-119 781,22 553,20 230,17 87,-201 87,-542 c 0,-338 145,-557 464,-560 225,-1 380,129 413,323"
inkscape:connector-curvature="0" />
<path
id="e"
d="m 571,-1102 c 350,0 480,196 482,560 2,357 -151,562 -488,562 -332,0 -479,-218 -479,-562 0,-373 162,-560 485,-560 z m -8,989 c 244,0 301,-164 301,-429 0,-266 -49,-427 -290,-427 -239,0 -299,165 -299,427 0,252 61,429 288,429"
inkscape:connector-curvature="0" />
<path
id="f"
d="m 663,-1102 c 251,0 343,119 343,381 V 0 H 825 v -686 c 0,-183 -40,-279 -223,-277 -184,2 -280,141 -280,336 V 0 H 142 c -3,-345 6,-754 -6,-1082 h 170 c 5,68 6,94 8,185 h 3 c 76,-134 157,-205 346,-205"
inkscape:connector-curvature="0" />
<path
id="g"
d="m 336,-268 c 2,98 22,139 114,141 24,0 59,-5 104,-14 V -8 C 495,8 434,16 372,16 228,16 156,-66 156,-229 V -951 H 31 v -131 h 132 l 53,-242 h 120 v 242 h 200 v 131 H 336 v 683"
inkscape:connector-curvature="0" />
<g
id="a">
<use
id="use26"
xlink:href="#d"
transform="scale(0.01302083)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use28"
xlink:href="#e"
transform="matrix(0.01302083,0,0,0.01302083,13.333333,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use30"
xlink:href="#f"
transform="matrix(0.01302083,0,0,0.01302083,28.164062,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use32"
xlink:href="#g"
transform="matrix(0.01302083,0,0,0.01302083,42.994792,0)"
x="0"
y="0"
width="100%"
height="100%" />
</g>
<path
id="h"
d="M 0,20 411,-1484 H 569 L 162,20 H 0"
inkscape:connector-curvature="0" />
<path
id="i"
d="M 797,-207 C 713,-60 620,16 414,20 203,24 88,-98 87,-302 c 0,-112 37,-198 111,-258 74,-60 192,-93 356,-96 l 243,-4 c 10,-201 -43,-307 -232,-305 -154,2 -223,43 -242,172 l -188,-17 c 31,-195 175,-292 434,-292 259,0 410,116 410,364 v 466 c 3,95 12,159 101,161 17,0 37,-2 59,-7 V -6 C 1094,5 1047,10 1000,10 857,8 812,-66 803,-207 Z m -525,-92 c -1,116 66,187 183,184 230,-7 361,-164 342,-419 -124,5 -305,-2 -389,30 -83,32 -136,92 -136,205"
inkscape:connector-curvature="0" />
<g
id="b">
<use
id="use37"
xlink:href="#h"
transform="scale(0.01736111)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use39"
xlink:href="#i"
transform="matrix(0.01736111,0,0,0.01736111,9.8784722,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use41"
xlink:href="#d"
transform="matrix(0.01736111,0,0,0.01736111,29.652778,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use43"
xlink:href="#d"
transform="matrix(0.01736111,0,0,0.01736111,47.430556,0)"
x="0"
y="0"
width="100%"
height="100%" />
<use
id="use45"
xlink:href="#g"
transform="matrix(0.01736111,0,0,0.01736111,65.208333,0)"
x="0"
y="0"
width="100%"
height="100%" />
</g>
<path
id="j"
d="m 492,-1341 c -103,3 -130,49 -131,162 v 97 h 211 v 131 H 361 V 0 H 181 V -951 H 29 v -131 h 152 v -122 c 0,-192 78,-276 264,-278 50,0 92,4 127,12 v 137 c -30,-5 -57,-8 -80,-8"
inkscape:connector-curvature="0"
style="fill:#b1001c" />
<use
id="c"
xlink:href="#j"
transform="scale(0.01736111)"
x="0"
y="0"
width="100%"
height="100%" />
</defs>
</defs>
<sodipodi:namedview
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1"
objecttolerance="10"
gridtolerance="10"
guidetolerance="10"
inkscape:pageopacity="0"
inkscape:pageshadow="2"
inkscape:window-width="3840"
inkscape:window-height="2031"
id="namedview56"
showgrid="false"
inkscape:zoom="1.8847584"
inkscape:cx="-83.692254"
inkscape:cy="345.83434"
inkscape:window-x="0"
inkscape:window-y="55"
inkscape:window-maximized="1"
inkscape:current-layer="svg54"
inkscape:pagecheckerboard="true"
fit-margin-top="0"
fit-margin-left="0"
fit-margin-right="0"
fit-margin-bottom="0" />
<path
style="fill:#ffffff;fill-opacity:0;stroke-width:0.45284379"
inkscape:connector-curvature="0"
id="path2"
d="M 1.4528438,1.9 H 629.5472 V 117.8 H 1.4528438 Z" />
<path
style="fill:none;stroke:#000000;stroke-width:0.90568757"
inkscape:connector-curvature="0"
id="path4"
d="m 10.555004,1.9 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.111206 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.54655,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.54655,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.5556,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10213 m 4.5511,0 h 9.1112 m 4.5511,0 h 9.1022 m 4.551,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5511,0 h 9.1022 m 4.5556,0 h 9.1022 m 4.551,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5511,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5556,0 h 9.1022 m 4.5511,0 h 9.1021 m 4.5466,0 h 9.1021 m 4.5511,0 h 9.1022 m 4.5465,0 h 9.1022 m 4.5556,0 h 9.1022 m 4.551,0 h 4.5421 v 4.3 m 0,4.2 V 19 m 0,4.3 v 8.6 m 0,4.3 v 8.6 m 0,4.3 v 8.6 m 0,4.3 v 8.6 m 0,4.3 v 8.6 m 0,4.2 v 8.6 m 0,4.3 v 8.6 m 0,4.3 v 4.3 h -4.5511 m -4.5511,0 h -9.1022 m -4.551,0 h -9.1022 m -4.5556,0 h -9.1022 m -4.5556,0 h -9.1021 m -4.5511,0 h -9.1022 m -4.5556,0 h -9.1021 m -4.5511,0 h -9.1112 m -4.5511,0 h -9.1022 m -4.5511,0 h -9.1021 m -4.5511,0 h -9.1022 m -4.551,0 h -9.1022 m -4.5556,0 H 461.13 m -4.5511,0 h -9.1021 m -4.5511,0 h -9.1022 m -4.551,0 h -9.1022 m -4.5511,0 h -9.1021 m -4.5556,0 h -9.1022 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.0931 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.11121 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.102156 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.5510802,0 h -4.55108 v -4.3 m 0,-4.3 v -8.6 m 0,-4.3 v -8.6 m 0,-4.3 v -8.5 m 0,-4.3 V 62 m 0,-4.3 v -8.6 m 0,-4.3 v -8.6 m 0,-4.3 v -8.6 m 0,-4.3 v -8.6 m 0,-4.2 V 1.9 h 4.55108" />
<path
style="fill:#ffffff;stroke:#000000;stroke-width:0.90568757"
inkscape:connector-curvature="0"
id="path6"
d="m 82.511884,44.4 c 0,-2.7 2.22799,-5 4.98128,-5 H 543.5068 c 2.7533,0 4.9813,2.3 4.9813,5 v 30.4 c 0,2.7 -2.228,5 -4.9813,5 H 87.493164 c -2.75329,0 -4.98128,-2.3 -4.98128,-5 z" />
<path
style="fill:#ffffff;fill-opacity:0;stroke:#000000;stroke-width:0.90568757;stroke-opacity:0"
inkscape:connector-curvature="0"
id="path10"
d="M 5.5284438,1.9 H 52.624194 V 32.2 H 5.5284438 Z" />
<path
style="fill:none;stroke-width:0.45284379"
inkscape:connector-curvature="0"
id="path14"
d="M 169.87329,39.4 V 79.8" />
<path
style="fill:#ffffff;fill-opacity:0;stroke:#000000;stroke-width:0.90568757;stroke-opacity:0"
inkscape:connector-curvature="0"
id="path18"
d="m 172.62779,87.4 h 14.94384 v 30.4 h -14.94384 z" />
<text
xml:space="preserve"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:14.49100113px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
x="9.9593773"
y="19.10981"
id="text281"><tspan
sodipodi:role="line"
id="tspan279"
x="9.9593773"
y="19.10981"
style="stroke-width:0.45284379">/acct</tspan></text>
<text
xml:space="preserve"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:14.49100113px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
x="299.41342"
y="64.584816"
id="text281-9"><tspan
sodipodi:role="line"
id="tspan279-1"
x="299.41342"
y="64.584816"
style="stroke-width:0.45284379">cont</tspan></text>
<path
style="fill:#800000;fill-opacity:1;stroke:#aa0000;stroke-width:0.90600002;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:7.24800014, 3.62400007;stroke-dashoffset:0;stroke-opacity:1"
d="m 169.27742,39.4 c 0,39.730565 0.0442,39.597985 0.0442,39.597985 v 0 0 0"
id="path4476-4"
inkscape:connector-curvature="0" />
<path
style="fill:#aa0000;fill-opacity:1;stroke:#aa0000;stroke-width:0.90600002;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:7.24800014, 3.62400007;stroke-dashoffset:0;stroke-opacity:1"
d="m 252.83355,39.4281 c 0,39.730565 0.0442,39.597985 0.0442,39.597985 v 0 0 0"
id="path4476-6"
inkscape:connector-curvature="0" />
<text
xml:space="preserve"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:14.49100113px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#aa0000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
x="157.79762"
y="110.58204"
id="text281-0"><tspan
sodipodi:role="line"
id="tspan279-4"
x="157.79762"
y="110.58204"
style="fill:#aa0000;stroke-width:0.45284379">cat</tspan></text>
<text
xml:space="preserve"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:14.49100113px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#aa0000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
x="229.34985"
y="109.59498"
id="text281-0-2"><tspan
sodipodi:role="line"
id="tspan279-4-7"
x="229.34985"
y="109.59498"
style="fill:#aa0000;stroke-width:0.45284379">giraffe</tspan></text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

1665
doc/source/images/sharding_scan_load.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 66 KiB

1650
doc/source/images/sharding_sharded_load.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 65 KiB

199
doc/source/images/sharding_unsharded.svg Normal file
View File

@ -0,0 +1,199 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="630"
height="120"
version="1.1"
id="svg3952"
sodipodi:docname="sharding_snip1.svg"
inkscape:version="0.92.2 (5c3e80d, 2017-08-06)">
<metadata
id="metadata3958">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title />
</cc:Work>
</rdf:RDF>
</metadata>
<defs
id="defs3956">
<defs
id="defs3948">
<path
inkscape:connector-curvature="0"
id="c"
d="m 779,-765 c -19,-119 -93,-196 -233,-196 -95,0 -164,32 -207,95 -43,63 -64,170 -64,320 0,144 23,251 68,320 45,69 114,104 205,104 139,0 224,-79 240,-212 l 182,12 C 946,-119 781,22 553,20 230,17 87,-201 87,-542 c 0,-338 145,-557 464,-560 225,-1 380,129 413,323" />
<path
inkscape:connector-curvature="0"
id="d"
d="m 571,-1102 c 350,0 480,196 482,560 2,357 -151,562 -488,562 -332,0 -479,-218 -479,-562 0,-373 162,-560 485,-560 z m -8,989 c 244,0 301,-164 301,-429 0,-266 -49,-427 -290,-427 -239,0 -299,165 -299,427 0,252 61,429 288,429" />
<path
inkscape:connector-curvature="0"
id="e"
d="m 663,-1102 c 251,0 343,119 343,381 V 0 H 825 v -686 c 0,-183 -40,-279 -223,-277 -184,2 -280,141 -280,336 V 0 H 142 c -3,-345 6,-754 -6,-1082 h 170 c 5,68 6,94 8,185 h 3 c 76,-134 157,-205 346,-205" />
<path
inkscape:connector-curvature="0"
id="f"
d="m 336,-268 c 2,98 22,139 114,141 24,0 59,-5 104,-14 V -8 C 495,8 434,16 372,16 228,16 156,-66 156,-229 V -951 H 31 v -131 h 132 l 53,-242 h 120 v 242 h 200 v 131 H 336 v 683" />
<g
id="a">
<use
height="100%"
width="100%"
y="0"
x="0"
id="use3926"
xlink:href="#c"
transform="scale(0.01302083)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use3928"
xlink:href="#d"
transform="matrix(0.01302083,0,0,0.01302083,13.333333,0)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use3930"
xlink:href="#e"
transform="matrix(0.01302083,0,0,0.01302083,28.164062,0)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use3932"
xlink:href="#f"
transform="matrix(0.01302083,0,0,0.01302083,42.994792,0)" />
</g>
<path
inkscape:connector-curvature="0"
id="g"
d="M 0,20 411,-1484 H 569 L 162,20 H 0" />
<path
inkscape:connector-curvature="0"
id="h"
d="M 797,-207 C 713,-60 620,16 414,20 203,24 88,-98 87,-302 c 0,-112 37,-198 111,-258 74,-60 192,-93 356,-96 l 243,-4 c 10,-201 -43,-307 -232,-305 -154,2 -223,43 -242,172 l -188,-17 c 31,-195 175,-292 434,-292 259,0 410,116 410,364 v 466 c 3,95 12,159 101,161 17,0 37,-2 59,-7 V -6 C 1094,5 1047,10 1000,10 857,8 812,-66 803,-207 Z m -525,-92 c -1,116 66,187 183,184 230,-7 361,-164 342,-419 -124,5 -305,-2 -389,30 -83,32 -136,92 -136,205" />
<g
id="b">
<use
height="100%"
width="100%"
y="0"
x="0"
id="use3937"
xlink:href="#g"
transform="scale(0.01736111)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use3939"
xlink:href="#h"
transform="matrix(0.01736111,0,0,0.01736111,9.8784722,0)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use3941"
xlink:href="#c"
transform="matrix(0.01736111,0,0,0.01736111,29.652778,0)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use3943"
xlink:href="#c"
transform="matrix(0.01736111,0,0,0.01736111,47.430556,0)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use3945"
xlink:href="#f"
transform="matrix(0.01736111,0,0,0.01736111,65.208333,0)" />
</g>
</defs>
</defs>
<sodipodi:namedview
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1"
objecttolerance="10"
gridtolerance="10"
guidetolerance="10"
inkscape:pageopacity="0"
inkscape:pageshadow="2"
inkscape:window-width="3840"
inkscape:window-height="2031"
id="namedview3954"
showgrid="false"
inkscape:zoom="4"
inkscape:cx="259.51356"
inkscape:cy="162.22523"
inkscape:window-x="0"
inkscape:window-y="55"
inkscape:window-maximized="1"
inkscape:current-layer="svg3952" />
<path
style="fill:#ffffff;fill-opacity:0;stroke-width:0.45284376"
inkscape:connector-curvature="0"
id="path3910"
d="M 0.95284148,2.5359665 H 629.04715 V 118.46397 H 0.95284148 Z" />
<path
style="fill:none;stroke:#000000;stroke-width:0.90568751"
inkscape:connector-curvature="0"
id="path3912"
d="m 10.055001,2.5359665 h 9.10216 m 4.55108,0 h 9.10216 m 4.555608,0 h 9.10216 m 4.555608,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.555608,0 h 9.10216 m 4.55108,0 h 9.111215 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.5556,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.54656,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.54655,0 h 9.10216 m 4.5556,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.11121 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.54655,0 h 9.10216 m 4.55108,0 h 9.10216 m 4.54655,0 h 9.10216 m 4.55561,0 h 9.10216 m 4.55108,0 h 4.54202 v 4.2929589 m 0,4.2929586 v 8.590447 m 0,4.292959 v 8.590446 m 0,4.292959 v 8.585918 m 0,4.302016 v 8.585917 m 0,4.292959 v 8.585918 m 0,4.302016 v 8.58139 m 0,4.292958 v 8.585918 m 0,4.288435 v 8.59044 m 0,4.29296 v 4.28843 h -4.55108 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55561,0 h -9.10215 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.11122 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55107,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.54655,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.0931 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.11122 m -4.55108,0 h -9.10215 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55561,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.55108,0 h -9.102161 m -4.555608,0 h -9.10216 m -4.55108,0 h -9.10216 m -4.546551,0 h -9.10216 m -4.55108,0 h -9.102159 m -4.546552,0 h -9.10216 m -4.555608,0 h -9.10216 m -4.5510796,0 H 0.95284148 v -4.29296 m 0,-4.29296 v -8.59044 m 0,-4.292962 v -8.590446 m 0,-4.292959 v -8.576861 m 0,-4.302016 v -8.585918 m 0,-4.292958 v -8.585918 m 0,-4.302016 v -8.58139 m 0,-4.292959 v -8.590446 m 0,-4.28843 v -8.590447 m 0,-4.2929586 V 2.5359665 H 5.5039214" />
<path
style="fill:#ffffff;stroke:#000000;stroke-width:0.90568751"
inkscape:connector-curvature="0"
id="path3914"
d="m 82.011876,45.103281 c 0,-2.75329 2.227992,-4.981282 4.981282,-4.981282 H 543.00683 c 2.75329,0 4.98129,2.227992 4.98129,4.981282 v 30.340532 c 0,2.753291 -2.228,4.981282 -4.98129,4.981282 H 86.993158 c -2.75329,0 -4.981282,-2.227991 -4.981282,-4.981282 z" />
<path
style="fill:#ffffff;fill-opacity:0;stroke:#000000;stroke-width:0.90568751;stroke-opacity:0"
inkscape:connector-curvature="0"
id="path3918"
d="M 5.0284354,2.5359665 H 52.124187 V 32.876499 H 5.0284354 Z" />
<text
xml:space="preserve"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:14.49100113px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
x="6.4593792"
y="19.109808"
id="text281"><tspan
sodipodi:role="line"
id="tspan279"
x="6.4593792"
y="19.109808"
style="stroke-width:0.45284379">/acct</tspan></text>
<text
xml:space="preserve"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:14.49100113px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
x="298.91342"
y="65.258369"
id="text281-8"><tspan
sodipodi:role="line"
id="tspan279-9"
x="298.91342"
y="65.258369"
style="stroke-width:0.45284379">cont</tspan></text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 10 KiB

219
doc/source/images/sharding_unsharded_load.svg Normal file
View File

@ -0,0 +1,219 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="642"
height="139"
version="1.1"
id="svg2012"
sodipodi:docname="sharding_lock1.svg"
inkscape:version="0.92.2 (5c3e80d, 2017-08-06)">
<metadata
id="metadata2018">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title></dc:title>
</cc:Work>
</rdf:RDF>
</metadata>
<defs
id="defs2016">
<defs
id="defs2008">
<path
inkscape:connector-curvature="0"
id="c"
d="m 779,-765 c -19,-119 -93,-196 -233,-196 -95,0 -164,32 -207,95 -43,63 -64,170 -64,320 0,144 23,251 68,320 45,69 114,104 205,104 139,0 224,-79 240,-212 l 182,12 C 946,-119 781,22 553,20 230,17 87,-201 87,-542 c 0,-338 145,-557 464,-560 225,-1 380,129 413,323" />
<path
inkscape:connector-curvature="0"
id="d"
d="m 571,-1102 c 350,0 480,196 482,560 2,357 -151,562 -488,562 -332,0 -479,-218 -479,-562 0,-373 162,-560 485,-560 z m -8,989 c 244,0 301,-164 301,-429 0,-266 -49,-427 -290,-427 -239,0 -299,165 -299,427 0,252 61,429 288,429" />
<path
inkscape:connector-curvature="0"
id="e"
d="m 663,-1102 c 251,0 343,119 343,381 V 0 H 825 v -686 c 0,-183 -40,-279 -223,-277 -184,2 -280,141 -280,336 V 0 H 142 c -3,-345 6,-754 -6,-1082 h 170 c 5,68 6,94 8,185 h 3 c 76,-134 157,-205 346,-205" />
<path
inkscape:connector-curvature="0"
id="f"
d="m 336,-268 c 2,98 22,139 114,141 24,0 59,-5 104,-14 V -8 C 495,8 434,16 372,16 228,16 156,-66 156,-229 V -951 H 31 v -131 h 132 l 53,-242 h 120 v 242 h 200 v 131 H 336 v 683" />
<g
id="a">
<use
height="100%"
width="100%"
y="0"
x="0"
id="use1986"
xlink:href="#c"
transform="scale(0.01302083)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use1988"
xlink:href="#d"
transform="matrix(0.01302083,0,0,0.01302083,13.333333,0)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use1990"
xlink:href="#e"
transform="matrix(0.01302083,0,0,0.01302083,28.164062,0)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use1992"
xlink:href="#f"
transform="matrix(0.01302083,0,0,0.01302083,42.994792,0)" />
</g>
<path
inkscape:connector-curvature="0"
id="g"
d="M 0,20 411,-1484 H 569 L 162,20 H 0" />
<path
inkscape:connector-curvature="0"
id="h"
d="M 797,-207 C 713,-60 620,16 414,20 203,24 88,-98 87,-302 c 0,-112 37,-198 111,-258 74,-60 192,-93 356,-96 l 243,-4 c 10,-201 -43,-307 -232,-305 -154,2 -223,43 -242,172 l -188,-17 c 31,-195 175,-292 434,-292 259,0 410,116 410,364 v 466 c 3,95 12,159 101,161 17,0 37,-2 59,-7 V -6 C 1094,5 1047,10 1000,10 857,8 812,-66 803,-207 Z m -525,-92 c -1,116 66,187 183,184 230,-7 361,-164 342,-419 -124,5 -305,-2 -389,30 -83,32 -136,92 -136,205" />
<g
id="b">
<use
height="100%"
width="100%"
y="0"
x="0"
id="use1997"
xlink:href="#g"
transform="scale(0.01736111)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use1999"
xlink:href="#h"
transform="matrix(0.01736111,0,0,0.01736111,9.8784722,0)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use2001"
xlink:href="#c"
transform="matrix(0.01736111,0,0,0.01736111,29.652778,0)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use2003"
xlink:href="#c"
transform="matrix(0.01736111,0,0,0.01736111,47.430556,0)" />
<use
height="100%"
width="100%"
y="0"
x="0"
id="use2005"
xlink:href="#f"
transform="matrix(0.01736111,0,0,0.01736111,65.208333,0)" />
</g>
</defs>
</defs>
<sodipodi:namedview
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1"
objecttolerance="10"
gridtolerance="10"
guidetolerance="10"
inkscape:pageopacity="0"
inkscape:pageshadow="2"
inkscape:window-width="3522"
inkscape:window-height="1971"
id="namedview2014"
showgrid="false"
inkscape:zoom="2.8284271"
inkscape:cx="450.01007"
inkscape:cy="76.915323"
inkscape:window-x="0"
inkscape:window-y="55"
inkscape:window-maximized="0"
inkscape:current-layer="svg2012" />
<path
style="fill:#ffffff;fill-opacity:0;stroke-width:0.46028754"
inkscape:connector-curvature="0"
id="path1962"
d="M 2.540593,20.136033 H 640.95941 V 137.96964 H 2.540593 Z" />
<path
style="fill:none;stroke:#000000;stroke-width:0.92057508"
inkscape:connector-curvature="0"
id="path1964"
d="m 11.792373,20.136033 h 9.251779 m 4.62589,0 h 9.251779 m 4.630493,0 h 9.25178 m 4.630492,0 h 9.25178 m 4.62589,0 h 9.251779 m 4.630493,0 h 9.251779 m 4.62589,0 h 9.260983 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.63049,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.63049,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62129,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62128,0 h 9.25178 m 4.6305,0 h 9.25178 m 4.62589,0 h 9.25177 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.6305,0 h 9.25178 m 4.63049,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.63049,0 h 9.25178 m 4.62589,0 h 9.26098 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.63049,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.63049,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62129,0 h 9.25178 m 4.62589,0 h 9.25178 m 4.62129,0 h 9.25177 m 4.6305,0 h 9.25178 m 4.62589,0 h 4.61668 v 4.363526 m 0,4.363526 v 8.731654 m 0,4.363526 v 8.731655 m 0,4.363526 v 8.727051 m 0,4.372732 v 8.727052 m 0,4.363526 v 8.727051 m 0,4.372732 v 8.72245 m 0,4.36352 v 8.72706 m 0,4.35892 v 8.73165 m 0,4.36353 v 4.35892 h -4.62589 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.63049,0 h -9.25178 m -4.63049,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.63049,0 h -9.25178 m -4.62589,0 h -9.26099 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.63049,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.63049,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62128,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62129,0 h -9.25178 m -4.63049,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.63049,0 h -9.24258 m -4.63049,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.63049,0 h -9.25178 m -4.62589,0 h -9.26098 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.63049,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.25178 m -4.62589,0 h -9.251781 m -4.630493,0 h -9.25178 m -4.625889,0 h -9.25178 m -4.621287,0 h -9.251779 m -4.62589,0 h -9.25178 m -4.621287,0 h -9.251779 m -4.630493,0 h -9.251779 m -4.6258903,0 H 2.540593 v -4.36352 m 0,-4.36353 v -8.73165 m 0,-4.36353 v -8.73165 m 0,-4.36353 v -8.717846 m 0,-4.372731 v -8.727052 m 0,-4.363526 v -8.727052 m 0,-4.372731 v -8.722449 m 0,-4.363526 v -8.731655 m 0,-4.358923 v -8.731654 m 0,-4.363526 v -4.363526 h 4.6258897" />
<path
style="fill:#ffffff;stroke:#000000;stroke-width:0.92057508"
inkscape:connector-curvature="0"
id="path1966"
d="m 84.932063,63.403062 c 0,-2.798549 2.264614,-5.063163 5.063163,-5.063163 H 553.50478 c 2.79855,0 5.06316,2.264614 5.06316,5.063163 v 30.839265 c 0,2.798548 -2.26461,5.063163 -5.06316,5.063163 H 89.995226 c -2.798549,0 -5.063163,-2.264615 -5.063163,-5.063163 z" />
<path
style="fill:#ffffff;fill-opacity:0;stroke:#000000;stroke-width:0.92057508;stroke-opacity:0"
inkscape:connector-curvature="0"
id="path1970"
d="M 6.6831808,20.136033 H 54.553085 V 50.975298 H 6.6831808 Z" />
<path
style="fill:none;stroke:#000000;stroke-width:0.92057508"
inkscape:connector-curvature="0"
id="path1974"
d="M 322.70095,58.339899 275.53989,1.7245313" />
<path
inkscape:connector-curvature="0"
id="path1976"
d="m 323.29933,58.339899 h -0.59838 l 0.34982,-0.294584 z"
style="stroke-width:0.46028754" />
<path
style="fill:none;stroke:#000000;stroke-width:0.92057508"
inkscape:connector-curvature="0"
id="path1978"
d="M 320.18778,59.260474 362.69073,8.1777626" />
<path
style="stroke:#000000;stroke-width:0.92057508"
inkscape:connector-curvature="0"
id="path1980"
d="m 367.10489,2.8752502 -2.5592,6.4072025 -3.28185,-2.7295051 z" />
<text
xml:space="preserve"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:13.2834177px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
x="307.00397"
y="83.392113"
id="text101-6-4"><tspan
sodipodi:role="line"
id="tspan99-88-4"
x="307.00397"
y="83.392113"
style="stroke-width:0.45284379">cont</tspan></text>
<text
xml:space="preserve"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:14.49100113px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.45284379"
x="10.03654"
y="38.250904"
id="text281"><tspan
sodipodi:role="line"
id="tspan279"
x="10.03654"
y="38.250904"
style="stroke-width:0.45284379">/acct</tspan></text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

1
doc/source/index.rst
View File

@ -62,6 +62,7 @@ Overview and Concepts
overview_erasure_code overview_erasure_code
overview_encryption overview_encryption
overview_backing_store overview_backing_store
overview_container_sharding
ring_background ring_background
ring_partpower ring_partpower
associated_projects associated_projects

1
doc/source/logs.rst
View File

@ -105,6 +105,7 @@ RL :ref:`ratelimit`
VW :ref:`versioned_writes` VW :ref:`versioned_writes`
SSC :ref:`copy` SSC :ref:`copy`
SYM :ref:`symlink` SYM :ref:`symlink`
SH :ref:`sharding_doc`
======================= ============================= ======================= =============================

2
doc/source/overview_architecture.rst
View File

@ -172,6 +172,8 @@ replicator for Replication type policies. See :doc:`overview_erasure_code`
for complete information on both Erasure Code support as well as the for complete information on both Erasure Code support as well as the
reconstructor. reconstructor.
.. _architecture_updaters:
-------- --------
Updaters Updaters
-------- --------

784
doc/source/overview_container_sharding.rst Normal file
View File

@ -0,0 +1,784 @@
.. _sharding_doc:
==================
Container Sharding
==================
Container sharding is an operator controlled feature that may be used to shard
very large container databases into a number of smaller shard containers
.. note::
Container sharding is currently an experimental feature. It is strongly
recommended that operators gain experience of sharding containers in a
non-production cluster before using in production.
The sharding process involves moving all sharding container database
records via the container replication engine; the time taken to complete
sharding is dependent upon the existing cluster load and the performance of
the container database being sharded.
There is currently no documented process for reversing the sharding
process once sharding has been enabled.
----------
Background
----------
The metadata for each container in Swift is stored in an SQLite database. This
metadata includes: information about the container such as its name,
modification time and current object count; user metadata that may been written
to the container by clients; a record of every object in the container. The
container database object records are used to generate container listings in
response to container GET requests; each object record stores the object's
name, size, hash and content-type as well as associated timestamps.
As the number of objects in a container increases then the number of object
records in the container database increases. Eventually the container database
performance starts to degrade and the time taken to update an object record
increases. This can result in object updates timing out, with a corresponding
increase in the backlog of pending :ref:`asynchronous updates
<architecture_updaters>` on object servers. Container databases are typically
replicated on several nodes and any database performance degradation can also
result in longer :doc:`container replication <overview_replication>` times.
The point at which container database performance starts to degrade depends
upon the choice of hardware in the container ring. Anecdotal evidence suggests
that containers with tens of millions of object records have noticeably
degraded performance.
This performance degradation can be avoided by ensuring that clients use an
object naming scheme that disperses objects across a number of containers
thereby distributing load across a number of container databases. However, that
is not always desirable nor is it under the control of the cluster operator.
Swift's container sharding feature provides the operator with a mechanism to
distribute the load on a single client-visible container across multiple,
hidden, shard containers, each of which stores a subset of the container's
object records. Clients are unaware of container sharding; clients continue to
use the same API to access a container that, if sharded, maps to a number of
shard containers within the Swift cluster.
------------------------
Deployment and operation
------------------------
Upgrade Considerations
----------------------
It is essential that all servers in a Swift cluster have been upgraded to
support the container sharding feature before attempting to shard a container.
Identifying containers in need of sharding
------------------------------------------
Container sharding is currently initiated by the ``swift-manage-shard-ranges``
CLI tool :ref:`described below <swift-manage-shard-ranges>`. Operators must
first identify containers that are candidates for sharding. To assist with
this, the :ref:`sharder_daemon` inspects the size of containers that it visits
and writes a list of sharding candidates to recon cache. For example::
"sharding_candidates": {
"found": 1,
"top": [
{
"account": "AUTH_test",
"container": "c1",
"file_size": 497763328,
"meta_timestamp": "1525346445.31161",
"node_index": 2,
"object_count": 3349028,
"path": <path_to_db>,
"root": "AUTH_test/c1"
}
]
}
A container is considered to be a sharding candidate if its object count is
greater than or equal to the ``shard_container_threshold`` option.
The number of candidates reported is limited to a number configured by the
``recon_candidates_limit`` option such that only the largest candidate
containers are included in the ``sharding_candidate`` data.
.. _swift-manage-shard-ranges:
``swift-manage-shard-ranges`` CLI tool
--------------------------------------
The ``swift-manage-shard-ranges`` tool provides commands for initiating
sharding of a container. ``swift-manage-shard-ranges`` operates directly on a
container database file.
.. note::
``swift-manage-shard-ranges`` must only be used on one replica of a
container database to avoid inconsistent results. The modifications made by
``swift-manage-shard-ranges`` will be automatically copied to other
replicas of the container database via normal replication processes.
There are three steps in the process of initiating sharding, each of which may
be performed in isolation or, as shown below, using a single command.
#. The ``find`` sub-command scans the container database to identify how many
shard containers will be required and which objects they will manage. Each
shard container manages a range of the object namespace defined by a
``lower`` and ``upper`` bound. The maximum number of objects to be allocated
to each shard container is specified on the command line. For example::
$ swift-manage-shard-ranges <path_to_db> find 500000
Loaded db broker for AUTH_test/c1.
[
{
"index": 0,
"lower": "",
"object_count": 500000,
"upper": "o_01086834"
},
{
"index": 1,
"lower": "o_01086834",
"object_count": 500000,
"upper": "o_01586834"
},
{
"index": 2,
"lower": "o_01586834",
"object_count": 500000,
"upper": "o_02087570"
},
{
"index": 3,
"lower": "o_02087570",
"object_count": 500000,
"upper": "o_02587572"
},
{
"index": 4,
"lower": "o_02587572",
"object_count": 500000,
"upper": "o_03087572"
},
{
"index": 5,
"lower": "o_03087572",
"object_count": 500000,
"upper": "o_03587572"
},
{
"index": 6,
"lower": "o_03587572",
"object_count": 349194,
"upper": ""
}
]
Found 7 ranges in 4.37222s (total object count 3349194)
This command returns a list of shard ranges each of which describes the
namespace to be managed by a shard container. No other action is taken by
this command and the container database is unchanged. The output may be
redirected to a file for subsequent retrieval by the ``replace`` command.
For example::
$ swift-manage-shard-ranges <path_to_db> find 500000 > my_shard_ranges
Loaded db broker for AUTH_test/c1.
Found 7 ranges in 2.448s (total object count 3349194)
#. The ``replace`` sub-command deletes any shard ranges that might already be
in the container database and inserts shard ranges from a given file. The
file contents should be in the format generated by the ``find`` sub-command.
For example::
$ swift-manage-shard-ranges <path_to_db> replace my_shard_ranges
Loaded db broker for AUTH_test/c1.
No shard ranges found to delete.
Injected 7 shard ranges.
Run container-replicator to replicate them to other nodes.
Use the enable sub-command to enable sharding.
The container database is modified to store the shard ranges, but the
container will not start sharding until sharding is enabled. The ``info``
sub-command may be used to inspect the state of the container database at
any point, and the ``show`` sub-command may be used to display the inserted
shard ranges.
Shard ranges stored in the container database may be replaced using the
``replace`` sub-command. This will first delete all existing shard ranges
before storing new shard ranges. Shard ranges may also be deleted from the
container database using the ``delete`` sub-command.
Shard ranges should not be replaced or deleted using
``swift-manage-shard-ranges`` once the next step of enabling sharding has
been taken.
#. The ``enable`` sub-command enables the container for sharding. The sharder
daemon and/or container replicator daemon will replicate shard ranges to
other replicas of the container db and the sharder daemon will proceed to
shard the container. This process may take some time depending on the size
of the container, the number of shard ranges and the underlying hardware.
.. note::
Once the ``enable`` sub-command has been used there is no supported
mechanism to revert sharding. Do not use ``swift-manage-shard-ranges`` to
make any further changes to the shard ranges in the container db.
For example::
$ swift-manage-shard-ranges <path_to_db> enable
Loaded db broker for AUTH_test/c1.
Container moved to state 'sharding' with epoch 1525345093.22908.
Run container-sharder on all nodes to shard the container.
This does not shard the container - sharding is performed by the
:ref:`sharder_daemon` - but sets the necessary state in the database for the
daemon to subsequently start the sharding process.
The ``epoch`` value displayed in the output is the time at which sharding
was enabled. When the :ref:`sharder_daemon` starts sharding this container
it creates a new container database file using the epoch in the filename to
distinguish it from the retiring DB that is being sharded.
All three steps may be performed with one sub-command::
$ swift-manage-shard-ranges <path_to_db> find_and_replace 500000 --enable --force
Loaded db broker for AUTH_test/c1.
No shard ranges found to delete.
Injected 7 shard ranges.
Run container-replicator to replicate them to other nodes.
Container moved to state 'sharding' with epoch 1525345669.46153.
Run container-sharder on all nodes to shard the container.
.. _sharder_daemon:
``container-sharder`` daemon
----------------------------
Once sharding has been enabled for a container, the act of sharding is
performed by the :ref:`container-sharder`. The :ref:`container-sharder` daemon
must be running on all container servers. The ``container-sharder`` daemon
periodically visits each container database to perform any container sharding
tasks that are required.
The ``container-sharder`` daemon requires a ``[container-sharder]`` config
section to exist in the container server configuration file; a sample config
section is shown in the `container-server.conf-sample` file.
.. note::
Several of the ``[container-sharder]`` config options are only significant
when the ``auto_shard`` option is enabled. This option enables the
``container-sharder`` daemon to automatically identify containers that are
candidates for sharding and initiate the sharding process, instead of using
the ``swift-manage-shard-ranges`` tool. The ``auto_shard`` option is
currently NOT recommended for production systems and shoud be set to
``false`` (the default value).
The container sharder uses an internal client and therefore requires an
internal client configuration file to exist. By default the internal-client
configuration file is expected to be found at
`/etc/swift/internal-client.conf`. An alternative location for the
configuration file may be specified using the ``internal_client_conf_path``
option in the ``[container-sharder]`` config section.
The content of the internal-client configuration file should be the same as the
`internal-client.conf-sample` file. In particular, the internal-client
configuration should have::
account_autocreate = True
in the ``[proxy-server]`` section.
A container database may require several visits by the ``container-sharder``
daemon before it is fully sharded. On each visit the ``container-sharder``
daemon will move a subset of object records to new shard containers by cleaving
new shard container databases from the original. By default, two shards are
processed per visit; this number may be configured by the ``cleave_batch_size``
option.
The ``container-sharder`` daemon periodically writes progress data for
containers that are being sharded to recon cache. For example::
"sharding_in_progress": {
"all": [
{
"account": "AUTH_test",
"active": 0,
"cleaved": 2,
"container": "c1",
"created": 5,
"db_state": "sharding",
"error": null,
"file_size": 26624,
"found": 0,
"meta_timestamp": "1525349617.46235",
"node_index": 1,
"object_count": 3349030,
"path": <path_to_db>,
"root": "AUTH_test/c1",
"state": "sharding"
}
]
}
This example indicates that from a total of 7 shard ranges, 2 have been cleaved
whereas 5 remain in created state waiting to be cleaved.
Shard containers are created in an internal account and not visible to clients.
By default, shard containers for an account ``AUTH_test`` are created in the
internal account ``.shards_AUTH_test``.
Once a container has started sharding, object updates to that container may be
redirected to the shard container. The ``container-sharder`` daemon is also
responsible for sending updates of a shard's object count and bytes_used to the
original container so that aggegrate object count and bytes used values can be
returned in responses to client requests.
.. note::
The ``container-sharder`` daemon must continue to run on all container
servers in order for shards object stats updates to be generated.
--------------
Under the hood
--------------
Terminology
-----------
================== ==================================================
Name Description
================== ==================================================
Root container The original container that lives in the
user's account. It holds references to its
shard containers.
Retiring DB The original database file that is to be sharded.
Fresh DB A database file that will replace the retiring
database.
Shard range A range of the object namespace defined by a lower
bound and and upper bound.
Shard container A container that holds object records for a shard
range. Shard containers exist a hidden account
mirroring the user's account.
Misplaced objects Items that don't belong in a container's shard
range. These will be moved to their correct
location by the container-sharder.
Cleaving The act of moving object records within a shard
range to a shard container database.
Shrinking The act of merging a small shard container into
another shard container in order to delete the
small shard container.
Donor The shard range that is shrinking away.
Acceptor The shard range into which a donor is merged.
================== ==================================================
Finding shard ranges
--------------------
The end goal of sharding a container is to replace the original container
database which has grown very large with a number of shard container databases,
each of which is responsible for storing a range of the entire object
namespace. The first step towards achieving this is to identify an appropriate
set of contiguous object namespaces, known as shard ranges, each of which
contains a similar sized portion of the container's current object content.
Shard ranges cannot simply be selected by sharding the namespace uniformly,
because object names are not guaranteed to be distributed uniformly. If the
container were naively sharded into two shard ranges, one containing all
object names up to `m` and the other containing all object names beyond `m`,
then if all object names actually start with `o` the outcome would be an
extremely unbalanced pair of shard containers.
It is also too simplistic to assume that every container that requires sharding
can be sharded into two. This might be the goal in the ideal world, but in
practice there will be containers that have grown very large and should be
sharded into many shards. Furthermore, the time required to find the exact
mid-point of the existing object names in a large SQLite database would
increase with container size.
For these reasons, shard ranges of size `N` are found by searching for the
`Nth` object in the database table, sorted by object name, and then searching
for the `(2 * N)th` object, and so on until all objects have been searched. For
a container that has exactly `2N` objects, the end result is the same as
sharding the container at the midpoint of its object names. In practice
sharding would typically be enabled for containers with great than `2N` objects
and more than two shard ranges will be found, the last one probably containing
less than `N` objects. With containers having large multiples of `N` objects,
shard ranges can be identified in batches which enables more scalable solution.
To illustrate this process, consider a very large container in a user account
``acct`` that is a candidate for sharding:
.. image:: images/sharding_unsharded.svg
The :ref:`swift-manage-shard-ranges` tool ``find`` sub-command searches the
object table for the `Nth` object whose name will become the upper bound of the
first shard range, and the lower bound of the second shard range. The lower
bound of the first shard range is the empty string.
For the purposes of this example the first upper bound is `cat`:
.. image:: images/sharding_scan_basic.svg
:ref:`swift-manage-shard-ranges` continues to search the container to find
further shard ranges, with the final upper bound also being the empty string.
Enabling sharding
-----------------
Once shard ranges have been found the :ref:`swift-manage-shard-ranges`
``replace`` sub-command is used to insert them into the `shard_ranges` table
of the container database. In addition to its lower and upper bounds, each
shard range is given a name. The name takes the form ``a/c`` where ``a`` is an
account name formed by prefixing the user account with the string
``.shards_``, and ``c`` is a container name that is derived from the original
container and includes the index of the shard range. The final container name
for the shard range uses the pattern of ``{original contianer name}-{hash of
parent container}-{timestamp}-{shard index}``.
The ``enable`` sub-command then creates some final state required to initiate
sharding the container, including a special shard range record referred to as
the container's `own_shard_range` whose name is equal to the container's path.
This is used to keep a record of the object namespace that the container
covers, which for user containers is always the entire namespace.
The :class:`~swift.common.utils.ShardRange` class
-------------------------------------------------
The :class:`~swift.common.utils.ShardRange` class provides methods for
interactng with the attributes and state of a shard range. The class
encapsulates the following properties:
* The name of the shard range which is also the name of the shard container
used to hold object records in its namespace.
* Lower and upper bounds which define the object namespace of the shard range.
* A deleted flag.
* A timestamp at which the bounds and deleted flag were last modified.
* The object stats for the shard range i.e. object count and bytes used.
* A timestamp at which the object stats were last modified.
* The state of the shard range, and an epoch, which is the timestamp used in
the shard container's database file name.
* A timestamp at which the state and epoch were last modified.
A shard range progresses through the following states:
* FOUND: the shard range has been identified in the container that is to be
sharded but no resources have been created for it.
* CREATED: A shard container has been created to store the contents of the
shard range.
* CLEAVED: the sharding container's contents for the shard range have been
copied to the shard container from *at least one replica* of the sharding
container.
* ACTIVE: shard ranges move to this state when all shard ranges in a sharding
container have been cleaved.
* SHRINKING: the shard range has been enabled for shrinking; or
* SHARDING: the shard range has been enabled for sharding.
* SHARDED: the shard range has completed sharding or shrinking.
..note::
Shard range state represents the most advanced state of the shard range on
any replica of the container. For example, a shard range in CLEAVED state
may not have completed cleaving on all replicas but has cleaved on at least
one replica.
Fresh and retiring database files
---------------------------------
As alluded to earlier, writing to a large container causes increased latency
for the container servers. Once sharding has been initiated on a container it
is desirable to stop writing to the large database; ultimately it will be
unlinked. This is primarily achieved by redirecting object updates to new shard
containers as they are created (see :ref:`redirecting_updates` below), but some
object updates may still need to be accepted by the root container and other
container metadata must still be modifiable.
To render the large `retiring` database effectively read-only, when the
:ref:`sharder_daemon` finds a container with a set of shard range records,
including an `own_shard_range`, it first creates a fresh database file which
will ultimately replace the existing `retiring` database. For a retiring db
whose filename is::
<hash>.db
the fresh database file name is of the form::
<hash>_<epoch>.db
where epoch is a timestamp stored in the container's `own_shard_range`.
The fresh DB has a copy of the shard ranges table from the retiring DB and all
other container metadata apart from the object records. Once a fresh DB file
has been created it is used to store any new object updates and no more object
records are written to the retiring DB file.
Once the sharding process has completed, the retiring DB file will be unlinked
leaving only the fresh DB file in the container's directory. There are
therefore three states that the container DB directory may be in during the
sharding process: UNSHARDED, SHARDING and SHARDED.
.. image:: images/sharding_db_states.svg
If the container ever shrink to the point that is has no shards then the fresh
DB starts to store object records, behaving the same as an unsharded container.
This is known as the COLLAPSED state.
In summary, the DB states that any container replica may be in are:
- UNSHARDED - In this state there is just one standard container database. All
containers are originally in this state.
- SHARDING - There are now two databases, the retiring database and a fresh
database. The fresh database stores any metadata, container level stats,
an object holding table, and a table that stores shard ranges.
- SHARDED - There is only one database, the fresh database, which has one or
more shard ranges in addition to its own shard range. The retiring database
has been unlinked.
- COLLAPSED - There is only one database, the fresh database, which has only
its its own shard range and store object records.
.. note::
DB state is unique to each replica of a container and is not necessarily
synchronised with shard range state.
Creating shard containers
-------------------------
The :ref:`sharder_daemon` next creates a shard container for each shard range
using the shard range name as the name of the shard container:
.. image:: /images/sharding_cleave_basic.svg
Shard containers now exist with a unique name and placed in a hidden account
that maps to the user account (`.shards_acct`). This avoids namespace
collisions and also keeps all the shard containers out of view from users of
the account. Each shard container has an `own_shard_range` record which has the
lower and upper bounds of the object namespace for which it is responsible, and
a reference to the sharding user container, which is referred to as the
`root_container`. Unlike the `root_container`, the shard container's
`own_shard_range` does not cover the entire namepsace.
Cleaving shard containers
-------------------------
Having created empty shard containers the sharder daemon will proceed to cleave
objects from the retiring database to each shard range. Cleaving occurs in
batches of two (by default) shard ranges, so if a container has more than two
shard ranges then the daemon must visit it multiple times to complete cleaving.
To cleave a shard range the daemon creates a shard database for the shard
container on a local device. This device may be one of the shard container's
primary nodes but often it will not. Object records from the corresponding
shard range namespace are then copied from the retiring DB to this shard DB.
Swift's container replication mechanism is then used to replicate the shard DB
to its primary nodes. Checks are made to ensure that the new shard container DB
has been replicated to a sufficient number of its primary nodes before it is
considered to have been successfully cleaved. By default the daemon requires
successful replication of a new shard broker to at least a quorum of the
container rings replica count, but this requirement can be tuned using the
``shard_replication_quorum`` option.
Once a shard range has been succesfully cleaved from a retiring database the
daemon transitions its state to ``CLEAVED``. It should be noted that this state
transition occurs as soon as any one of the retiring DB replicas has cleaved
the shard range, and therefore does not imply that all retiring DB replicas
have cleaved that range. The significance of the state transition is that the
shard container is now considered suitable for contributing to object listings,
since its contents are present on a quorum of its primary nodes and are the
same as at least one of the retiring DBs for that namespace.
Once a shard range is in the ``CLEAVED`` state, the requirement for
'successful' cleaving of other instances of the retirng DB may optionally be
relaxed since it is not so imperative that their contents are replicated
*immediately* to their primary nodes. The ``existing_shard_replication_quorum``
option can be used to reduce the quorum required for a cleaved shard range to
be considered successfully replicated by the sharder daemon.
.. note::
Once cleaved, shard container DBs will continue to be replicated by the
normal `container-replicator` daemon so that they will eventually be fully
replicated to all primary nodes regardless of any replication quorum options
used by the sharder daemon.
The cleaving progress of each replica of a retiring DB must be
tracked independently of the shard range state. This is done using a per-DB
CleavingContext object that maintains a cleaving cursor for the retiring DB
that it is associated with. The cleaving cursor is simply the upper bound of
the last shard range to have been cleaved *from that particular retiring DB*.
Each CleavingContext is stored in the sharding container's sysmeta under a key
that is the ``id`` of the retiring DB. Since all container DB files have unique
``id``s, this guarantees that each retiring DB will have a unique
CleavingContext. Furthermore, if the retiring DB file is changed, for example
by an rsync_then_merge replication operation which might change the contents of
the DB's object table, then it will get a new unique CleavingContext.
A CleavingContext maintains other state that is used to ensure that a retiring
DB is only considered to be fully cleaved, and ready to be deleted, if *all* of
its object rows have been cleaved to a shard range.
Once all shard ranges have been cleaved from the retiring DB it is deleted. The
container is now represented by the fresh DB which has a table of shard range
records that point to the shard containers that store the container's object
records.
.. _redirecting_updates:
Redirecting object updates
--------------------------
Once a shard container exists, object updates arising from new client requests
and async pending files are directed to the shard container instead of the root
container. This takes load off of the root container.
For a sharded (or partially sharded) container, when the proxy receives a new
object request it issues a GET request to the container for data describing a
shard container to which the object update should be sent. The proxy then
annotates the object request with the shard container location so that the
object server will forward object updates to the shard container. If those
updates fail then the async pending file that is written on the object server
contains the shard container location.
When the object updater processes async pending files for previously failed
object updates, it may not find a shard container location. In this case the
updater sends the update to the `root container`, which returns a redirection
response with the shard container location.
.. note::
Object updates are directed to shard containers as soon as they exist, even
if the retiring DB object records have not yet been cleaved to the shard
container. This prevents further writes to the retiring DB and also avoids
the fresh DB being polluted by new object updates. The goal is to
ultimately have all object records in the shard containers and none in the
root container.
Building container listings
---------------------------
Listing requests for a sharded container are handled by querying the shard
containers for components of the listing. The proxy forwards the client listing
request to the root container, as it would for an unsharded container, but the
container server responds with a list of shard ranges rather than objects. The
proxy then queries each shard container in namespace order for their listing,
until either the listing length limit is reached or all shard ranges have been
listed.
While a container is still in the process of sharding, only *cleaved* shard
ranges are used when building a container listing. Shard ranges that have not
yet cleaved will not have any object records from the root container. The root
container continues to provide listings for the uncleaved part of its
namespace.
..note::
New object updates are redirected to shard containers that have not yet been
cleaved. These updates will not threfore be included in container listings
until their shard range has been cleaved.
Example request redirection
---------------------------
As an example, consider a sharding container in which 3 shard ranges have been
found ending in cat, giraffe and igloo. Their respective shard containers have
been created so update requests for objects up to "igloo" are redirected to the
appropriate shard container. The root DB continues to handle listing requests
and update requests for any object name beyond "igloo".
.. image:: images/sharding_scan_load.svg
The sharder daemon cleaves objects from the retiring DB to the shard range DBs;
it also moves any misplaced objects from the root container's fresh DB to the
shard DB. Cleaving progress is represented by the blue line. Once the first
shard range has been cleaved listing requests for that namespace are directed
to the shard container. The root container still provides listings for the
remainder of the namespace.
.. image:: images/sharding_cleave1_load.svg
The process continues: the sharder cleaves the next range and a new range is
found with upper bound of "linux". Now the root container only needs to handle
listing requests up to "giraffe" and update requests for objects whose name is
greater than "linux". Load will continue to diminish on the root DB and be
dispersed across the shard DBs.
.. image:: images/sharding_cleave2_load.svg
Container replication
---------------------
Shard range records are replicated between container DB replicas in much the
same way as object records are for unsharded containers. However, the usual
replication of object records between replicas of a container is halted as soon
as a container is capable of being sharded. Instead, object records are moved
to their new locations in shard containers. This avoids unnecessary replication
traffic between container replicas.
To facilitate this, shard ranges are both 'pushed' and 'pulled' during
replication, prior to any attempt to replicate objects. This means that the
node initiating replication learns about shard ranges from the destination node
early during the replication process and is able to skip object replication if
it discovers that it has shard ranges and is able to shard.
.. note::
When the destination DB for container replication is missing then the
'complete_rsync' replication mechanism is still used and in this case only
both object records and shard range records are copied to the destination
node.
Container deletion
------------------
Sharded containers may be deleted by a ``DELETE`` request just like an
unsharded container. A sharded container must be empty before it can be deleted
which implies that all of its shard containers must have reported that they are
empty.
Shard containers are *not* immediately deleted when their root container is
deleted; the shard containers remain undeleted so that they are able to
continue to receive object updates that might arrive after the root container
has been deleted. Shard containers continue to update their deleted root
container with their object stats. If a shard container does receive object
updates that cause it to no longer be empty then the root container will no
longer be considered deleted once that shard container sends an object stats
update.
Sharding a shard container
--------------------------
A shard container may grow to a size that requires it to be sharded.
``swift-manage-shard-ranges`` may be used to identify shard ranges within a
shard container and enable sharding in the same way as for a root container.
When a shard is sharding it notifies the root of its shard ranges so that the
root can start to redirect object updates to the new 'sub-shards'. When the
shard has completed sharding the root is aware of all the new sub-shards and
the sharding shard deletes its shard range record in the root container shard
ranges table. At this point the root is aware of all the new sub-shards which
collectively cover the namespace of the now-deleted shard.
There is no hierarchy of shards beyond the root and its immediate shards. When
a shard shards, its sub-shards are effectively re-parented with the root
container.
Shrinking a shard container
---------------------------
A shard's contents may reduce to a point where the shard is no longer required.
If this happens then the shard may be shrunk into another shard range.
Shrinking is achieved in a similar way to sharding: an 'acceptor' shard range
is written to the shrinking shard container's shard ranges table; unlike
sharding, where shard ranges each cover a subset of the sharding container's
namespace, the acceptor shard range is a superset of the shrinking shard range.
Once given an acceptor shard range the shrinking shard will cleave itself to
its acceptor, and then delete itself from the root container shard ranges
table.