From b8fa48080812ab0f6bac7ae19c0290da08f6a6e4 Mon Sep 17 00:00:00 2001 From: Donagh McCabe Date: Thu, 4 Feb 2016 16:19:13 +0000 Subject: [PATCH] Added links to API reference and usage guides Added a link to the API reference (where headers and query strings are documented; click the "detail" button to see). Also added a reference to Swift section of the OpenStack end user guide. This contains some additional details about the API. No attempt was made to reconcile duplicate information. Instead this patch links documents that might otherwise be overlooked. However, I fixed text, originally in a table, that had become garbled in a prior patch. Change-Id: I0910cbeb0c8bffc00e510f35585603e7b7a67790 --- doc/source/api/large_objects.rst | 111 +++++++++++++++----------- doc/source/index.rst | 12 +++ doc/source/overview_large_objects.rst | 4 +- 3 files changed, 80 insertions(+), 47 deletions(-) diff --git a/doc/source/api/large_objects.rst b/doc/source/api/large_objects.rst index 144605f117..273cde81e4 100644 --- a/doc/source/api/large_objects.rst +++ b/doc/source/api/large_objects.rst @@ -58,7 +58,7 @@ header. This ensures that the upload cannot corrupt your data. List the name of each segment object along with its size and MD5 checksum in order. -Create a manifest object. Include the *``?multipart-manifest=put``* +Create a manifest object. Include the ``multipart-manifest=put`` query string at the end of the manifest object name to indicate that this is a manifest object. @@ -112,8 +112,8 @@ set to be the MD5 checksum of the concatenated ``ETag`` values of the object segments. You can also set the ``Content-Type`` request header and custom object metadata. -When the **PUT** operation sees the *``?multipart-manifest=put``* query -parameter, it reads the request body and verifies that each segment +When the **PUT** operation sees the ``multipart-manifest=put`` query +string, it reads the request body and verifies that each segment object exists and that the sizes and ETags match. If there is a mismatch, the **PUT**\ operation fails. @@ -124,18 +124,18 @@ this is a static object manifest. Normally when you perform a **GET** operation on the manifest object, the response body contains the concatenated content of the segment objects. To download the manifest list, use the -*``?multipart-manifest=get``* query parameter. The resulting list is not +``multipart-manifest=get`` query string. The resulting list is not formatted the same as the manifest you originally used in the **PUT** operation. If you use the **DELETE** operation on a manifest object, the manifest object is deleted. The segment objects are not affected. However, if you -add the *``?multipart-manifest=delete``* query parameter, the segment +add the ``multipart-manifest=delete`` query string, the segment objects are deleted and if all are successfully deleted, the manifest object is also deleted. To change the manifest, use a **PUT** operation with the -*``?multipart-manifest=put``* query parameter. This request creates a +``multipart-manifest=put`` query string. This request creates a manifest object. You can also update the object metadata in the usual way. @@ -238,67 +238,88 @@ Comparison of static and dynamic large objects While static and dynamic objects have similar behavior, here are their differences: -**Comparing static and dynamic large objects** +End-to-end integrity +-------------------- -Static large object: Assured end-to-end integrity. The list of segments -includes the MD5 checksum (``ETag``) of each segment. You cannot upload the -manifest object if the ``ETag`` in the list differs from the uploaded segment -object. If a segment is somehow lost, an attempt to download the manifest -object results in an error. You must upload the segment objects before you -upload the manifest object. You cannot add or remove segment objects from the -manifest. However, you can create a completely new manifest object of the same -name with a different manifest list. +With static large objects, integrity can be assured. +The list of segments may include the MD5 checksum (``ETag``) of each segment. +You cannot upload the manifest object if the ``ETag`` in the list differs +from the uploaded segment object. If a segment is somehow lost, an attempt +to download the manifest object results in an error. -With static large objects, you can upload new segment objects or remove -existing segments. The names must simply match the ``{prefix}`` supplied -in ``X-Object-Manifest``. The segment objects must be at least 1 MB in size -(by default). The final segment object can be any size. At most, 1000 segments -are supported (by default). The manifest list includes the container name of -each object. Segment objects can be in different containers. - -Dynamic large object: End-to-end integrity is not guaranteed. The eventual +With dynamic large objects, integrity is not guaranteed. The eventual consistency model means that although you have uploaded a segment object, it might not appear in the container listing until later. If you download the manifest before it appears in the container, it does not form part of the content returned in response to a **GET** request. +Upload Order +------------ + +With static large objects, you must upload the +segment objects before you upload the manifest object. + With dynamic large objects, you can upload manifest and segment objects in any order. In case a premature download of the manifest occurs, we recommend users upload the manifest object after the segments. However, -the system does not enforce the order. Segment objects can be any size. All -segment objects must be in the same container. +the system does not enforce the order. + +Removal or addition of segment objects +-------------------------------------- + +With static large objects, you cannot add or +remove segment objects from the manifest. However, you can create a +completely new manifest object of the same name with a different manifest +list. + +With dynamic large objects, you can upload new segment objects or remove +existing segments. The names must simply match the ``{prefix}`` supplied +in ``X-Object-Manifest``. + +Segment object size and number +------------------------------ + +With static large objects, the segment objects must be at least 1 byte in size. +However, if the segment objects are less than 1MB (by default), +the SLO download is (by default) rate limited. At most, +1000 segments are supported (by default) and the manifest has a limit +(by default) of 2MB in size. + +With dynamic large objects, segment objects can be any size. + +Segment object container name +----------------------------- + +With static large objects, the manifest list includes the container name of each object. +Segment objects can be in different containers. + +With dynamic large objects, all segment objects must be in the same container. Manifest object metadata ------------------------ -For static large objects, the object has ``X-Static-Large-Object`` set to -``true``. You do not set this metadata directly. Instead the system sets -it when you **PUT** a static manifest object. +With static large objects, the manifest object has ``X-Static-Large-Object`` +set to ``true``. You do not set this +metadata directly. Instead the system sets it when you **PUT** a static +manifest object. -For dynamic object,s the ``X-Object-Manifest`` value is the -``{container}/{prefix}``, which indicates where the segment objects are -located. You supply this request header in the **PUT** operation. +With dynamic large objects, the ``X-Object-Manifest`` value is the +``{container}/{prefix}``, which indicates +where the segment objects are located. You supply this request header in the +**PUT** operation. Copying the manifest object --------------------------- -With static large objects, you include the *``?multipart-manifest=get``* +The semantics are the same for both static and dynamic large objects. +When copying large objects, the **COPY** operation does not create +a manifest object but a normal object with content same as what you would +get on a **GET** request to the original manifest object. + +To copy the manifest object, you include the ``multipart-manifest=get`` query string in the **COPY** request. The new object contains the same manifest as the original. The segment objects are not copied. Instead, both the original and new manifest objects share the same set of segment objects. -When creating dynamic large objects, the **COPY** operation does not create -a manifest object but a normal object with content same as what you would -get on a **GET** request to original manifest object. -To duplicate a manifest object: - -* Use the **GET** operation to read the value of ``X-Object-Manifest`` and - use this value in the ``X-Object-Manifest`` request header in a **PUT** - operation. -* Alternatively, you can include *``?multipart-manifest=get``* query - string in the **COPY** request. - -This creates a new manifest object that shares the same set of segment -objects as the original manifest object. diff --git a/doc/source/index.rst b/doc/source/index.rst index 30bfe31808..839de9c694 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -90,6 +90,10 @@ Administrator Documentation Object Storage v1 REST API Documentation ======================================== +See `Complete Reference for the Object Storage REST API `_ + +The following provides supporting information for the REST API: + .. toctree:: :maxdepth: 1 @@ -104,6 +108,14 @@ Object Storage v1 REST API Documentation api/use_content-encoding_metadata.rst api/use_the_content-disposition_metadata.rst +OpenStack End User Guide +======================== + +The `OpenStack End User Guide `_ +has additional information on using Swift. +See the `Manage objects and containers `_ +section. + Source Documentation ==================== diff --git a/doc/source/overview_large_objects.rst b/doc/source/overview_large_objects.rst index 89a3fd93d7..85a972120c 100644 --- a/doc/source/overview_large_objects.rst +++ b/doc/source/overview_large_objects.rst @@ -45,8 +45,8 @@ Direct API SLO support centers around the user generated manifest file. After the user has uploaded the segments into their account a manifest file needs to be -built and uploaded. All object segments, except the last, must be above 1 MB -(by default) in size. Please see the SLO docs for :ref:`slo-doc` further +built and uploaded. All object segments, must be at least 1 byte +in size. Please see the SLO docs for :ref:`slo-doc` further details. ----------------