SCSP APPEND

This section provides general information about SCSP APPEND that applies to both named and unnamed objects.


APPEND is a request to the storage cluster to append arbitrary content data onto the end of an existing named object or aliased object while maintaining the previously populated metadata and object name or alias UUID. No whitespace or other characters are inserted by Swarm between the original and appended data. APPEND is not valid for immutable unnamed objects.

  • Replicated - For replicated objects, the original content data is copied by Swarm from the original object and then the data supplied in the request is appended to it. Data appended to a replicated object can cause the object to become erasure-coded if the additional appended data pushes the object size over the configured policy.ecMinStreamSize threshold.

  • Erasure-coded - APPEND requests for previously erasure-coded objects with version 6.0 are optimized to write a new set of segments with the appended data and update the manifest, instead of rewriting the whole object to include the appended data as with replicated objects. The request fails if the EC constraints cannot be met on the APPEND request. For example, if encoding is 5:2 and there are only six nodes, the APPEND request fails.

Special Query Arguments

replicate

Protects rapid updates

Objects can be updated at a maximum frequency of once per second. Updating more frequently can cause unpredictable results with the stored object version. Include the replicate=immediate query argument to guarantee more than one node can return the latest version in a subsequent read if an application updates objects faster than once per second.

newname

Renames object

To rename a named object within the same bucket, use the newname query argument, which provides a new name with the update request (PUT, COPY, APPEND). After renaming an object, requests for the original name return a 404 Not Found and the prior search metadata is removed. (Note the newname argument allows renaming domains and buckets.)

preserve

Updates custom headers

APPEND only saves the existing headers, but the preserve argument updates the existing headers with those included on the request, if any. Cannot be used with replace. (v9.5)

replace

Replaces custom headers

APPEND only saves the existing headers, but the replace argument removes the existing headers and saves the new ones included on the request, if any. Cannot be used with preserve. (v9.5)

Guidelines for APPEND

  • Include header for known or unknown size. Must include either the Content-length or Transfer-Encoding: chunked header.

    • Use the Content-length header if the size of the object is known. The Content-length value must specify the correct length of the appended content data. The Content-length header in the object is adjusted to reflect the actual length of the original data plus the appended data.

    • Use the Transfer-Encoding: chunked header (or the UNDETERMINED_LENGTH parameter if using the SDK) if the size of the object is not known (such as a live video feed). This header tells Swarm the size of the appended data is unknown. Do not combine this header with the Content-length header. All other headers stored with the object are copied without change to the newly-updated object. The x-acme-meta-* and lifepoint headers in the preceding examples are ignored.

  • Content-MD5 Headers corrected. Swarm computes the digest of the content data plus the appended data and compares it with the provided MD5 hash if providing a Content-MD5 header with the APPEND request. This assumes either access to the original data or a running digest is maintained to which appended data is added before each APPEND request. A Content-MD5 header is removed when new data is appended to the object if it was persisted with the original object. Any new, correct Content-MD5 supplied with an append is persisted with the new revision and returned on any subsequent GET or HEAD.

  • Omit Range Headers. Range headers are incompatible with the APPEND method. Including a Range header with an APPEND request results in a 400 Bad Request error response. Other aspects of the APPEND method, including response codes, are the same as PUT.

APPEND for Named and alias Objects

The syntax of an APPEND request is similar to a PUT. As with PUT, the object name or UUID returned after a successful APPEND matches the one supplied in the request. APPEND for an alias object is the same as for a named object except the object's UUID is used instead of a name on the first line of the command.

Example APPEND for named object
APPEND /mybucket/samplefile.txt HTTP/1.1    Host: cluster.example.com    Content-length: 29     x-acme-meta-color: blue     x-acme-meta-weight: 42     lifepoint: [Sunday, 06-Nov-2010 08:49:37 GMT] reps=3, deletable=no     lifepoint: [] delete    Status: Approved

APPEND for unnamed objects

APPEND is not supported for unnamed immutable objects: the UUID must be an alias object. The query argument alias=yes is optional as of v7.0.

Swarm returns a 404 Not Found error when the object is not alias (appending to an immutable object or to a UUID that does not exist).

Normal Responses to APPEND

The APPEND method returns a response only after all data is copied and the update is complete.

See https://perifery.atlassian.net/wiki/spaces/public/pages/edit-v2/2443821386# for a list of response headers.

Error Responses to APPEND

Swarm responds with HTTP 409 Conflict if the APPEND method is executed on an object in a domain but the domain does not exist or is not in the content cache on the node receiving the request.

© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.