Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Amazon S3 is two distinct things: "S3 The Service" and "S3 The Protocol." Since the S3 protocol reflects characteristics of the Amazon service that may not be applicable outside of Amazon, the Gateway adapts these characteristics so they make sense when hosting storage within the environment. These adaptations are transparent to most applications and enhance the protocol features by making use of the unique strengths of Swarm storage.

...

An Amazon AWS region is roughly analogous to a Swarm storage domain. A Swarm bucket is tied to a domain as an S3 bucket is tied to a region. They are fundamentally different in the following ways:

  • AWS regions are finite, but any number of Swarm domains can be created; this allows achieving the optimal granularity for assigning content user permissions.

  • AWS buckets must have a name unique across all regions, but Swarm buckets need only be unique within the domain.

  • AWS buckets are fixed geographically, but Swarm content dynamically distributes across the entire Swarm cluster, according to the content protection settings.

  • AWS buckets can be replicated to a differently named bucket, but Swarm replication preserves the domain, bucket, and object names identically across every target cluster, which supports content sharing and direct DR fail-over.

  • AWS buckets can replicate in one direction, but Swarm clusters can use mirrored replication. A domain's content can be created, updated, and accessed across distributed storage clusters while remaining synchronized.

...

  • Supply the location in the bucket creation operation using LocationConstraint. [AWS S3 docs link]

  • Set the [s3]region option in the Gateway configuration file to the preferred region. This applies to all buckets unless the location is specified during creation.

...

Tip

Best practice

Unless you have an application that requires otherwise, either use the default, or choose another valid AWS region, as there are clients that validate the location against a list of known AWS regions."

Storage Class

Amazon S3 allows clients to set a storage class preference in the x-amz-storage-class header, which defines the data durability and access frequency of content. See the Amazon S3 Storage Classes.

...

AWS S3 requires every part (except the last) of a multipart upload must be at least 5 MB in size. The Gateway's S3 protocol implementation does not impose this limitation and allows each part of a multipart upload to be of any size.

Multipart writes are long-running operations with initial and final responses. For S3 compatibility, the initial response returns x-amz-version-id with the value of the expected ETag. There is no new object if there is an error completing the write, and the expected ETag given is not valid. (v6.3)

List after Update Timing

After an object is created, the delay before that object appears in a list operation can vary depending upon the Swarm metadata feed batch timeout setting. New objects are available within two seconds following a create when the batch timeout is set to 1 second. The Amazon S3 documentation has specific developer guidance about this eventual consistency behavior.

...