Changes

This release of Content Gateway focuses on adding support for the next generation of SCSP SEND, which is foundational to future capabilities. SCSP SEND allows forcing an object to be written immediately another cluster for which a replication feed exists. (CLOUD-2068)

Upgrade Impacts

See https://perifery.atlassian.net/wiki/spaces/public/pages/edit-v2/2443806577#. See https://perifery.atlassian.net/wiki/spaces/public/pages/2443817356 if migrating from Elasticsearch 2.3.3 and are ready to upgrade from Gateway 5 to upgrade from a version of Gateway 6.

Address the upgrade impacts for each of the versions since the one being upgraded from:

Watch Items and Known Issues

These are known operational limitations and watch items that exist for Gateway.

• Traffic to the Gateway is blocked unless action is taken to disable IPTABLES or to enable inbound traffic to the front-end protocol port(s) when using the default RHEL/CentOS configuration of IPTABLES.

• Gateway is not compatible with Linux PAM modules that depend upon interactive validation operations such as OTP or biometric scanners.

• SCSP reading operations that request a Content-MD5 hash validation and for which there is a hash mismatch cause a storage node to be temporarily removed for the Gateway's connection pool due to the way Swarm reports a hash validation failure.

• Swarm Integrity Seal upgrades cannot be performed through Gateway. They may be done directly to the back-end Swarm cluster.

• Review the discussion of these in the https://perifery.atlassian.net/wiki/spaces/public/pages/2443820816/Storage+SCSP+Development?search_id=9934921c-f36c-4deb-bb59-dab5c2b6d9f0 if the HTTP cache control headers If-Modified-Since and If-Unmodified-Since are used.

The following are known issues in this release:

• A valid APPEND request on an alias (unnamed) object returns 404 Not Found. (CLOUD-3181)

• The AWS S3 SDK for C# does not properly sign S3-compatible requests with spaces in the name unless the domain contains ".s3." or ".s3-". See https://github.com/aws/aws-sdk-net/issues/933. (CLOUD-3068)

• When buckets are created, the x-amz-storage-class header is not preserved. (CLOUD-3062)

• The Gateway error "Failed reading from client" on a PUT due to "EofException: Early EOF" may occur when clients do not send the full body. This may point to a bug in the client's retry logic, such as not resetting the position marker to the beginning of the file or part. (CLOUD-3010)

• During new object creation as part of renaming with ?newname, Gateway does not verify the user has permission to create the new object name (although it is highly likely, because it is a write within the same context). (CLOUD-2966)

• An s3cmd or rclone server-side copy request may time out on a multipart copy for >5GB objects (s4cmd performs it correctly). Workaround: After you verify the HTTPS proxy is not timing out, increase the client timeout: set s3mcd socket_timeout = 600 in ~/.s3cfg or use rclone copy --timeout=10m --contimeout=2m caringo:mybucket/5gb caringo:mybucket/subfolder/. (CLOUD-2949)

• Listings with max-keys may be shorter than expected because CommonPrefixes are included in the count of keys returned. (CLOUD-2917)

• Uploading files / photos using Panic's Transmit app on iOS fails due to a 403 Invalid Signature error. (CLOUD-2886)

• Usernames are case-insensitive, but listings exclude a token if the username (myadmin) does not match the case used when the token was created (myAdmin). (CLOUD-2837)

• Multipart PUT requests via recent Cyberduck versions fail with 403 SignatureDoesNotMatch when using AWS Signature Version 4. Install the Caringo .cyberduckprofiles from Using the Cyberduck application with Content Gateway S3 which force V2 signatures. (CLOUD-2799)

• The policy fails to take effect without warning if a policy document includes a Principal that has plural "users" or "groups" instead of "user" or "group". (CLOUD-2783)

• Versioning-enabled buckets with large numbers of objects may generate Gateway server.log warnings that can be safely ignored: "S3BucketRequestHandler: WARNING: problem with versioned bucket listing. Number of CommonPrefix (2000) exceeds max-size limit (1000)." (CLOUD-2643)

• 403 S3 V4 Signature mismatch errors may result when using Cyberduck with the "pound" proxy in front of Gateway S3. Workaround: Disable the Expect header in the Cyberduck preferences, or (recommended) use a different proxy such as HAProxy. (CLOUD-2628)

• The errors may erroneously report this as being related to Storage nodes when Gateway cannot connect to Elasticsearch nodes. (CLOUD-2595)

• Because of issues with Range and ETag header handling, video playback of .mp4 streams may not work correctly when served via the Gateway S3 port. It does work when served via the Gateway SCSP port. (CLOUD-1964)

• Gateway caches the Swarm version from the "Server:" response header, so after upgrading Swarm you must restart Gateway to consistently see the new version. (CLOUD-1271)

• Gateway responds with a 500 (Internal Server Error) instead of 400 (Bad Request) if the size of the metadata headers sent to Swarm is too large. (CLOUD-800)

• The S3 bucket listing StorageClass response element always reports STANDARD. (CLOUD-766)

• The Gateway audit log escapes the "%" characters used by the client as escape characters if an S3 client escapes URI path characters such as "/". URI audit log processing for S3 clients require double-unescaping when this occurs. (CLOUD-703)