Versions Compared

Old Version 12

changes.mady.by.user Kyle Miller

Saved on

compared with

New Version Current

changes.mady.by.user Bala Harish A(AC)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents
minLevel1
maxLevel2
outlinefalse
typelist
printablefalse

This section provides information specific to running Swarm Storage with Gateway. Install and configure Swarm, the storage cluster (storage nodes running on dedicated hardware) before proceeding.

...

Network Placement

When deployed with Gateway, the storage nodes should be placed on a network subnet not directly accessible to client applications. All user communications with the storage cluster must go through the Gateway.

Infonote

Caution

If users are allowed to communicate directly with the storage cluster nodes, they may bypass access security, the business rules for content metadata, and audit logging performed by the Gateway and may render content in the cluster unusable to the Gateway.

Only allow direct access to the storage cluster nodes under highly controlled circumstances, such as administrator-only operations or trusted applications.

Domain Management

The Swarm cluster provides for logical separation of content among multiple tenants through the use of storage domain names. Gateway has the following requirements beyond those for a baseline storage deployment and client usage.

...

In the example zone file, 10.100.100.100 is the IP address used by client applications to communicate with the Gateway or a front-end load balancer. The names hydrogen2.cloud.example.com and oxygen.cloud.example.com both resolve to the same IP address.

Elasticsearch Servers

When using the S3 storage protocol, the metadata search service must be accessible to the Gateway servers.

When deployed with Gateway, like the storage nodes, the typical placement is on a network subnet not directly accessible to the client applications. There are no end-user supported API calls directly to the metadata search service.

Info

Listing

consistency

Consistency

Search feeds show eventual consistency as content changes, but enabling the Gateway Configuration [s3] option enhancedListingConsistency improves the search-after-create response to the client applications using the Gateway.

Configuration Requirements

Use these Swarm configuration settings and adhere to the following operational changes when using Swarm Storage with Gateway. These configuration changes refer to the configuration file(s) for Swarm.

  • CSN — : This is the cluster-wide file: /var/opt/caringo/netboot/content/cluster.cfg

  • Platform Server — : This is the cluster-wide file used to deploy, which is located by default here: /etc/caringo/cluster.cfg

  • No platform server — Platform Server: This is the node-specific configuration file: node.cfg

Infonote

Caution

Failure to use these settings and operational changes can prevent Gateway from working properly with the storage cluster.

Requirement

Description

Optimize GETs 

With Swarm 12.0 and higher, a setting can be added to improve performance through Gateway. Enable scsp.enableVolumeRedirects to permit Gateway to redirect GET requests to volume processes. These redirects increase efficiency, especially with reading small objects.

Code Block
languagetext
scsp.enableVolumeRedirects = True

Enable an EC encoding

S3 multipart (large file) writes fail if erasure coding is not configured; define an ecEncoding if using S3.

Code Block
languagetext
policy.ecEncoding = {k:p}

See Implementing EC Encoding Policy

.info

Note

the

The value put into the configuration file is used on first boot. After the cluster is running, the original values are persisted and must be updated dynamically (using UI or SNMP). See Persisted Settings (SNMP) .

Clear legacy settings

Unless needed for backwards compatibility (because untenanted objects are used in the cluster and do not need S3), enable tenancy for unnamed objects, which verifies every object is written to a domain (see How enforceTenancy Works):

Code Block
languagetext
cluster.enforceTenancy = True

Set it to True and reboot the cluster if this was set to False.

Storage Domain Management

Only create and manage storage domains through the Content UI or programmatically through the Gateway's management API.

The cluster configuration contains security.noauth=False if storage domain management displays in the legacy Admin Console (port 90), which is not supported by Content Gateway. Set it to True and reboot the cluster.

Troubleshooting: If the Content UI reports "Page Not Found: The original bucket to which this collection refers cannot be found or has been replaced", it is likely the domain was created by the legacy Admin Console (port 90) and contains the legacy Castor-Authorization header. DataCore Support for help correcting the domain.

...