Versions Compared

Old Version 25

changes.mady.by.user Kyle Miller

Saved on

compared with

New Version Current

changes.mady.by.user Kyle Miller

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents
maxLevel3

SCSP Proxy Essentials

The Swarm SCSP Proxy accepts HTTP requests from a host network, forwards them to a Swarm cluster, handles redirects transparently, and then supplies the response back to the requestor. In many deployments, a Swarm cluster may be isolated on an internal network, protecting it from undesired interaction with the host network, and also protecting the host network from services like PXE boot and multicast traffic that can interfere with other network resources.

...

Response

Description

Code Block
Scsp-Proxy-Cluster

Cluster name (for example, if the cluster is configured on a CSN, it . It is the value of scsp.clusterName in /etc/caringo/scspproxy/scspproxy.cfg )if the cluster is configured on a CSN.

Code Block
Scsp-Proxy-Nodes

ASCII string count of the number of node IP addresses returned in the body of the response. If this count is zero, an additional reason is supplied to explain why there are no nodes returned. More information about the zero-node response is discussed following the table.

Code Block
Scsp-Proxy-Agent

SCSP Proxy and its software version.

Code Block
Castor-System-TotalGBAvailable 
Castor-System-TotalGBCapacity

Obtained by polling any available node on the local cluster. These headers are provided to maintain consistency with current SCSP Proxy and Swarm responses.

Note: These headers are returned if a cluster can be contacted.

For example, the The following response indicates there are 10 nodes in the cluster:

...

Reason code

Meaning

no-nodes

The SCSP Proxy reports no nodes when , for example, the cluster is off-line or is being rebooted.

bad-subnet

The request was made from a subnet from which the node IP addresses are not routable.
The SCSP Proxy determines whether or not a request originated from the same private subnet as its cluster and responds only if the request did originate in that subnet. The SCSP Proxy does this to help prevent malicious discovery about the cluster and because private IP addresses are not routable anyway.

bad-cluster

The cluster name supplied in the Scsp-Proxy-Cluster header did not match the currently configured cluster.

disabled

Indicates the SCSP Proxy's reportHosts configuration parameter is set to False.

...

To properly route subsequent requests, the SCSP Proxy returns one Location response header with its own external IP address and discards any other Location headers. For Replicate on Write requests, all Location response headers are rewritten and only the one generated Location header is returned.

Remote Cluster Communication and Coordination

Developers who need to interact with multiple Swarm clusters, either independently or in multi-cluster requests, can use the SCSP Proxy for remote cluster communication and can even coordinate requests between more than one cluster at a time.

...

You must supply a list of either URL-encoded names or UUIDs. (Use percent encoding for object names, if needed.) The name, UUID, or the consistency checkpoint, is stored as a Swarm object.

...

For each line of the checkpoint manifest, the SCSP Proxy uses chunked encoding to send in the AggregateInfo response body either a parse error for the line or the line's name or UUID followed by the Swarm's verbatim response for the Info query for it. A CRLF line- end sequence separates the uuid line from the Info response. For example, The following representative response body is returned for a manifest that included including two UUIDs, both of which the SDK was able to Info successfully, the following representative response body is returned:

Code Block
Response for stream uuid = 64f9e40bfcb7046d79cc02c5876c1904
HTTP/1.1 200 OK
Castor-System-Alias: 64f9e40bfcb7046d79cc02c5876c1904 Castor-System-Cluster: DR Cluster
Castor-System-Created: Tue, 03 Aug 2010 18:49:27 GMT
Castor-System-Version: 1280861367.827
Content-Length: 11
Content-Type: application/x-www-form-urlencoded Last-Modified: Tue, 03 Aug 2010 18:49:27 GMT Etag: "aa9ba7fb4648f05c21411cce06c7c3d5"
Date: Tue, 03 Aug 2010 18:49:28 GMT
Server: CAStor Cluster/4.0.2

Response for stream uuid = f6a0c60d53dd1047ce670c747b1aba91 HTTP/1.1 200 OK
Castor-System-Cluster: DR Cluster
Castor-System-Created: Tue, 03 Aug 2010 18:49:27 GMT
Content-Length: 11
Content-Type: application/x-www-form-urlencoded Last-Modified: Tue, 03 Aug 2010 18:49:27 GMT Etag: "f6a0c60d53dd1047ce670c747b1aba91"
Date: Tue, 03 Aug 2010 18:49:28 GMT
Server: CAStor Cluster/4.0.2

If the first line does not have a valid UUID, the SCSP Proxy responds with a single error message in the response body stating that the checkpoint stream is incorrectly formatted and stops processing.

Installing the SCSP Proxy

The Swarm SCSP Proxy installs as an RPM and is managed like other RPM packages. The installation and base configuration are performed automatically as part of CSN setup if the Swarm SCSP Proxy is being used on a CSN.

...

Red Hat does not support in-place upgrades between major versions, so to use RHEL 6.x for an existing installation running RHEL 5.x, see the Red Hat documentation.

Info

Important

Before starting the SCSP Proxy installation, verify the RHEL installation is running the correct version.

...

The SCSP Proxy installation script installs the watchdog process monit, which performs a variety of functions, including starting and restarting SCSP Proxy services if they fail unexpectedly. If monit is not already installed on your system, SCSP Proxy installs and configures it automatically.

...

  • Changes /etc/monit.conf by adding the following:

    Code Block
    #START Caringo additions
set daemon 15 					# check services at 15 second intervals
set httpd port 2812 and
	use address localhost 		# only accept connection from localhost
	allow localhost 			# allow localhost to connect to server
set logfile /var/log/monit.log	# logs to local file
##END Caringo Additions

  • Saves your original /etc/monit.conf as /etc/monit.conf.orig

  • Installs a product-specific configuration file under /etc/monit.d. The filename ends with .monitrc

  • With RHEL 6: Adds /etc/init/monit.conf to enable the upstart service to monitor monit so it runs when the machine restarts.

...

Option Name

Default

Description

[proxy] interface

none

Required. The IP address of the external interface of the SCSP Proxy machine . For example, (e.g. 192.168.0.1).

[proxy] port

80

Required. The SCSP Proxy listen port. The port must be on the SCSP Proxy machine's external interface.

[proxy] reportHosts

True

Enables or disables the ability to detect the addresses of cluster nodes. Valid case- sensitive values are True and False.

[proxy] validationMode

False

Whether or not the SCSP Proxy should be running in validation mode or sending received requests to the Swarm cluster.

False

Determines whether or not the SCSP Proxy returns the IP addresses of Swarm nodes located in a different subnet.

  • False (default) means SCSP Proxy returns IP address of Swarm nodes only if the nodes are in the same subnet as the SCSP Proxy.

  • True means all Swarm IP addresses are returned, even if those nodes are in a subnet different from the SCSP Proxy.

localhost

Syslog server's fully qualified host name or IP address. Comment out this parameter if you are using file-based logging instead of syslog . For example, (e.g. 192.168.0.2).

514

The port number of the syslog server to send log messages to

stdout

Local log file name if you are not using syslog. To use file-based logging, the value of host must be null or the parameter must be absent.

0

The number of bytes allowed for all file-based log files. The default value, 0, means the log file size is unlimited. Comment out this parameter if you use remote logging.

40

One of the following log levels, listed from most to least verbose:

  • 10 = Debug (Request data, locator, and connection pool)

  • 20 = Info (Request, and response tracing at a header level)

  • 30 = Warn (Run-time warnings)

  • 40 = Error (System-level errors (failed to get IP address, port, host name, and so on for an existing connection), protocol errors (missing Host header, invalid body data received), configuration errors (bad remote host definition, invalid SCSP Proxy configuration), dependency failures (such as Avahi), caught exceptions, connection pool errors, internal state watching errors)

Note: More verbose log levels include all information in all levels below it. For example, Debug includes all nformation information in Info, Warn, and Error.

0

The log facility (that is, channel) to log to when using a syslog. Valid values are 0-6.

localhost

The IP address to use in log message headers.

200

The number of connections the SCSP Proxy saves for reuse. DataCore recommends you set this parameter to five times the number of Swarm cluster nodes. Valid values are 200 (minimum) to 4000 (maximum).

60

The length of time, in seconds, to wait for a connection to be made or maintained. Lack of response for longer than the specified timeout result in the connection being closed by Proxy. This timeout may need to be increased for unusually long operations, like a COPY operation on a large object.

300

The length of time, in seconds, an unused connection remains in the connection pool

80

The local Swarm cluster listen port. The value of the cluster's scspport configuration parameter in the node or cluster configuration file.

Note: This value must match the other SCSP Proxy's [proxy] port configuration parameter if this SCSP Proxy is configured to communicate with another SCSP Proxy.

none

Used by the StaticLocator (starting the SCSP Proxy with – staticlocator). If this is how you use the SCSP Proxy, set the value of this parameter to a space-separated list of IP addresses of hosts in the local Swarm cluster. For example, 192: “192.168.0.11 192.168.0.12 12”
Comment this parameter out otherwise.

none

The name of the local Swarm cluster for use in discovering node IP addresses. This must match the configured cluster parameter in the cluster's node or cluster configuration file.
If the SCSP Proxy is installed on a Cluster Services Node (CSN), the cluster parameter is set during the initial network bootstrap process.
For example, :
cluster.example.com

/etc/caringo/scspproxy/ hosts.cfg

The path to the location of the hosts.cfg file on the local server.

...

Code Block
# Sample scspproxy Remote Host Configuration file #
# ClusterName RemoteAddress Port RemoteAdminName RemoteAdminPassword # One entry per line
# Fields are whitespace (space and tab) separated, although the password field # may contain whitespace.
# Note that this implies that none of the other fields may contain whitespace. #
#DRCluster 192.168.1.123 80 admin yourpwdofchoicehere
#dr 192.168.1.123 80 admin yourpwdofchoicehere
#dr2 192.168.1.123 80 admin yourpwdofchoicehere

NewYorkCluster 192.168.1.148 80 admin yourpwdofchoicehere

Running the SCSP Proxy

After the configuration file has been updated, start and stop the SCSP Proxy either way discussed below. You must run these commands as a user with root privileges.

...

Validation

GET

HEAD

POST

DELETE

PUT

COPY

APPEND

Header: Lifepoint (date)



V


V

V


Header: Lifepoint (deletion)



V


V

V


Header: replica count



V


V

V


Header: Host

V

V

V

V

V

V

V

Header: Content- Length



V


V


V

Header: Allow



V


V

V


Header: Content- Type



V


V

V


Header: Content- Disposition



V


V

V


Query Argument: replicate



V


V

V

V

Query Argument: admin

V

V

V

V

V

V

V

Query Argument: alias

V

V

V

V

V

V

V

Query Argument: domain

V

V

V

V

V

V

V

Query Argument: validate

V







Query Argument: hashtype

V


V


V

V

V

Query Argument: hash

V







Query Argument: newhashtype

V







Query Argument: countreps


V






Query Argument: indirect

V







Uninstalling the SCSP Proxy

Enter the following command as a user with root privileges to uninstall the SCSP Proxy:

...