The Swarm Cluster Services (SCS) server isolates Swarm within an internal network. The network is defined by both the IP address assigned to the interface on the SCS server connected to that network as well as the subnet defined for that network. The SCS server runs a DHCP service on the network for booting Swarm storage nodes up in the network.

The DHCP server allows the following IP addresses:

• Reserved Range: Two configurable reserved ranges in DHCP are at the upper and lower extremities of the subnet. These reserved ranges are special IP addresses for routers, network devices, etc., and are used for any servers that require direct participation in the Swarm cluster (Elasticsearch nodes, Content Gateway instances, etc.). None of these ranges has any default size., and at least one must be specified during DHCP initialization.

• Storage Pool: IP addresses that are used for Swarm storage nodes. This pool is used to provide temporary IP addresses when storage nodes are first booting, and any static IP addresses are also within this range.
  See https://perifery.atlassian.net/wiki/spaces/KB/pages/3118530607 for details about static IP address configuration.

The DCHP server utilizes the following breakdown of the subnet:

Range Sizing

For most installations, the DHCP-managed range (the “middle” of the subnet) spans the majority of the network’s address space, with small or nonexistent lower or upper ranges. However, at least one of the lower or upper ranges must have a non-zero number of IP addresses set aside for it.

SCS Internal Interface

The Swarm-facing network interface on the SCS server acts as a contact point for all SCS services for the cluster (logging, PXE booting, NTP, etc.) and as a default network gateway. Select an appropriate static IP address cautiously for this interface. The IP address is often at the bottom of the lower range (but it is not required).

However, the IP address must be within either the upper or lower range; placing it outside of these ranges may conflict with the IP allocation.

Other Swarm Services

Gateway or Elasticsearch instances run for the Swarm cluster on the Swarm network. Their IP addresses are statically allocated in either the lower or upper reserved ranges.

Static IP Address Pool

If the Swarm deployment requires storage nodes to be given static IP addresses, then see https://perifery.atlassian.net/wiki/spaces/KB/pages/3118530607.

Other IP Addresses

Sometimes, the organizational policy represents that other services have access to the network while the Swarm network is private. The static IP addresses are provided to these services in either the lower or upper reserved ranges.

Swarm (Internal) Network MTU

If the MTU needs to be configured for the internal Swarm network, see https://perifery.atlassian.net/wiki/spaces/public/pages/1223491667/SCS+Administration#Swarm-(Internal)-Network-MTU.

Examples

The below table represents the mapping of CSN concepts with SCS:

Small Network (/24)

Implications

• 222 Storage nodes (size of storage pool) on the network

• Up to 222 chassis can PXE boot simultaneously

Large Network (/16)

Implications

• 65,000 Storage nodes (size of storage pool) on the network

• Up to 65,000 chassis can PXE boot simultaneously

Podman Network Ranges

Podman sets up an internal network for containers to talk to each other. The range 10.88.0.0/16 is used internally for the podman network on SCS nodes and is reserved for container communication. If this IP range (or an overlapping IP address range) is used for normal network communications, conflicts occur and network traffic becomes unpredictable. Verify the IP range is not used outside of Swarm.

External Network Connectivity

Storage Cluster Connectivity

When the storage cluster needs to communicate outside of its network (replication feeds, for example), the SCS machine provides network address translation (NAT) to keep the internal network better shielded from external access.

HTTP Proxy

SCS does not support use of environment variables that govern the use of http proxies (http_proxy, https_proxy, etc.). If these variables are set on SCS, network communication will be disrupted.