Erasure Coding across Subclusters in Swarm 9

In addition to spreading segments out across nodes (so no two segments for the same erasure set reside on the same node), Swarm automatically distributes segments evenly across subclusters. If it is configured correctly, this distribution keeps your data is protected even if an entire site fails.

In a subcluster configuration, your data is safe when the following is true:

(Remaining subclusters) / (Total subclusters) >= k data-segments / (k data-segments + p parity-segments)

Example: Subcluster setting that does not guarantee full recovery

With two subclusters containing four nodes each and EC set to 5:2 for seven total segments, one subcluster with three segments and the other with four segments. The number of segments in each subcluster is greater than the parity count (in this example), so the object is NOT fully recoverable if an entire subcluster is lost. Using the formula above to double check shows that the data is NOT safe if a single subcluster is lost:

1/2 IS NOT >= 5 /(5+2)

Example: Subcluster setting that does guarantee full recovery

With 3 subclusters containing 5+ nodes each and EC set to 9:6 for a total of 15 segments, each subcluster has five segments. If an entire subcluster is lost, five segments are lost, which is less than the set parity level, so the object is recoverable.

Using the formula above to double check shows that the data is safe if one of the three subclusters is lost:

2/3 >= 9 / (9+6)

The following table compares these examples.

EC Encoding	Nodes needed to write an EC object	Max nodes that can fail and still read EC object	Protected with loss of 1 of 2 subclusters?	Protected with loss of 1 of 3 subclusters?

EC Encoding	Nodes needed to write an EC object	Max nodes that can fail and still read EC object	Protected with loss of 1 of 2 subclusters?	Protected with loss of 1 of 3 subclusters?
5:2	7	2	NO. With 7 segments, one subcluster with 3 segments and the other with 4 segments, both over the maximum allowed.	NO. With 7 segments, one subcluster has 3 segments and the other two have 2 segments each. The object is not be recoverable because that value is over the maximum allowed if the subcluster with 3 objects went offline.
9:6	15	6	NO. With 15 segments, one subcluster with 8 segments and the other with 7 segments, both over the maximum allowed.	YES. With 15 segments, each subcluster has 5 segments, which is less than the maximum allowed.