| Info |
|---|
Swarm 9 combines the scalable software-defined object storage of Swarm Storage with the components to support diverse implementations:
|
...
Usability Enhancements for UI — The Content UI has significant new capability around object renaming and version management. From the UI, you can now rename an object, including any pseudo path. Renaming a versioned object has the effect of marking the existing object as deleted and creating a new object using the new name. The rename dialog alerts you to this behavior if the object being renamed is versioned.
The Content UI also allows precise control over managing and deleting versioned objects so the full storage API expressiveness is incorporated into UI actions. This allows for deleting the object so it can be restored later, deleting a specific version of the object from the version history, or completely deleting the object along with its entire version history. The UI also clearly denotes special delete markers in the version history.
Next-Generation Replication Method — Swarm Storage now supports a new replication method, replication by direct POST. Swarm's new replication technology offers better performance and flow management between source and target clusters, so the rate can be increased (for faster hydration) or lowered, to prevent swamping a smaller target. Now, when you define a replication feed, you specify which replication mode to use: either the legacy bidirectional GET method of replication (which you may need for specific application compatibility or network requirements) or the new direct POST method.
The Swarm Storage UI supports this new direct POST method of replication. Each replication feed shows which type of replication is in force and what speed (number of concurrent threads) is set for direct POST.
Platform CLI Expansion — The set of CLI commands for Swarm Platform server is extensively expanded and simplified to improve system administration. In addition to UEFI boot support and many other enhancements, Platform now implements an API to orchestrate rolling reboots of live Swarm storage clusters. The CLI provides broad control over the rolling reboot process with commands to pause, resume, skip specific nodes, and cancel rolling reboots altogether.
Documentation Reorganization — In response to client requests for more role-focused documentation and to unify similar topics across Swarm components, the Swarm documentation is fundamentally reorganized into three main phases and audiences: Deployment, Administration, and Development:
Deployment Planning Documentation — To better support planning for and implementing Swarm storage solutions, new guidance and numerous architectural diagrams are available in the new Swarm Deployment section of the documentation:
SwarmNFS Configuration UI Updates — The Swarm Storage UI has several improvements to support the latest releases of SwarmNFS.
...
* Mandatory update for those using erasure coding (EC) with 9.3.0 or higher.
Efficient Hardware Refreshes — Swarm Storage has significantly increased the efficiency of refreshing cluster hardware and retiring whole chassis. The processes for retiring volumes and cleaning up over-replication are optimized.
Filtering Response Headers — For MSPs and other direct-to-Internet deployments, Swarm Storage now offers response metadata filtering, for managing bandwidth and information flow. Swarm provides cluster settings for restricting which headers are returned for GET and HEAD responses, using either a whitelist or blacklist approach. Filtering the response headers best serves those needing to conserve bandwidth (eliminating as many bytes as possible when serving content) and those needing enhanced security (to reveal as little as possible about the content and its context).
Updating Custom Headers — Swarm allows management of an object's custom headers in bulk during updates. Adding the
preservequery argument to PUT or APPEND allows keeping the existing custom headers as well as add new ones; addingreplaceto an APPEND allows stripping off the existing custom headers as well as add new ones on the request.Improved Tracking — Swarm now keeps track of EC conversions and consolidations that are potentially and actually performed during HP cycles, and it publishes these statistics via SNMP. In addition, the gradual cleanup of the orphaned contents of removed bucket and domain objects is now tracked and reported in health report summaries.
Streamlined Metrics — To minimize the impact of metrics indices on Elasticsearch clusters, Swarm Metrics now has fewer shards in its schemas. After upgrade, it starts creating the new, improved indices.
Interactive Metadata Editing — The Content UI now allows users to edit object metadata directly in the UI. While applications have always been able to edit object metadata, this new interface enables users to correct and extend metadata from the convenience of a browser.
SwarmNFS 2.0 — Swarm 9.5 supports SwarmNFS 2.0, the second major release of SwarmNFS, which is a lightweight protocol converter that integrates Swarm object storage with NFS v4. SwarmNFS provides a traditional file interface to Swarm object storage for content generators who use native NFS-based applications to create, access, and manage that content while allowing for the same content to be created, accessed, and managed through modern cloud and object APIs such as S3 and SCSP. SwarmNFS integrates with the Storage UI for configuration of exports that map disks to buckets in Swarm domains.
SwarmNFS Interface through Gateway — The SwarmNFS storage interface now supports Content Gateway in addition to Direct to Swarm. The Cloud Security section of export configuration allows setting up the method best fitting the situation: Session Token (token admin credentials with expiration), Single User (user, password, and token), or Pass-through.
Swarm 9.4 — launched December 2017
9.0 (8.3) | 9.4 | 5.2 | 1.2 | 5.3 |
New Access Policy Editor — The Content UI includes a interactive policy editor for creating and updating access control policies for a provided tenant, domain, or bucket, greatly simplifying the process. The editor presents selections for quickly adding the most commonly needed policies (such as public read-only and authorized user full access) as well as options for designing granular access for users and groups. Safeguards help protect from unintended consequences, such as denying access to All Authorized Users.
Dynamic Subcluster Reassignment — Swarm Storage now supports dynamic reassigning of nodes to subclusters, so the nodes do not need to be restarted for subcluster assignments to take effect.
Elasticsearch Config Automation — Implementing Elasticsearch for Swarm requires making a set of configuration and settings file changes consistently across all nodes of the Elasticsearch cluster. The Elasticsearch package now provides a configuration script that automates these changes.
OSS Updates — Swarm is now based on the Debian 8 (jessie) distribution, which brings numerous operating system bug fixes and improvements. Many drivers and services are also updated: Linux Kernel to 4.4.84, Linux Firmware to 1.167, NTP to 4.2.8-p10, and Intel drivers ixgbe 5.2.3, igb 5.3.5.10, ixgbevf 4.2.1, e1000e 3.3.5.10, i40e 2.1.26, and i40evf 3.0.8.
S3 Compatibility — Gateway includes several new S3 protocol enhancements in order to track with Amazon S3 changes.
Swarm Hybrid Cloud — Swarm Hybrid Cloud is a new Microsoft Azure Marketplace solution for seamlessly converting files from an S3 object, Swarm object, or NFS-accessible file to Azure's native Blob format. It allows implementing hands-free replication of content from on-premises Swarm Storage into an Azure storage account, using a standard Swarm replication feed.
...
Metadata Typing Expanded — In addition to indexing content metadata as string types, Swarm now supports alternative representations — such as numeric, time, and geo-location — for the same metadata. This provides applications with powerful query capabilities when performing searching operations. For example, a metadata field value of "30.3675, -97.7878" can simultaneously be searched for as a string value or as a latitude/longitude geographic coordinate. Swarm automatically adds any supported type representations for metadata as it is indexed.
S3 Multipart Listing Compatibility Improvements — Some S3 applications naively codify formatting validations for multipart operations contrary to AWS S3 API guidance. In order to support these applications, Swarm now mimics the behavior of AWS S3 behavior with regard to the formatting of the opaque ETag string on multipart upload and listing operations to support these applications.
Volume Upgrades and Management Simplification — Swarm volumes that hold mostly small objects and are formatted by Swarm 7.5 or earlier can be automatically upgraded in-place in order to better handle objects <100KB. This upgrade expands the journal space and eliminates a "full" volume due to journal exhaustion. Prior to this improvement, an An administrator had to manually retire and reformat the old volume in order to expand the journal space prior to this improvement.
Performance and Stability Improvements — Load balancing of small objects (<100KB) onto newly added volumes is improved. Hot-plug add/remove of volumes is improved for rapidly occurring events on multiple volumes in a chassis. When using a replication feed, the target cluster's replicate-on-write policy is honored, which both reduces the load impact to the cluster and eliminates any time before full protection for the replicated objects.
Live Changes to Cluster Settings — The Cluster Settings page of Storage UI allows making dynamic changes to persisted cluster settings. These dynamic settings are previously only modified using SNMP.
...
Automatic EC Segment Optimization — Because design constraints may require applications to write EC objects with excessive segmentation, you can now configure the health processor to automatically consolidate these segments into an optimal number in the background.
Automatic Computation of Content-MD5 — For better SCSP and S3 application compatibility, a new Swarm setting automatically adds the Content-MD5 header on all writes. Previously, SCSP applications had to request this behavior on a per-transaction basis, and it can cause problems when sharing content with some S3 applications if not computed.
Composite Content-MD5 for Multipart Writes — In order Swarm computes and saves a composite MD5 header with an object to improve S3 multipart upload compatibility and to provide SCSP clients with end-to-end transfer validation with parallel writes, Swarm now computes and saves a composite MD5 header with an object. This complements the use of the Content-MD5 request header on the pieces of a multipart object so transmission integrity is validated by both parties during all phases of a multipart (parallel) write.
Synchronous Remote Object Replication — In some deployments, an application needs proof that critical content is replicated across independent clusters prior to completing a transaction. A new SCSP method is added that allows an application to request the immediate remote replication of an object and to receive positive confirmation that the object exists in the remote cluster.
Storage UI hosted on CSN — CSN includes the redesigned storage interface so transitioning from the legacy Swarm Admin Console while using the familiar CSN software stack is possible.
The new UI is includes the ability to view, configure, and manage the storage cluster.Historical Metrics hosted on CSN — CSN includes all components needed to allow the CSN to collect and store the operational time-series data produced by the storage cluster and used by the new UI. Deployments not making use of a separate Elasticsearch cluster for metadata searching can collect these metrics.
New Service Proxy — The Service Proxy (ServiceProxyServlet) works as a special instance of Gateway configured with additions and overrides to its configuration. The Service Proxy provides a proxy to a Swarm cluster, leveraging the same IDSYS authorization and authentication as the Content Gateway. With the Service Proxy, host the Storage UI and API from a server accessible to administrators and have Service Proxy manage communication with the storage cluster.
...