Use Cases and Architectures

Most use cases for Swarm involve ingesting petabytes of unstructured data, such as image, video, and document files, which must be secured, preserved, searched, and retrieved on demand.

Active Archive: Video evidence, medical imaging, and cultural media
Cloud: Cloud services, hosting (multi-tenant) and backup to the cloud
Content Delivery: Social media (millions of photos per day), streaming video (millions of videos), and content publishing (millions of images)
Big Data: Evidence analysis, medical insurance records and analysis, IoT/M2M and analytics
Compliance: Legal documents, court materials, and digital evidence

Swarm supports many usage scenarios based on four fundamental access methods:

Direct Access	Native (SCSP API)	Native client/application integration using a vendor API (RESTful HTTP 1.1 compliant) The application works directly with the object store
Web Access	Content Gateway (S3 API)	Data in the object store is presented via web browser (Content UI) S3 endpoint is provided Support for actions such as upload, download, and browse Back-end access to the object store are native API calls
File Protocol Gateway	SwarmFS (to Native SCSP)	Provides translation of traditional file protocols (such as NFS and SMB) to object storage protocols Usually translates to object store native API Used as a “drop box” target for clients or applications coded to work with traditional filers Advanced protocol gateway support for manipulation of metadata, in addition to data via traditional utilities (such as shell sessions) Placement into object store supports alternate access methods (SCSP or S3) and metadata queries, listings, and collections
Automated Tiering	FileFly	Application/agent integration (native API integration) Agents exist on data sources (such as filers) The relationship between local file reference and data stored at object tier is maintained by agent software Data is moved from local to object tier based on policy (scheduled or ad hoc) Retrieval of data when client requests access is automatic and transparent

These are common architectures for object storage:

Archiving

Medium- to long-term storage
“Write once, read rarely”
Library of unstructured data (documents, graphics, pictures, videos)
Query and list, based on metadata tags
Conduct “Data Lake” analysis (by pooling a vast amount of raw data in a native format)

Data Tiering

Relocation of data from traditional filers to object storage
Scheduled tiering based on policy
Automated recall when an access request is made
Transparent access to the end user
“Cheap and deep” object-store tier to reduce filer expansion costs

FileFly and Virtualization

Remote Replication and Disaster Recovery

Automated replication from a local object store to a remote object store
Data is usually populated in a local store, then replicated to remote or DR
“Hot” sites can also act as replication targets or DR for each other
Can be whole-site replication or policy-based (per domain)
Varying complexity in replication topologies supported (site-to-site, M to N, single or bi-directional)

Dual Site with Single Interface

Dual Site with Dual Interface

Managed Service (“Storage as a Service”)

Storage protocol endpoints made available to service subscribers
Support for multiple RESTful access protocols
SSL/TLS
Provides authentication and authorization
Allows for metering and billing
Supports quota control
Multi-tenancy (individuals, business organizations, business units)

Hybrid Cloud (Local Storage with Cloud)

A local object store integrated with a cloud service endpoint (such as Azure)
“Copy to Cloud” for backup and/or publication of data
“Retrieve from Cloud” for data recovery
Lower CapEx when meeting backup, replication, or DR requirements