Swarm integrates Elasticsearch and extends the Swarm API with commands for querying Swarm objects in metadata. Through this feature, Swarm indexes object metadata in near real-time and allows performing ad hoc searches (via Search Query Arguments) on the attributes and metadata of your stored objects.

Swarm uses Elasticsearch servers for its metadata searching operations. You can deploy these servers for high availability and horizontal scaling. Although the high availability of the search cluster is not needed for the high availability of the storage cluster, you may need it to service third-party analytics applications.

Important

For production-level responsiveness and redundancy, deploy at least three search servers. Follow the Hardware Requirements for Elasticsearch

You can return the results as JSON or XML, which you can import into your third-party analytics applications.

Search Components

The search infrastructure includes these components:

Swarm Storage cluster is connected to the Elasticsearch servers through a Search Feed.
Search feed(s) transmit the metadata from the storage cluster. Feeds iterate over data on storage nodes and use intermittent channel connections to distribute data to one or more configured destinations, including metadata search servers. See Managing Feeds

Tip

You can configure additional feeds that use the same Elasticsearch cluster because Swarm uniquely names each search feed index; Plan for doubling or tripling the space demands on that server.

Elasticsearch servers index the metadata and service search requests. This metadata can be reconstructed from the storage cluster if needed.
Client applications access the Swarm cluster through SCSP commands.

Best Practice

Devote the search cluster to Swarm-only usage, and do not store non-Swarm data in your search installation.