Swarm integrates Elasticsearch and extends the Swarm API with commands for querying Swarm objects in metadata. Through this feature, Swarm indexes object metadata in near real-time and allows performing ad hoc searches (via https://perifery.atlassian.net/wiki/spaces/public/pages/2443821835) on the attributes and metadata of your stored objects.

Swarm uses Elasticsearch servers for its metadata searching operations. You can deploy these servers for high availability and horizontal scaling. Although the high availability of the search cluster is not needed for the high availability of the storage cluster, you may need it to service third-party analytics applications.

You can return the results as JSON or XML, which you can import into your third-party analytics applications.

See also these sections:

• https://perifery.atlassian.net/wiki/spaces/public/pages/2443805406

•  (Configuration and Administration)

Search Components

The search infrastructure includes these components:

• Swarm Storage cluster is connected to the Elasticsearch servers through a Search Feed.

• Search feed(s) transmit the metadata from the storage cluster. Feeds iterate over data on storage nodes and use intermittent channel connections to distribute data to one or more configured destinations, including metadata search servers. See  

• Elasticsearch servers index the metadata and service search requests. This metadata can be reconstructed from the storage cluster if needed.

• Metrics curator service can be installed on one of the Elasticsearch servers or another system running RHEL/CentOS 7 or 8.

• Client applications access the Swarm cluster through SCSP commands.