Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents
minLevel1
maxLevel2
outlinefalse
typelist
printablefalse

...

Info

Info

Elasticsearch was previously used to store Historical Metrics but that has moved to Prometheus starting with Swarm 14 . Gateway Content Metering stores csmeter indices in Elasticsearch, but his that does not impact Elasticsearch hardware requirements as much as a Swarm Search Feed.

This software requires one or more servers running RHEL/CentOS 7 or 8 Linux. Although Elasticsearch runs on other Linux platforms, DataCore currently provides support and testing for these versions. The Elasticsearch version provided with the Swarm distribution is supported.

...

Following are overall best practices, with hardware recommendations from Elasticsearch:

  • Provision the machines with CPUs with at least 4 cores and 64 GB memory. Between faster processors or more cores, choose more cores.

  • Choose Use Solid-state drives (SSD), to boost performancepreferably local not SAN and never NFS. This is critical for S3, especially rapid small object writes, and for the listing of buckets with millions of objects.

  • Perform the following if Let Support know if you are using hard disks which do not handle concurrent I/O as well as SSDs:

    • Select high-performance server disks.

    • Use RAID 0 with a writeback cache.

    • Set index.merge.scheduler.max_thread_count to 1, to prevent too many merges from running at once

      Code Block
      languagebash
      curl -X PUT <ES_SERVER>:9200/<SWARM_SEARCH_INDEX>/_settings \
      	-d '{ “index”: {“merge.scheduler.max_thread_count”: 1}}'
  • As with the storage cluster, choose similar, moderate configurations, for balanced resource usage.

...

  • 64 GB RAM per machine is optimal (recommended by Elasticsearch).

  • Dedicate half of total RAM to the Java Virtual Machine (JVM) running Elasticsearch, but do not exceed 31 GB, for best performance. Heap assignment is automatic with 7.17.

  • Disable swapping of the Elasticsearch image. (For ES 2.3.3, allow in-memory caching of all shards on the server.)

Optimal performance can be achieved by adding adequate RAM in the ES servers to store all database shards in memory. Take steps to disable or mitigate swapping. Memory page swapping on an ES server impacts Elasticsearch performance.

...

Elasticsearch heavily utilizes disks, so higher throughput results in more stable nodes. Follow these Elasticsearch guidelines for optimizing disk I/O:

  • Use SSDs: SSDs boost are critical for performance. With SSDs, verify the OS I/O scheduler is configured correctly.

  • Use RAID 0: Striped RAID increases disk I/O, at the expense of potential failure if a disk dies. Do not use mirrored or parity RAIDS, because replicas provide this functionality.

  • Do not use remote-mounted storage: such as NFS or SMB/CIFS; the latency negatively impacts performance.

  • Avoid virtualized storage, such as a SAN or EBS (Amazon Elastic Block Store). Even when SSD-backed it is often slower than local instance storage and it conflicts with the purpose of replicas and sharding.

...