Setup Elasticsearch Cluster

Partition Size

The following is the set of requirements for partitioning specific to OS capability:

Device 

Mount 

Partition Type 

File System Type 

Size 

Options 

Device 

Mount 

Partition Type 

File System Type 

Size 

Options 

/dev/sda1 

/boot 

GPT 

ext4 

10 GiB 

Bootable 

/dev/sda2 

GPT 

ext4 

20 GiB 

NA 

/dev/sda3 

/var/lib

GPT 

ext4 

>=120GiB (expand to Fill Disk) 

NA 

/dev/sda4

/var/log

GPT

ext4

15 GiB

NA

Important

It is recommended to run Elasticsearch on SSDs.

CentOS Setup

See here to set up time synchronization for RHEL/CentOS 7 or RHEL/Rocky Linux 8.

The below table is an example:

Minimal Install Mode

Hostname

  • es75-01.datacore.internal

  • es75-02.datacore.internal

  • es75-03.datacore.internal

Private Network

IP Address

  • 192.168.9.21/24

  • 192.168.9.22/24

  • 192.168.9.23/24 

Network Gateway

172.16.33.1

NTP

  • 172.16.33.4

  • 172.16.33.5

Download the Latest Swarm Bundle Package

For OVF-based installation, see https://perifery.atlassian.net/wiki/spaces/public/pages/3173613575 to download the latest Swarm VMWare bundle.

For RPM-based installations, see https://perifery.atlassian.net/wiki/spaces/public/pages/3041394691 to download the latest Swarm bundle.

 

Install Elasticsearch

Refer to the following steps:

  1. Update CentOS.

    yum -y update
  2. Install EPEL.

    yum -y install epel-release
  3. Install NTP Server.

    yum -y install ntp systemctl enable ntpd hwclock --systohc systemctl restart ntpd
  4. Unzip the Swarm bundle package to /root/datacore.

  5. Obtain the latest Elasticsearch RPM and Swarm Search RPM from the downloaded Swarm bundle package.

  6. Install Swarm RPM public key included with the distribution bundle.

  7. Install and configure the Elasticsearch components on each Elasticsearch node.

Configure a 3-Node Elasticsearch Cluster for Swarm

Verify the Data Folder for Swarm Elasticsearch

  1. The default data folder for Swarm Elasticsearch on all nodes is /var/lib/elasticsearch. This folder must not be on the boot partition to verify Elasticsearch cannot fill the boot partition.

  2. Verify ownership of that folder. Use ls -la /var/lib/elasticsearch. It should be elasticsearch:elasticsearch.

  3. Configure the firewall to allow Elasticsearch vi /etc/firewalld/services/elasticsearch.xml.

  4. Restart the firewall service:

  5. Add Elasticsearch firewall rule permanently:

  6. Reload the firewall service:

Configure Elasticsearch Cluster

The following information is about the example deployment used to configure a 3-node Elasticsearch cluster:

Role 

Private Network

vCPU 

RAM 
(GB) 

Disk Size (GB) 

Remark 

ES75-01 

192.168.9.21 

100 

Elasticsearch 7.5.2 Cluster (SSD) 

ES75-02 

192.168.9.22 

100 

Elasticsearch 7.5.2 Cluster (SSD) 

ES75-03 

192.168.9.23 

100 

Elasticsearch 7.5.2 Cluster (SSD) 

  1. Run the Swarm Elasticsearch configuration script /usr/share/caringo-elasticsearch-search/bin/configure_elasticsearch_with_swarm_search.py on the first Elasticsearch node (e.g., es75-01).

  2. Enter the Elasticsearch cluster name (e.g., swarm-es75).

  3. Enter the list of all Elasticsearch server names or private network IP addresses.

  4. Enter the name of the current Elasticsearch node.

    The configuration script generates a custom Elasticsearch configure file for each node in the Elasticsearch cluster:

    1. The other nodes’ files are /etc/elasticsearch/elasticsearch.yml.<node-ip>

    2. The current node’s file is /etc/elasticsearch/elasticsearch.yml

  5. Verify the Elasticsearch configuration files of all Elasticsearch nodes.

  6. Copy the generated Elasticsearch configure file to other nodes.

  7. SSH to other nodes of the Elasticsearch cluster to run the configuration script.

    1. On ES75-02 node,

    2. On ES75-03 node,

See https://perifery.atlassian.net/wiki/spaces/public/pages/2443809957 to update the environment and log settings. Perform a rolling restart of Elasticsearch services to apply changes once all updates are made.
See to know the hardware requirements for Elasticsearch.

Only for the New Setup of Elasticsearch Cluster

Refer to the below steps if Elasticsearch nodes are unable to join the cluster and return the ‘master not discovered exception’ error:

  1. Stop Elasticsearch service on all nodes.

  2. Delete files/folders inside /var/lib/elasticsearch/ on all nodes.

  3. Restart Elasticsearch nodes one by one.

© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.