Elastic Search

elasticsearch 7.2.0

Download

Download elasticsearch from elastic and extract it.

Config

change config/elasticsearch.yml to this settings.

cluster.name: CLUSTER_NAME

node.name: node-1

network.host: 0.0.0.0

discovery.seed_hosts: ["127.0.0.1", "server(1)-ip", "server(2)-ip", ..., "server(n)-ip"]

cluster.initial_master_nodes: ["node-1", "node-2", ..., "node-n"]

Run

nohup bin/elasticsearch &

exporter

wget https://github.com/justwatchcom/elasticsearch_exporter/releases/download/v1.1.0/elasticsearch_exporter-1.1.0.linux-amd64.tar.gz
tar xzf elasticsearch_exporter-1.1.0.linux-amd64.tar.gz
mv elasticsearch_exporter-1.1.0.linux-amd64/ elasticsearch_exporter
mv elasticsearch_exporter/ /usr/local/bin
cd /etc/systemd/system
nano elastic_exporter.service

after opening elastic_exporter.service file with nano, write the following into it:

[Unit]
Description=Prometheus
Documentation=https://github.com/justwatchcom/elasticsearch_exporter
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=root
Group=root
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/elasticsearch/elasticsearch_exporter \
  --es.all \ #If true, query stats for all nodes in the cluster, rather than just the node we connect to.
  --es.indices \ #If true, query stats for all indices in the cluster.
  --es.shards \ #If true, query stats for all indices in the cluster, including shard-level stats
  --web.listen-address=:9114 \
  --es.uri=http://slave3:9200 # this should be a node that is alive and has elasticsearch

SyslogIdentifier=es_exporter
Restart=always

[Install]
WantedBy=multi-user.target

now you are ready to start the service:

systemctl start elastic_exporter.service
#check if it is started with following command:
systemctl status elastic_exporter.service

finally you should connect to master and change prometheus.yml file and add this exporter to it:

cd /usr/local/prometheus
nano prometheus.yml

copy this part to your yml file or if it already exists, just edit it:

- job_name: 'es_exporter'
    static_configs:
      - targets: ['master:9114'] # this is assumed that exporter is on master and only one is enough 

Useful links

https://www.elastic.co/guide/en/elasticsearch/reference/7.2/deb.html#deb-repo https://www.elastic.co/guide/en/elasticsearch/reference/7.x/modules-discovery-bootstrap-cluster.html

Some Queries

DELETE sites/
{
  "query": {
    "match_all": {}
  }
}

PUT sites-en
{
   "settings":{
      "index":{
         "number_of_shards":6,
         "number_of_replicas":1
      },
      "analysis":{
         "tokenizer":{
            "comma":{
               "type":"pattern",
               "pattern":","
            }
         },
         "filter":{
            "english_stop":{
               "type":"stop",
               "stopwords_path":"stopwords/english.txt"
            },
            "en_US":{
               "type":"hunspell",
               "language":"en_US"
            }
         },
         "analyzer":{
            "en_US":{
               "tokenizer":"standard",
               "filter":[
                  "lowercase",
                  "en_US",
                  "english_stop"
               ]
            },
            "tag_analyzer":{
               "type":"standard",
               "tokenizer":"comma",
               "filter" : [
                  "trim"
               ]
            }
         }
      }
   },
   "mappings":{
      "properties":{
         "title":{
            "type":"text",
            "analyzer":"en_US",
            "fields":{
               "auto-complete":{
                  "type":"search_as_you_type"
               }
            }
         },
         "link":{
            "type":"keyword"
         },
         "domain":{
            "type":"keyword"
         },
         "page-rank":{
            "type":"double",
            "fields":{
               "rank_feature":{
                  "type":"rank_feature"
               }
            }
         },
         "keywords":{
            "type":"text",
            "analyzer":"en_US"
         },
         "tags":{
            "type":"text",
            "analyzer":"tag_analyzer"
         },
         "text":{
            "type":"text",
            "term_vector":"yes",
            "analyzer":"en_US",
            "fields":{
               "raw":{
                  "type":"keyword"
               }
            }
         }
      }
   }
}

PUT sites-ar
{
   "settings":{
      "index":{
         "number_of_shards":"6",
         "number_of_replicas":"1"
      }
   },
   "mappings":{
      "properties":{
         "domain":{
            "type":"keyword"
         },
         "keywords":{
            "type":"text",
            "analyzer":"arabic"
         },
         "link":{
            "type":"keyword"
         },
         "page-rank":{
            "type":"double",
            "fields":{
               "rank_feature":{
                  "type":"rank_feature"
               }
            }
         },
         "text":{
            "type":"text",
            "fields":{
               "raw":{
                  "type":"keyword"
               }
            },
            "analyzer":"arabic"
         },
         "title":{
            "type":"text",
            "fields":{
               "auto-complete":{
                  "type":"search_as_you_type"
               }
            },
            "analyzer":"arabic"
         }
      }
   }
}

Analyzers :

persian, english, arabic, russian, spanish

lang Detector :

fa, en, ar, ru, es