技術文章 » Centos 7 通過 targz 檔案安裝 Elastic Search 服務

Centos 7 通過 targz 檔案安裝 Elastic Search 服務

2023-06-08 21:00:46

區別於通過發行版自帶的倉庫, 介紹如何通過 targz 檔案安裝 Elastic Search 服務, 使用的 Linux 為 Centos 7

下載

https://www.elastic.co/downloads/elasticsearch

選擇 Linux x86_64, 下載 elasticsearch-8.8.0-linux-x86_64.tar.gz

安裝

解壓到 /opt/elasticsearch, 並加上軟鏈

tar xvf elasticsearch-8.8.0-linux-x86_64.tar.gz 

cd /opt/
sudo mkdir elasticsearch
cd elasticsearch/
sudo mv ~/backup/elasticsearch-8.8.0 .
sudo chown -R milton:milton elasticsearch-8.8.0/
sudo ln -s elasticsearch-8.8.0 latest

這個版本的 Elastic Search 自帶 JVM, 版本為 openjdk version "20.0.1" 2023-04-18

設定

可能需要修改的設定

# Use a descriptive name for your cluster:
#cluster.name: my-application
# Use a descriptive name for the node:
node.name: centos7001
# Add custom attributes to the node:
#node.attr.rack: r1
# Path to directory where to store the data (separate multiple locations by comma):
path.data: /home/milton/es_run/data
# Path to log files:
path.logs: /home/milton/es_run/logs
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
network.host: 192.168.9.10
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#http.port: 9200
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#discovery.seed_hosts: ["centos7001"]
# Bootstrap the cluster using an initial set of master-eligible nodes:
cluster.initial_master_nodes: ["centos7001"]
# For more information, consult the discovery and cluster formation module documentation.
# Allow wildcard deletion of indices:
#action.destructive_requires_name: false
xpack.security.enabled: false
  • cluster.name: my-application 叢集名稱
  • node.name 要改成當前伺服器的hostname
  • path.data: /somew/data 資料路徑
  • path.logs: /somewhere/logs 紀錄檔路徑
  • network.host: 192.168.123.123 監聽的網口, 預設只監聽127.0.0.1
  • http.port: 9200 監聽的埠, 預設為9200
  • discovery.seed_hosts: ["192.168.123.123"] 叢集主機列表, 和下面的cluster.initial_master_nodes必須寫一個, 不然啟動會報錯. 如果只是單節點, 這行可以註釋掉
  • cluster.initial_master_nodes: ["centos7001"] 啟動時初始化的參與選主的node 對應的 hostname, 要能解析為IP

node.name 和 cluster.initial_master_nodes, 可以填IP也可以填hostname, 但是要一致

系統設定

以下的設定用於解決下面的問題

  1. max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
  2. max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
  3. the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
  4. Transport SSL must be enabled if security is enabled. Please set [xpack.security.transport.ssl.enabled] to [true] or disable security by setting [xpack.security.enabled] to [false]

1. max file descriptors 65535

修改/etc/security/limits.conf (或者 /etc/security/limits.d/20-nproc.conf), 增加或修改為以下內容

*          soft    nofile    65535
*          hard    nofile    65535
*          soft    nproc     65535
*          hard    nproc     65535
root       soft    nproc     unlimited

需要重啟, 用 ulimit -n 檢查

2. vm.max_map_count 262144

修改/etc/sysctl.conf 或者 /etc/sysctl.d/99-sysctl.conf檔案，增加或修改為以下內容

vm.max_map_count=262144

3. the default discovery settings are unsuitable for production use

需要設定 discovery.seed_hosts，discovery.seed_providers，cluster.initial_master_nodes中的至少一個引數

  • discovery.seed_hosts: 叢集主機列表
  • discovery.seed_providers: 基於組態檔設定叢集主機列表
  • cluster.initial_master_nodes: 啟動時初始化的參與選主的node

修改組態檔 config/elasticsearch.yml, 設定以下兩項

discovery.seed_hosts: ["127.0.0.1"]
cluster.initial_master_nodes: ["node-1"]

4. Transport SSL must be enabled if security is enabled

修改組態檔 config/elasticsearch.yml, 增加

xpack.security.enabled: false

5. WARN: This node is a fully-formed single-node cluster

如果在紀錄檔中看到類似這樣的錯誤

[2023-06-09T07:29:43,781][WARN ][o.e.c.c.Coordinator      ] [centos7001] This node is a fully-formed single-node cluster with cluster UUID [6ejfGD71SVe6OpypK-1HmA], but it is configured as if to discover other nodes and form a multi-node cluster via the [discovery.seed_hosts=[192.168.123.123]] setting. Fully-formed clusters do not attempt to discover other nodes, and nodes with different cluster UUIDs cannot belong to the same cluster. The cluster UUID persists across restarts and can only be changed by deleting the contents of the node's data path(s). Remove the discovery configuration to suppress this message.

說明這是一個單節點的ES, 但是組態檔中設定其去發現另一個節點. 需要將 discovery.seed_hosts 中設定的節點去掉

執行

直接執行, 這樣會將紀錄檔直接輸出到控制檯

/opt/elasticsearch/latest/bin/elasticsearch

後臺執行, 在命令後加 -d -p pid-file, 在輸出一段控制檯紀錄檔後, 如果沒有報錯, 會轉入後臺執行

/opt/elasticsearch/latest/bin/elasticsearch -d -p /opt/elasticsearch/latest/logs/pid

停止

根據記錄的 pid 停止, 啟動時記錄用的哪個檔案, 這裡就用對應的檔案

pkill -F /opt/elasticsearch/latest/logs/pid

存取服務

瀏覽器開啟 http://192.168.123.123:9200/ 能看到ES的輸出, 就說明執行成功

{
  "name" : "centos70",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "8.8.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "c01029875a091076ed42cdb3a41c10b1a9a5a22f",
    "build_date" : "2023-05-23T17:16:07.179039820Z",
    "build_snapshot" : false,
    "lucene_version" : "9.6.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

查詢叢集執行狀況

curl -XGET "127.0.0.1:9200/_cat/health?v"

查詢叢集所有索引

$ curl -XGET "192.168.123.123:9200/_cat/indices?v"
health status index        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   commodity002 XIrCTL_XQq2vteuEflY6vA   1   1          0            0       247b           247b
yellow open   commodity001 Z-LKjzsuR8uMLgVlYYALEw   1   1          0            0       247b           247b
yellow open   commodity004 sSxEiwNBSvernMH6EYsEvw   1   1          0            0       247b           247b
yellow open   commodity003 JSRUndkHQ8mQVdTkN9eCPw   1   1          0            0       247b           247b

建立索引

不帶引數, ?pretty用於格式化響應的json

curl -X PUT "localhost:9200/commodity?pretty"

帶引數

curl -H 'Content-Type: application/json' -X PUT 'http://192.168.123.123:9200/commodity007?pretty' \
--data '{
    "settings": {
        "number_of_shards": 3,
        "number_of_replicas": 2
    }
}'

帶索引欄位,

curl -H 'Content-Type: application/json' -X PUT 'http://192.168.123.123:9200/commodity008?pretty' \
--data '{
  "settings": {
    "number_of_shards": 2,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "name":{
        "type": "text"
      },
      "studymodel":{
        "type": "keyword"
      },
      "price":{
        "type": "double"
      },
      "timestamp": {
         "type": "date",
         "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
      },
      "pic":{
        "type":"text",
        "index": false
      }
    }
  }
}'

對於巢狀存在的欄位, mappings是可以用層級的, 例如對 type1下的obj1的索引

{
  "mappings": {
    "type1": {
      "properties": {
        "obj1": {
          "type": "nested"
        }
      }
    }
  }
}

檢視索引欄位及設定

curl -X GET 'http://192.168.123.123:9200/commodity008?pretty'

往索引寫入內容

通過路徑指定 _id = 1, 對同一個 _id可以再次呼叫進行更新, 結果中的_version會遞增

curl --location --request PUT 'http://192.168.123.123:9200/commodity008/_doc/1?pretty' \
--header 'Content-Type: application/json' \
--data '{
    "name": "commodity008001",
    "studymodel": "202306",
    "price": 123.12,
    "timestamp": "2023-05-25 19:11:35",
    "pic": "23/06/01/a123b1fde0428.jpg"
}'

查詢

可以通過URL路徑區分不同索引

  • /_search 所有索引
  • /commodity008/_search commodity008索引
  • /commodity007,commodity008/_search commodity007 和 commodity008
  • /commodity*/_search 以 commodity 開頭的索引

查詢所有索引下的內容

curl -X GET 'http://192.168.123.123:9200/_search?pretty'

查詢一個索引下的內容

curl -X GET 'http://192.168.123.123:9200/commodity008/_search?pretty'

帶條件查詢

curl -H 'Content-Type: application/json' -X GET 'http://192.168.9.10:9200/commodity008/_search?pretty=null' \
--data '{
    "query" : {
        "match" : {
            "name": "commodity008001"
        }
    }
}'

帶偏移和結果數量, 請求加上 from 和 size 引數

{
  "from":10,
  "size":20,
  "query":{
    "match_all": {}
  }
}

排序, 請求加上 sort 引數

{
  "sort":[{"year":"desc"}],
  "query":{
    "match_all": {}
  }
}

限制返回的欄位, 請求加上 _source 欄位

{
  "_source":["title"],
  "query":{
    "match_all": {}
  }
}

結果格式

{
    "took": 422,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "commodity008",
                "_id": "1",
                "_score": 1.0,
                "_source": {
                    "name": "commodity008001",
                    "studymodel": "202307",
                    "price": 123.53,
                    "timestamp": "2023-05-25 19:11:35",
                    "pic": "23/06/01/a123b1fde0428.jpg"
                }
            },
            ...
        ]
    }
}

參考