【Docker】之安装 ElasticSearch集群
准备工作
单节点配置
拉取镜像
docker pull elasticsearch:7.17.6
docker pull kibana:7.17.6创建目录
# 创建挂载目录
mkdir -p /root/docker/elasticsearch/{config,data,logs,plugins}
mkdir -p /root/docker/elasticsearch/config/certs
mkdir -p kibana/config
# 创建配置文件
echo "http.host: 0.0.0.0" > /root/docker/elasticsearch/config/elasticsearch.yml
# 给挂载目录添加权限 chmod 777 -R /data/
chmod -R 777 /root/docker/elasticsearch/data//root/docker/elasticsearch/kibana/config/kibana.yml
#
# ** THIS IS AN AUTO-GENERATED FILE **
#
# Default Kibana configuration for docker target
server.name: kibana
server.host: "0.0.0.0"
server.shutdownTimeout: "5s"
elasticsearch.hosts: [ "http://192.168.3.202:9200" ]
monitoring.ui.container.elasticsearch.enabled: true
# 此处设置elastic的用户名和密码
elasticsearch.username: "elastic"
elasticsearch.password: "123456"
# 设置kibana中文显示
i18n.locale: "zh-CN"
/root/docker/elasticsearch/config/elasticsearch.yml
http.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization
xpack.license.self_generated.type: basic
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true构建容器并运行
docker run -d -p 9200:9200 -p 9300:9300 \
-e "discovery.type=single-node" \
-e ES_JAVA_OPTS="-Xms84m -Xmx512m" \
-v /root/docker/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \
-v /root/docker/elasticsearch/data:/usr/share/elasticsearch/data \
-v /root/docker/elasticsearch/plugins:/usr/share/elasticsearch/plugins \
-v /root/docker/elasticsearch/logs:/usr/share/elasticsearch/logs \
--name elasticsearch \
elasticsearch:7.17.6 或者直接利用docker-compose.yml文件，可以在容器运行后拷贝出配置文件等进行编辑然后挂载
version: '3.9'
services:
es01:
container_name: es01
image: elasticsearch:7.17.6
ports:
- "9200:9200"
- "9300:9300"
volumes:
- /root/docker/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- /root/docker/elasticsearch/data:/usr/share/elasticsearch/data
- /root/docker/elasticsearch/logs:/usr/share/elasticsearch/logs
environment:
- TZ=Asia/Shanghai
# - "ES_JAVA_OPTS=-Xms1024m -Xmx2048m"
- "discovery.type=single-node"
restart: always
networks:
- elastic
b01:
depends_on:
- es01
image: kibana:7.17.6
container_name: kib01
ports:
- 5601:5601
environment:
ELASTICSEARCH_URL: http://es01:9200
ELASTICSEARCH_HOSTS: http://es01:9200
volumes:
- /root/docker/elasticsearch/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml
networks:
- elastic
networks:
elastic:
driver: bridge设置授权访问的账号和密码
开启x-pack验证,/root/docker/elasticsearchconfig/elasticsearch.yml 中添加如下配置
# 添加如下内容
# 配置X-Pack
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization
xpack.security.enabled: true
xpack.license.self_generated.type: basic
xpack.security.transport.ssl.enabled: true
或
bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12重启elasticsearch服务
# systemctl restart elasticsearch
docker restart es01设置elasticsearch密码
# 进入容器
docker exec -it es01 /bin/bash
# 重置密码
# bin/elasticsearch-reset-password -u elastic
# 自动生成好几个默认用户和密码
bin/elasticsearch-setup-passwords auto
# 如果想手动生成密码，则使用
bin/elasticsearch-setup-passwords interactive 提示是否重置，输入y，控制台会打印新密码，请记住这个密码，稍后要用到
因为需要设置 elastic，apm_system，kibana，kibana_system，logstash_system，beats_system，remote_monitoring_user 这些用户的密码，故这个过程比较漫长，耐心设置；
修改elasticsearch密码
- 知道原密码
下面的示例为将elastic的密码设置为"123456"。
或# password 后为新密码, 控制台提示输入老密码 curl -H "Content-Type:application/json" -XPOST -u elastic 'http://127.0.0.1:9200/_xpack/security/user/elastic/_password' -d '{ "password" : "123456" }'# 不提示直接改 curl -XPOST --u elastic:现密码 "http://localhost:9200/_security/user/elastic/_password" -H 'Content-Type: application/json' -d '{"password":"newpassword"}' - 忘记原密码
修改配置文件config/elasticsearch.yml，注释掉支持x-pack的xpack.security.enabled: true配置，保存退出；
重启Elasticsearch
使用curl查看当前Elasticsearch的索引：
控制台打印，多了一个curl -XGET "127.0.0.1:9200/_cat/indices" -H 'Content-Type: application/json'.security-7的索引，删除``.security-7的索引.
有如下打印，表示已成功删除：curl -XDELETE 127.0.0.1:9200/.security-7
现在，重新设置elasticsearch密码。{"acknowledged":true} - 用elasticsearch-users操作
找回集群密码, 在Elasticsearch的安装目录下，有一个 utility 叫做elasticsearch-users,我们可以使用这个工具找回我们的用户密码
首先，我们来创建另外一个超级用户newadmin：
我们接下来使用这个刚刚创建的超级用户来针对原有的bin/elasticsearch-users useradd newadmin -p password -r superuserelastic用户进行重置密码：
在上面，我们重置用户curl -s --user newadmin:password -XPUT "http://localhost:9200/_xpack/security/user/elastic/_password?pretty" -H 'Content-Type: application/json' -d '{"password":"password1"}'elastic的密码为password1。我们可以通过如下的命令来检查我们是否可以通过这个重置的密码来对集群进行访问：curl --user elastic:password1 -X GET "http://localhost:9200?pretty"$ curl --user elastic:password1 -X GET "http://localhost:9200?pretty" { "name" : "liuxg", "cluster_name" : "elasticsearch", "cluster_uuid" : "-vtUaRdHSnGJb5XdGC32bA", "version" : { "number" : "7.13.0", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "5ca8591c6fcdb1260ce95b08a8e023559635c6f3", "build_date" : "2021-05-19T22:22:26.081971330Z", "build_snapshot" : false, "lucene_version" : "8.8.2", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" }
验证
浏览器直接访问http://127.0.0.1:9200,会出现输入用户名、密码的弹窗，输入elastic和密码后，才能看到elasticsearch信息；
kinana.yml 修改
# 末尾增加
elasticsearch.username: "elastic"
elasticsearch.password: "123456"多节点集群配置
创建目录
mkdir -p /root/docker/elasticsearch/node-1/{data,logs,config,plugins}
mkdir -p /root/docker/elasticsearch/node-2/{data,logs,config,plugins}
mkdir -p /root/docker/elasticsearch/node-3/{data,logs,config,plugins}
mkdir -p /root/docker/elasticsearch/kibana/config
cd /root/docker/elasticsearch/
# 修改每个节点文件夹下data目录的权限(不修改后面会报错,权限不够)
chmod 777 ./node-1/data/ -R
chmod 777 ./node-1/logs/ -R
chmod 777 ./node-1/config/ -R编写配置文件(elasticsearch.yml)
新建/root/docker/elasticsearch/node-1/config/elasticsearch.yml
chmod +x node-1/config/elasticsearch.yml#-------------------------------------------
# es01
# 集群的名称(各节点*集群名称*需要保持一致)
cluster.name: "docker-cluster"
# # # 节点的名称
node.name: node-1
# # # 此节点是否可以用作master节点
node.master: true
# # # 此节点是否是存储节点
node.data: true
# # # 此节点是否是预处理节点 如果是master节点的话 建议这里是true
node.ingest: true
#设置主机IP 0.0.0.0时外网可以访问到
network.host: 0.0.0.0
# 配置端口
http.port: 9200
# 集群通信端口
transport.port: 9300
#集群内节点信息 每个节点会共享自己的此参数
discovery.seed_hosts: ["es01:9300","es02:9300","es03:9300"]
#集群的master候选节点目录。只有在初始化的时候才生效。
#这里只写node-1 并且配置这个参数 是用于快速搭建集群。集群已启动自动node-1 是master
cluster.initial_master_nodes: ["node-1"]
#cross 跨域访问 配置这个之后 head就可以用了
http.cors.enabled: true
http.cors.allow-origin: "*"#-------------------------------------------
# es02
# 集群的名称(各节点*集群名称*需要保持一致)
cluster.name: "docker-cluster"
# # # 节点的名称
node.name: node-2
# # # 此节点是否可以用作master节点
node.master: true
# # # 此节点是否是存储节点
node.data: true
# # # 此节点是否是预处理节点 如果是master节点的话 建议这里是true
node.ingest: true
#设置主机IP 0.0.0.0时外网可以访问到
network.host: 0.0.0.0
# 配置端口
http.port: 9200
# 集群通信端口
transport.port: 9300
#集群内节点信息 每个节点会共享自己的此参数
discovery.seed_hosts: ["es01:9300","es02:9300","es03:9300"]
#集群的master候选节点目录。只有在初始化的时候才生效。
#这里只写node-1 并且配置这个参数 是用于快速搭建集群。集群已启动自动node-1 是master
cluster.initial_master_nodes: ["node-1"]
#cross 跨域访问 配置这个之后 head就可以用了
http.cors.enabled: true
http.cors.allow-origin: "*"#-------------------------------------------
# es03
# 集群的名称(各节点*集群名称*需要保持一致)
cluster.name: "docker-cluster"
# # # 节点的名称
node.name: node-3
# # # 此节点是否可以用作master节点
node.master: true
# # # 此节点是否是存储节点
node.data: true
# # # 此节点是否是预处理节点 如果是master节点的话 建议这里是true
node.ingest: true
#设置主机IP 0.0.0.0时外网可以访问到
network.host: 0.0.0.0
# 配置端口
http.port: 9200
# 集群通信端口
transport.port: 9300
#集群内节点信息 每个节点会共享自己的此参数
discovery.seed_hosts: ["es01:9300","es02:9300","es03:9300"]
#集群的master候选节点目录。只有在初始化的时候才生效。
#这里只写node-1 并且配置这个参数 是用于快速搭建集群。集群已启动自动node-1 是master
cluster.initial_master_nodes: ["node-1"]
#cross 跨域访问 配置这个之后 head就可以用了
http.cors.enabled: true
http.cors.allow-origin: "*"编写docker-compose.yml配置文件
version: "3.9"
services:
es01:
image: elasticsearch:7.17.6
container_name: es01
privileged: true
environment:
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- TZ=Asia/Shanghai
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./node-1/data:/usr/share/elasticsearch/data
- ./node-1/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
ports:
- 9201:9200
- 9301:9300
networks:
- elastic
es02:
image: elasticsearch:7.17.6
container_name: es02
environment:
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- TZ=Asia/Shanghai
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./node-2/data:/usr/share/elasticsearch/data
- ./node-2/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
ports:
- 9202:9200
- 9302:9300
networks:
- elastic
es03:
image: elasticsearch:7.17.6
container_name: es03
environment:
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- TZ=Asia/Shanghai
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./node-3/data:/usr/share/elasticsearch/data
- ./node-3/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
ports:
- 9203:9200
- 9303:9300
networks:
- elastic
kibana:
image: kibana:7.17.6
container_name: kibana
restart: always
ports:
- 5601:5601
volumes:
- ./kibana/config:/usr/share/kibana/config
environment:
I18N_LOCALE: zh-CN
ELASTICSEARCH_URL: http://es01:9201
ELASTICSEARCH_HOSTS: '["http://es01:9201","http://es02:9202","http://es03:9203"]'
ELASTICSEARCH_SSL_VERIFICATIONMODE: none
ELASTICSEARCH_USERNAME: elastic
ELASTICSEARCH_PASSWORD: abc
networks:
- elastic
networks:
elastic:
external: true查看集群状态
http://IP:9201/_cluster/health
使用X-Pack设置授权加密
按照Elasticsearch的要求，如果我们在docker的环境中启动xpack.security.enabled，我们必须也启动xpack.security.transport.ssl.enabled。否则，我们将会看到如下的错误信息：
X-Pack 是 ElasticSearch 的一个插件，这个插件将提供与ElasticSearch来往的安全性。通过安装这个插件，我们就可以对 ElasticSearch 的集群节点生成证书，配置服务访问密码，以及使用TLS来确保HTTP客户端与集群之间的通信是加密的。
生成证书
docker exec -it es01 bash 进入容器后，前往工作目录下（即/usr/share/elasticsearch），为Elasticearch集群创建一个证书颁发机构。使用elasticsearch-certutil命令输出一个默认名为elastic-stack-ca.p12的PKCS#12密钥存储库文件，它包含CA的公共证书和用于为每个节点签名证书的私钥。
cd /usr/share/elasticsearch
./bin/elasticsearch-certutil ca集群中的任意一个节点生成证书和私钥
./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12此时已生成证书elastic-certificates.p12，将该证书复制到每一个节点的config/certs目录下
上面的命令将使用我们的 CA 来生成一个证书 elastic-certificates.p12:
执行exit退出容器，我们把上面的 elastic-certificates.p12 证书移至./elasticsearch/config/certs文件夹。
docker cp es01:/usr/share/elasticsearch/elastic-certificates.p12 ./elasticsearch/config/certs
sudo chmod -R 777 ./elasticsearch/config
docker-compose down在docker-compose.yaml配置好证书映射
volumes:
- ./elasticsearch/config/certs:/usr/share/elasticsearch/config/certs修改config/elasticsearch.yml来使用加密授权
cluster.name: "docker-cluster"
network.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type
xpack.license.self_generated.type: basic
xpack.security.enabled: true
# 传输层通信:传输协议用于Elasticsearch节点之间的内部通信
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
# HTTP层通信:客户端到Elasticsearch集群的通信
xpack.security.authc.api_key.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12
xpack.security.http.ssl.verification_mode: certificate
xpack.monitoring.collection.enabled: false verification_mode 我们选择certificate，这个模式不会去检查证书的CN,只验证证书是否是信任机构签名的即可.如果我们需要验证,并且配置了IP，则需要把这个模式该为full
如果证书是PEM格式，则使用下方配置
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /home/es/config/node01.key
xpack.security.transport.ssl.certificate: /home/es/config/node01.crt
xpack.security.transport.ssl.certificate_authorities: [ "/home/es/config/ca.crt" ]问题
elasticsearch查看当前集群中哪个节点是主节点
浏览器输入 : http://ip:9200/_cat/nodes?v
http://ip:9200/_cat/nodes?v
# 含有 * 的代表当前主节点
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
172.16.16.188 52 99 5 2.59 1.70 1.45 mdi - elastic3
172.16.16.187 32 99 5 0.99 0.99 1.19 mdi - elastic2
172.16.16.231 69 99 4 0.87 1.00 1.03 mdi - elastic4
172.16.21.28 57 98 4 2.45 1.18 1.05 mdi - elastic5
172.16.21.29 51 99 4 1.05 1.06 1.08 mdi - elastic6
172.16.16.186 55 99 5 1.33 1.35 1.38 mdi * elastic1 其中 master 列为 * 号表示该节点为主节点；如上所示为最后一行的 elastic1 为 master 节点。