window docker-compose elk搭建

elk介绍

elk 是 elasticsearch、logash、kibana的结合。

elasticsearch的功能：

搜索、全文检索、分析数据、处理海量数据PB,对海量数据进行近实时的处理(ES可以自动将海量数据分散到多台服务器上去存储和检索)、高可用高性能分布式搜索引擎数据库。

elasticsearch的应用场景：

网页搜索，新闻搜索，商品标签，日志收集分析展示

开发环境

windows(version=11)

软件

docker-desktop

docker --version

Docker version 26.1.1, build 4cf5afa

docker-compose --version

Docker Compose version v2.27.0-desktop.2

docker images

REPOSITORY                TAG                IMAGE ID       CREATED         SIZE
elasticsearch             7.17.5             11df7a62573d   23 months ago   610MB
kibana                    7.17.5             5756b819359b   23 months ago   802MB
mobz/elasticsearch-head   5                  b19a5c98e43b   7 years ago     824MB

docker-compose

下载镜像及生成容器

docker-compose 单机部署 es 和 kibana

version: '1.0'
services:
  es_master:
    image: elasticsearch:7.17.5
    #这里我尝试改成es_master,但是下面的kibana.elasticsearch.hosts=https://es_master:9200无法访问es，不知道为什么
    container_name: "elasticsearch"
    hostname: es_master
    #restart: always
    ports:
      - "9200:9200"
      - "9300:9300"
    expose:
      - "9200"
      - "9300"
    volumes:
      - "/f/docker-elk/elk/es/config/conf/es-master.yml:/usr/share/elasticsearch/config/elasticsearch.yml"
      - "/f/docker-elk/elk/es/data/data:/usr/share/elasticsearch/data"
      - "/f/docker-elk/elk/es/logs:/usr/share/elasticsearch/logs"
      - "/f/docker-elk/elk/es/plugins:/usr/share/elasticsearch/plugins"
      - "/f/docker-elk/elk/es/plugins/ik/config/ext.dict:/usr/share/elasticsearch/plugins/ik/config/ext.dict"
    environment:
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
      #下面注释了 # 不会生效
      #- 'cluster.name=elasticsearch' 设置集群名称为elasticsearch
      #- 'discovery.type=single-node' 以单一节点模式启动
    networks:
      elk: #使用networks创建网络
        ipv4_address: 192.168.20.10
  kibana:
    container_name: "kibana"
    hostname: kibana
    #kibana的版本要与es的版本对应（必须）
    image: kibana:7.17.5
    #restart: always
    environment:
      - elasticsearch.hosts=https://elasticsearch:9200
    ports:
      - "5601:5601"
    volumes:
      - "/f/docker-elk/elk/kibana/config/kibana.yml:/usr/share/elasticsearch/config/kibana.yml"
    depends_on:
      - es_master
    networks:
      elk:
        ipv4_address: 192.168.20.20
  #es视图化工具插件
  es-head:
    container_name: "es-head"
    image: mobz/elasticsearch-head:5
    #restart: always
    ports:
      - "9100:9100"
    #依赖es-master
    depends_on:
      - es_master
networks:
  #使用已创建的网络
  #mynetwork:
  #  external: true
  #创建网络
  elk:
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 192.168.20.0/24
          gateway: 192.168.20.1

es_master容器

es_master容器 container_name 尝试过定义为 container_name: es_master kibana容器的 environment: - elasticsearch.hosts=https://es_master:9200 的时候，kibana会报错，提示找不到es的节点，所以我直接用container_name: elasticsearch

这是一个参考链接参考链接

配置es_master容器的config的es_master.yml

# 集群名称
cluster.name: es-cluster
# 节点名称
node.name: es-node1
# 是否可以成为master节点
node.master: true
# 是否允许该节点存储数据,默认开启
node.data: true
# 网络绑定
network.host: 0.0.0.0
# 设置对外服务的http端口
http.port: 9200
# 设置节点间交互的tcp端口
transport.port: 9300
# 集群发现
discovery.seed_hosts: ["192.168.20.10"] #这里的ip是network自定义的网络
# 手动指定可以成为 mater 的所有节点的 name 或者 ip，这些配置将会在第一次选举中进行计算
cluster.initial_master_nodes: ["es-node1"]
# 支持跨域访问
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization,Content-Type
# xpack安全认证，如果不需要验证账号密码，下面配置改为 false
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

kibana的config的kibana.yml配置

# 汉化
i18n.locale: "zh-CN"
# 服务
server.port: 5601
server.host: "0.0.0.0"
server.shutdownTimeout: "5s"
# ES
elasticsearch.hosts: [ "https://elasticsearch:9200" ]
# 此处设置elastic的用户名和密码，es设置开启安全验证的时候必须要设置
#elasticsearch.username: "elastic"
#elasticsearch.password: "***"

elasticsearch创建安全验证用户

docker exec -it elasticsearch bash
root@es_master:/usr/share/elasticsearch# ./bin/elasticsearch-setup-passwords auto
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y

Changed password for user apm_system
PASSWORD apm_system = wpikxFN3xc4HBt1w51YM

Changed password for user kibana_system
PASSWORD kibana_system = vn5Fb13KWaJBzeP5HhaX

Changed password for user kibana
PASSWORD kibana = vn5Fb13KWaJBzeP5HhaX

Changed password for user logstash_system
PASSWORD logstash_system = 5liq6JGl91RtlhQZ7eHt

Changed password for user beats_system
PASSWORD beats_system = hDucuF5OVakrl2M3K9P4

Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = 5HlDA8OVbAuGFoIQdHkn

Changed password for user elastic
PASSWORD elastic = ITojfvi9pKQj1pgQ4VvC

创建Kibana密钥库并添加配置（就是将es的用户和密码存储在密钥库中，用了进行Kibana访问带安全认证的ES）

以root账号进入kibana容器中

docker exec -it -u root kibana bash
root@kibana:/usr/share/kibana# ./bin/kibana-keystore create
Created Kibana Keystore in /usr/share/kibana/config/kibana.keystore
root@kibana:/usr/share/kibana# ./bin/kibana-keystore add elasticsearch.username
Enter value for elasticsearch.username: *******************（elastic）
root@kibana:/usr/share/kibana# ./bin/kibana-keystore add elasticsearch.password
Enter value for elasticsearch.password: *******************（ITojfvi9pKQj1pgQ4VvC）

容器重启

http://localhost:9100/?auth_user=elastic&auth_password=ITojfvi9pKQj1pgQ4VvC