云原生（四十） | Kubernetes篇之整合EFK

Lansonli • 2023-01-02 • 云技术社区 • 271 阅读

整合EFK

部署前预习

ElasticSearch 的配置文件位置
简单的ElasticSearch 配置管理章节
ElasticStack 安装
kompose 转换 compose为k8s文件
kubectl get pvc -n devops| grep es | awk '{print $1}' | xargs kubectl delete pvc -n devops

使用Es官方Operator方式

一、安装operator

Quickstart | Elastic Cloud on Kubernetes [2.4] | Elastic

kubectl apply -f https://download.elastic.co/downloads/eck/1.6.0/all-in-one.yaml

#查看状态
kubectl -n elastic-system logs -f statefulset.apps/elastic-operator

ECK安装的每个组件如何配置

二、部署ES集群

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: es-cluster
  # 可以指定名称空间
spec:
  version: 7.13.1
  nodeSets:
  - name: masters
    count: 3
    config:
      node.roles: ["master"]
      xpack.ml.enabled: true
    volumeClaimTemplates:
    - metadata:
        name: es-master
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi
        storageClassName: "rook-ceph-block"
  - name: data
    count: 4
    config:
      node.roles: ["data", "ingest", "ml", "transform"]
    volumeClaimTemplates:
    - metadata:
        name: es-node
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi
        storageClassName: "rook-ceph-block"

测试访问
Deploy an Elasticsearch cluster | Elastic Cloud on Kubernetes [2.4] | Elastic

1、本地访问密码测试

## elastic的访问
kubectl get secret es-cluster-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode; echo

## 1、集群内组件访问
###账号 elastic
###密码 t5upEg4l5J37629558659y8Ww
curl -u "elastic:2WC5On8Xio6E767K4x4ph1T7Q54" -k "https://es-cluster-es-http:9200"
curl -u "elastic:2WC5On8Xio6E767K4x4ph1T7Q54" -k "https://10.96.9.9:9200"

## 2、集群本地访问
kubectl port-forward service/es-cluster-es-http 9200
curl -u "elastic:$PASSWORD" -k "https://localhost:9200"

## 3、做成下面的Ingress访问

2、部署ingress访问

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: elastic-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/server-snippet: |
      proxy_ssl_verify off;
spec:
  tls:
  - hosts:
      - elastic.it.com
    secretName: it.com
  rules:
  - host: elastic.it.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: es-cluster-es-http
            port:
              number: 9200

三、部署kibana

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: kibana
spec:
  version: 7.13.1
  count: 1
  elasticsearchRef:
    name: es-cluster

1、访问密码

kubectl get secret es-cluster-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode; echo
### 账号  elastic 
### 密码  618FAZBH5a26990707098798798
###登录即可

2、配置Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kibana-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/server-snippet: |
      proxy_ssl_verify off;
spec:
  tls:
  - hosts:
      - kibana.it.com
    secretName: it.com
  rules:
  - host: kibana.it.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kibana-kb-http
            port:
              number: 5601

四、部署FileBeats

apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: beats
spec:
  type: filebeat
  version: 7.13.1
  elasticsearchRef:
    name: es-cluster
  config:
    filebeat.inputs:
    - type: container
      paths:
      - /var/log/containers/*.log
  daemonSet:
    podTemplate:
      spec:
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        securityContext:
          runAsUser: 0
        containers:
        - name: filebeat
          volumeMounts:
          - name: varlogcontainers
            mountPath: /var/log/containers
          - name: varlogpods
            mountPath: /var/log/pods
          - name: varlibdockercontainers
            mountPath: /var/lib/docker/containers
        volumes:
        - name: varlogcontainers
          hostPath:
            path: /var/log/containers
        - name: varlogpods
          hostPath:
            path: /var/log/pods
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers

本站文章资源均来源自网络，除非特别声明，否则均不代表站方观点，并仅供查阅，不作为任何参考依据！
如有侵权请及时跟我们联系，本站将及时删除！
如遇版权问题，请查看本站版权声明

THE END