收集k8s集群节点系统日志

通过在 worker 节点以 DaemonSet 方法运行 filebeat 应用实现

下载 filebeat 镜像

所有 worker 节点拉取镜像

docker pull elastic/filebeat:7.17.2

创建 filebeat 资源清单文件

filebeat-to-logstash.yaml

cat > filebeat-to-logstash.yaml << "EOF"
apiVersion: v1
kind: ConfigMap
metadata:
  name: k8s-logs-filebeat-config
  namespace: kube-system

data:
  filebeat.yml: |
    filebeat.inputs:
      - type: log
        paths:
          - /var/log/messages
        fields:
          app: k8s
          type: module
        fields_under_root: true

    setup.ilm.enabled: false
    setup.template.name: "k8s-module"
    setup.template.pattern: "k8s-module-*"

    output.logstash:
      hosts: ['192.168.1.99:5044']
      index: "k8s-module-%{+yyyy.MM.dd}"
      enabled: true
      worker: 1
      compression_level: 3
      loadbalance: true

---

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: k8s-logs
  namespace: kube-system
spec:
  selector:
    matchLabels:
      project: k8s
      app: filebeat
  template:
    metadata:
      labels:
        project: k8s
        app: filebeat
    spec:
      containers:
      - name: filebeat
        image: docker.io/elastic/filebeat:7.17.2
        args: [
          "-c", "/etc/filebeat.yml",
          "-e",
        ]
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 500m
            memory: 500Mi
        securityContext:
          runAsUser: 0
        volumeMounts:
        - name: filebeat-config
          mountPath: /etc/filebeat.yml
          subPath: filebeat.yml
        - name: k8s-logs
          mountPath: /var/log/messages
      volumes:
      - name: k8s-logs
        hostPath:
          path: /var/log/messages
      - name: filebeat-config
        configMap:
          name: k8s-logs-filebeat-config
EOF

应用 filebeat 资源清单文件

kubectl apply -f filebeat-to-logstash.yaml

验证结果

kubectl get pods -n kube-system -o wide

在 kibana 中添加索引