Skip to content

Kubernetes cluster level logging solution, utilizing Elasticseach, Kibana and Fluent Bit.

Notifications You must be signed in to change notification settings

nanmu42/k8s-eck-logging

Repository files navigation

K8S ECK Logging

This walk-through guides you to setup an in-cluster Elasticsearch and Kibana suite, with cluster-level logging data gathered by Fluent Bit. You may access and search logs from every pod in cluster, as long as the workload in pod writes log to stdout or stderr.

Prerequisites

  • Kubernetes 1.11 or higher (minikube not working)
  • Predefined storage class called hdd-ssd (you may change it in eck.yaml)

Deployment Steps

Clone this repo to get necessary yaml files.

Elasticsearch and Kibana

Elastic Cloud on Kubernetes(ECK) is now generally available. ECK makes it easy to deploy Elasticsearch and Kibana on k8s with various topology.

  • Deploy ECK
kubectl apply -f https://download.elastic.co/downloads/eck/1.0.1/all-in-one.yaml
  • Create Namespace logging
kubectl create -f ./namespace.yml
  • Deploy Elasticsearch and Kibana
kubectl create -f ./eck.yml

Fluent Bit

FluentBit runs as DaemonSet on every node in cluster, gathering logs from every workload. FluentBit attach metadata like pod name and label to logs delivered to Elasticsearch.

Well-structured log(in JSON) can be searched/filtered by term in Elasticsearch.

kubectl create -f fluent-bit-service-account.yaml
kubectl create -f fluent-bit-role.yaml
kubectl create -f fluent-bit-role-binding.yaml
kubectl create -f fluent-bit-configmap.yaml
kubectl create -f fluent-bit-ds.yaml

And off you go.

Reference

About

Kubernetes cluster level logging solution, utilizing Elasticseach, Kibana and Fluent Bit.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published