To provisioning EFK inside a Kubernetes cluster to monitor the logs of the Kubernetes cluster.
Elasticsearch is a distributed, RESTful search and analytics engine capable of solving a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data so you can discover the expected and uncover the unexpected.
Fluent Bit is an open-source and multi-platform Log Processor and Forwarder which allows you to collect data/logs from different sources, unify and send them to multiple destinations. It's fully compatible with Docker and Kubernetes environments.
Fluent Bit is written in C, has a pluggable architecture supporting around 30 extensions. It's fast and lightweight and provides the required security for network operations through TLS.
Kibana is an open-source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on an Elasticsearch cluster. Users can create bar, line, and scatter plots, or pie charts and maps on top of large volumes of data.
Kibana also provides a presentation tool, referred to as Canvas, that allows users to create slide decks that pull live data directly from Elasticsearch.
Docker is a set of coupled software-as-a-service and platform-as-a-service products that use operating-system-level virtualization to develop and deliver software in packages called containers. The software that hosts the containers is called Docker Engine.
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications.
3 centos7 machines with each 2 GB RAM and 2 Core CPUs
Provising EFK using Rancher in kubernetes
Step 1: Create a Docker-compose file for Racher with docker volume
# vi docker-compose.yaml
Step 2: Execute the docker-compose file
# docker-compose up -d
To check the logs
# docker-compose -f (optional)
Step 3: Open the browser and Enter the IP address of the docker-machine
Step 4: Save the URL with the IP address of the docker-machine
Step 5: Click Add Cluster
Step 6: Click Import for importing the existing Kubernetes Cluster
Step 7: Enter a Name for the Kubernetes Cluster and click Create
Step 8: Copy the CURL link (3rd link) and Press Done
Step 9: Run that Command in the Kubernetes Master Node
Step 10: Get back Browser and Wait for a while. (5 min approx)
Step 11: Go to Default Namespace
Step 12: Click on Apps Tab and Click Launch and Select EFK
Step 13: Enter the default configurations and change the ElasticSearch Service Type to ClusterIP and Remove LoadBalancer for Kibana and Click Launch
Step 14: Dashboard