Kubernetes monitoring

To provisioning EFK inside a Kubernetes cluster to monitor the logs of the Kubernetes cluster.

introduction

Elastic Search:

Elasticsearch is a distributed, RESTful search and analytics engine capable of solving a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data so you can discover the expected and uncover the unexpected.



Fluent Bit:

Fluent Bit is an open-source and multi-platform Log Processor and Forwarder which allows you to collect data/logs from different sources, unify and send them to multiple destinations. It's fully compatible with Docker and Kubernetes environments. 

Fluent Bit is written in C, has a pluggable architecture supporting around 30 extensions. It's fast and lightweight and provides the required security for network operations through TLS.


Kibana:

Kibana is an open-source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on an Elasticsearch cluster. Users can create bar, line, and scatter plots, or pie charts and maps on top of large volumes of data.

Kibana also provides a presentation tool, referred to as Canvas, that allows users to create slide decks that pull live data directly from Elasticsearch.

Docker:

Docker is a set of coupled software-as-a-service and platform-as-a-service products that use operating-system-level virtualization to develop and deliver software in packages called containers. The software that hosts the containers is called Docker Engine.

Kubernetes

Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications.


Software Requirements

Docker

Kubernetes Cluster

Docker-compose

Rancher

Hardware Requirements

3 centos7 machines with each 2 GB RAM and 2 Core CPUs


ARCHITECTURE

Level 0:



Level 1:

Provising EFK using Rancher in kubernetes

Step 1: Create a Docker-compose file for Racher with docker volume

# vi docker-compose.yaml

version: '2.2'

services:

  rancher:

    image: rancher/rancher:latest

    container_name: rancher

    volumes:

      - rancher-data:/var/lib/rancher

    ports:

      - 80:80

      - 443:443

volumes:

  rancher-data: {}


Step 2: Execute the docker-compose file

        # docker-compose up -d

         To check the logs

         # docker-compose -f (optional)


Step 3: Open the browser and Enter the IP address of the docker-machine

Step 4: Save the URL with the IP address of the docker-machine

Step 5: Click Add Cluster

Step 6: Click Import for importing the existing Kubernetes Cluster


Step 7: Enter a Name for the Kubernetes Cluster and click Create

Step 8: Copy the CURL link (3rd link) and Press Done

Step 9: Run that Command in the Kubernetes Master Node


Step 10: Get back Browser and Wait for a while. (5 min approx)


Step 11: Go to Default Namespace


Step 12: Click on Apps Tab and Click Launch and Select EFK

Step 13:   Enter the default configurations and change the ElasticSearch Service Type to ClusterIP and Remove LoadBalancer for Kibana and Click Launch


Step 14: Dashboard

dashboard.png





Recent Comments

No comments

Leave a Comment