Learn how to install Elastic Stack (Elasticsearch, Logstash, Kibana, and Beats) on CentOS 8 with our comprehensive guide. Set up powerful search, logging, and data visualization tools easily and efficiently. #centlinux #linux #elasticsearch #kibana #logstash
Table of Contents
What is Elastic Stack?
The Elastic Stack, formerly known as the ELK Stack, is a collection of open-source software tools developed by Elastic for searching, analyzing, and visualizing data in real-time. The Elastic Stack consists of four core components:
Elasticsearch
- Elasticsearch is a distributed, RESTful search and analytics engine built on Apache Lucene. It provides scalable and real-time search capabilities, allowing users to store, search, and analyze large volumes of data quickly and efficiently.
Kibana
- Kibana is a powerful data visualization and exploration tool that works seamlessly with Elasticsearch. It enables users to create interactive dashboards, charts, and graphs to visualize data stored in Elasticsearch indices. Kibana also provides features for ad-hoc analysis, data exploration, and real-time monitoring.
Beats
- Beats are lightweight data shippers that collect and send various types of data to Elasticsearch or Logstash for further processing. Beats are designed to be easy to deploy and use minimal system resources. There are different types of Beats for collecting different types of data, such as Filebeat for log files, Metricbeat for system metrics, Packetbeat for network traffic, and Auditbeat for audit data.
Logstash
- Logstash is a data processing pipeline that ingests, transforms, and enriches data before sending it to Elasticsearch or other destinations. It supports a wide range of input sources, including logs, metrics, events, and other data formats. Logstash can parse, filter, and manipulate data using a rich set of plugins, allowing users to preprocess data before indexing it into Elasticsearch.
Together, these components form a comprehensive data platform that enables organizations to collect, store, search, analyze, and visualize data in real-time. The Elastic Stack is widely used for various use cases, including log and event data analysis, monitoring and observability, security analytics, business intelligence, and more. Its flexibility, scalability, and extensibility make it a popular choice for organizations of all sizes looking to derive insights from their data.
Recommended Online Training: Elastic Stack 7. Beats, Logstash, Elasticsearch, Kibana. ELK
Environment Specification
We are using a minimal CentOS 8 virtual machine with following specifications.
- CPU – 3.4 Ghz (4 cores)
- Memory – 4 GB
- Storage – 40 GB
- Operating System – CentOS Linux 8.3
- Hostname – elastic-stack.centlinux.com
- IP Address – 192.168.116.230 /24
Read Also: How to install ELK Stack on CentOS 7
Update your Linux Server
Use a ssh client to connect with elastic-stack.centlinux.com server as root user.
It is a best practice to update existing software packages in your Linux operating system before installing any new software thereon.
Therefore, if your Linux server is not updated yet then you can execute the following command to update it.
# dnf update -y
After updating software packages, verify the Linux operating system and Kernel version.
# cat /etc/redhat-release CentOS Linux release 8.3.2011 # uname -r 4.18.0-240.1.1.el8_3.x86_64
Install Java on CentOS 8
Elasticsearch software is written in Java, therefore it requires Java runtime environment for execution.
You can either install Oracle JDK on CentOS 8 or use the open source alternative OpenJDK on your Linux server.
# dnf install -y java-11-openjdk
After installation check the version of Java.
# java -version openjdk version "11.0.9" 2020-10-20 LTS OpenJDK Runtime Environment 18.9 (build 11.0.9+11-LTS) OpenJDK 64-Bit Server VM 18.9 (build 11.0.9+11-LTS, mixed mode, sharing)
Install Elastic Stack Yum Repository
All Elastic Stack software are provided through a common Elastic yum repository. If you add this yum repository in your Linux server then, you can install Elastic Stack very easily.
Import the GPG key of the Elastic yum repository using rpm command.
# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
Create a yum repository file in /etc/yum.repos.d directory.
# vi /etc/yum.repos.d/elasticsearch.repo
And add following directives therein.
[elasticsearch] name=Elasticsearch repository for 7.x packages baseurl=https://artifacts.elastic.co/packages/7.x/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md
You can use the baseurl=https://artifacts.elastic.co/packages/oss-7.x/yum if you wish to install only open source components of Elastic Stack.
Build yum cache for Elastic repository.
# dnf makecache CentOS Linux 8 - AppStream 1.1 kB/s | 4.3 kB 00:03 CentOS Linux 8 - BaseOS 1.6 kB/s | 3.9 kB 00:02 CentOS Linux 8 - Extras 541 B/s | 1.5 kB 00:02 Elasticsearch repository for 7.x packages 262 kB/s | 19 MB 01:12 Metadata cache created.
Elastic yum repository has been installed successfully.
Install Elasticsearch on CentOS 8
Since, you have setup the Elastic yum repository. Therefore, you can install the latest stable release of Elasticsearch by using the dnf command.
# dnf install -y elasticsearch
At the time of this writing, the Elasticsearch 7.10.1 is available. You must ensure that you have installed the same versions of the other Elastic Stack members for better compatibility.
If you are installing on a non-production server with limited memory, then you should reduced the Java memory pool size to run Elasticsearch in a limited memory server. Edit the jvm.options file in vim text editor.
# vi /etc/elasticsearch/jvm.options
Find the following settings in this file.
-Xms1g -Xmx1g
And update with the following values.
-Xms256m -Xmx512m
Enable and start Elasticsearch service.
# systemctl enable --now elasticsearch.service
To verify that the Elasticsearch is configured successfully, you can execute the following command.
# curl -X GET "localhost:9200/?pretty"
{
"name" : "elastic-stack.centlinux.com",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "KdBYBVSVT8aZ7DqJCrQayQ",
"version" : {
"number" : "7.10.1",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "1c34507e66d7db1211f66f3513706fdf548736aa",
"build_date" : "2020-12-05T01:00:33.671820Z",
"build_snapshot" : false,
"lucene_version" : "8.7.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}Elasticsearch has been installed and configured successfully.
Install Kibana on CentOS 8
Just like Elasticsearch, you can also install Kibana software from the same Elastic yum repository. You can use the dnf command as follows.
# dnf install -y kibana
After successful installation of Kibana software, you are required to configure it for use.
Kibana configuration file is located at /etc/kibana/kibana.yml. You can either find and update the required settings or execute the following script to configure Kibana settings in one go.
# cat >> /etc/kibana/kibana.yml << EOF > server.port: 5601 > server.host: "0.0.0.0" > server.name: "elastic-stack.centlinux.com" > elasticsearch.hosts: ["http://localhost:9200"] > EOF
Create a Linux user to own Kibana software files and processes.
# useradd kibana
Change ownership of the following directory.
# chown -R kibana:kibana /usr/share/kibana/* # chown -R kibana:kibana /var/lib/kibana/
Enable and start Kibana service.
# systemctl enable --now kibana.service
Kibana service listens on default port 5601/tcp.
To make Kibana service usable for the network computers, you have to allow incoming traffic to this port in Linux firewall.
Execute the following commands to allow Kibana service port in Linux firewall.
# firewall-cmd --permanent --add-port=5601/tcp success # firewall-cmd --reload success
Open URL http://elastic-stack.centlinux.com:5601 in a web browser.
If you see the above web page then your Kibana software has been installed and configured successfully.
Install Logstash on CentOS 8
Logstash is also available in Elastic yum repository and you can execute dnf command to install it on your Linux server.
# dnf install -y logstash
Logstash can be run with default configurations, you are only required to enable and start the service by using systemctl command.
# systemctl enable --now logstash.service
Install Beats on CentOS 8
For the sake of demonstration, we are only installing Filebeat on our Elastic Stack server. However, you can install any other member of Beats family by using same procedure.
Beats are also available in Elastic yum repository. Therefore use dnf command and install it on your Linux servers that you want to monitor via Elastic Stack.
# dnf install -y filebeat
Add the system module to examine the local system logs.
# filebeat modules enable system Enabled system
Run the filebeat setup. It will scan your local system and connect itself with Kibana dashboard.
# filebeat setup Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling. Index setup finished. Loading dashboards (Kibana must be running and reachable) Loaded dashboards Setting up ML using setup --machine-learning is going to be removed in 8.0.0. Please use the ML app instead. See more: https://www.elastic.co/guide/en/machine-learning/current/index.html Loaded machine learning job configurations Loaded Ingest pipelines
Enable and start Filebeat service.
# systemctl enable --now filebeat.service
Click on the Logs link under Elastic Observatory menu.
Install APM Server on CentOS 8
APM (Application Performance Monitoring) Server is the new entrant in Elastic Stack.
APM Server is an optional component, but it is recommended that you should install it alongwith Elastic Stack to monitor performance of your application servers and identify the bottlenecks therein.
Since, we already have all the system logs collected in our Elasticsearch database, therefore, installing APM server adds a analytical frontend in Elastic Observatory to pinpoint the actual cause of performance bottlenecks.
APM server is also available in Elastic yum repository. Therefore, install it by using dnf command.
# dnf install -y apm-server
Enable and start APM Service.
# systemctl enable --now apm-server.service
If you are new to Linux and facing difficulty in working at Linux Bash prompt. We recommend that, you should read The Linux Command Line, 2nd Edition: A Complete Introduction by William Shotts.
Final Thoughts
Mastering the installation of Elastic Stack on CentOS 8 opens a world of real-time data exploration, visualization, and analysis. With our guide, you’ll effortlessly set up Elasticsearch, Logstash, Kibana, and Beats, transforming your data into actionable insights.
For personalized assistance or to ensure a flawless setup, consider leveraging my expertise. Check out my Fiverr gig here for tailored installation services and unleash the full potential of Elastic Stack on CentOS 8. Let’s turn your data into value together!
thanks for the guide – for oss options be sure to append -oss to the dnf install commands