Graylog. AlmaLinux 8. Установка

Описание

Установка Graylog 4.3 + ElasticSearch 7.10.x + MongoDB 4.4 на AlmaLinux 8

Компоненты

• MongoDB – Stores the configurations and meta information.
• Elasticsearch – Stores the log messages and offers a searching facility, nodes should have high memory as all the I/O operations are happens here.
• GrayLog – Log parser, it collect the logs from various inputs.

Системные требования

!!!

ПРОВЕРЯЕМ СОВМЕСТИМОСТЬ КОМПОНЕНТОВ

ИСХОДЯ ИЗ ЭТОГО РЕШАЕМ КАКИЕ ВЕРСИИ ПРИЛОЖЕНИЙ УСТАНАВЛИВАТЬ

https://docs.graylog.org/docs/installing

!!!

System requirements

We caution you not to install or upgrade Elasticsearch to 7.11 or later! It is not supported. Doing so will break your instance!

The Graylog server application has the following prerequisites:

   Some modern Linux distribution (Debian Linux, Ubuntu Linux, or CentOS recommended)

   One of the following databases:

       Elasticsearch 6.8, and version 7 up to 7.10

       OpenSearch 1.1, 1.2, or 1.3

Warning

OpenSearch is currently only supported in Graylog v4.3.

   MongoDB 3.6, 4.0, 4.2 or 4.4

   Oracle Java SE 17 (OpenJDK 17 also works; latest stable update is recommended)

Установка

Отключение Selinux

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

reboot

Отключение Firewalld

systemctl stop firewalld

systemctl disable firewalld

Установка Epel

rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm

Установка Java

dnf install -y java

java -version

    ...
    java version "1.8.332-b09"
    ...

Установка Elasticsearch

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

nano /etc/yum.repos.d/elasticsearch.repo

[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/oss-7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

yum install elasticsearch-oss

tee -a /etc/elasticsearch/elasticsearch.yml > /dev/null <<EOT

cluster.name: graylog

action.auto_create_index: false

EOT

systemctl daemon-reload

systemctl enable elasticsearch.service

systemctl restart elasticsearch.service

Подождите минуту чтобы Elasticsearch запустился. Elastisearch теперь слушает на порту 9200

Проверим

curl -X GET http://localhost:9200

...
{
  "status" : 200,
  "name" : "Silver Fox",
  "cluster_name" : "graylog2",
  "version" : {
    "number" : "1.7.2",
    "build_hash" : "e43676b1385b8125d647f593f7202acbd816e8ec",
    "build_timestamp" : "2015-09-14T09:49:53Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
}
...

Опциональная проверка

curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'

...
{
  "cluster_name" : "graylog2",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0
}

Установка MongoDB

Вариант 1. DNF

   (ОБХОДИМ БЛОКИРОВКУ, САНКЦИИ)

   nano /etc/yum.repos.d/mongodb-org-4.4.repo

    [mongodb-org-4.4]
    name=MongoDB Repository
    baseurl=http://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.4/x86_64/
    gpgcheck=0
    enabled=1

   dnf -y install mongodb-org

Вариант 1. RPM

   Скачиваем (в обход блокировок) пакеты отсюда: https://repo.mongodb.org/yum/redhat/8/mongodb-org/4.4/x86_64/RPMS/)

       mongodb-org = 4.4.15

       mongodb-org-mongos = 4.4.15

       mongodb-org-server = 4.4.15

       mongodb-org-shell = 4.4.15

       mongodb-org-tools = 4.4.15    

       mongodb-org-database-tools-extra 4.4.15

       mongodb-database-tools-100.5.4.x86_64.rpm

   dnf install -y python3 cyrus-sasl

   rpm -i /root/mongodb-*

Если в ОС используется Selinux

yum -y install policycoreutils-python

semanage port -a -t mongod_port_t -p tcp 27017

systemctl enable mongod

systemctl start mongod

Установка Graylog

rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-4.3-repository_latest.rpm

yum -y install graylog-server

yum -y install pwgen

Генерация пароля

pwgen -N 1 -s 96

...
5uxJaeL4vgP9uKQ1VFdbS5hpAXMXLq0KDvRgARmlI7oxKWQbH9tElSSKTzxmj4PUGlHIpOkoMMwjICYZubUGc9we5tY1FjLB
...

Вносим пароль в конфиг

nano /etc/graylog/server/server.conf

...
password_secret = 5uxJaeL4vgP9uKQ1VFdbS5hpAXMXLq0KDvRgARmlI7oxKWQbH9tElSSKTzxmj4PUGlHIpOkoMMwjICYZubUGc9we5tY1FjLB
...

Пароль для веб-интерфейса (логин: admin)

echo -n <your_password_for_web_login_admin> | sha256sum

...
e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951
...

Вносим пароль в конфиг

nano /etc/graylog/server/server.conf

...
root_password_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951
...

Правим конфиг

nano /etc/graylog/server/server.conf

...
root_email = "itzgeek.web@gmail.com"
root_timezone = Europe/Moscow

# Set only one master node by defining the below variable, default setting is true, you must set it as a false to make the particular node as a slave. Master node performs some periodic tasks that slave won’t perform.
is_master = true

# The following variable sets the number of log messages to keep per index, it is recommended to have several smaller indices instead of larger ones.
elasticsearch_max_docs_per_index = 20000000

# The following parameter defines to have total number of indices, if the this number is reached old index will be deleted.
elasticsearch_max_number_of_indices = 20

# Graylog will try to find the Elasticsearch nodes automatically, it uses multicast mode for the same. But when it comes to larger network, it is recommended to use unicast mode which is best suited one for production setups. So add the following two entries to graylog server.conf file, replace ipaddress with live hostname or ipaddress, multiple hosts can be added with comma separated.

#Shards setting is really depends on the number of nodes in the Elasticsearch cluster, if you have only one node, set it as 1.
elasticsearch_shards = 1

#The number of replicas for your indices, if you have only one node in Elasticsearch cluster; set it as 0.
elasticsearch_replicas = 0

#Add MongoDB authentication information.
mongodb_useauth = false

#Enable the web-based Graylog interface by uncommenting #web_enable = false and setting the value to true. Also uncomment and change the following lines as specified.
rest_listen_uri = http://0.0.0.0:9000/api/
web_enable = true

# graylog v2.x
#web_listen_uri = http://0.0.0.0:9000/

# graylog v3.x
http_bind_address = 0.0.0.0:9000
...

Запуск Graylog

systemctl enable graylog-server

systemctl restart graylog-server

Смотрим логи запуска. Ошибок быть не должно.

tail /var/log/graylog-server/server.log

Настроим Firewall для получения логов Syslog по сети

На Graylog сервере

Настроим приемник логов Syslog от клиентов.

Логи Syslog от клиентов отправляются на порт 514/UDP

На использование портов 1-1024 в Linux необходимы привилегии суперпользователя.

Поэтому перенаправим входящие соединения 514/UDP на порт >1024

Например, порт 5514

Соответственно необходимо на сервере Graylog необходимо настроить ноду на порту 5514 и редиректить входящие соединения в Linux с порта 514 на порт >1024

systemctl stop firewalld

systemctl disable firewalld

yum install iptables-services

systemctl enable iptables

systemctl start iptables

iptables -t nat -A PREROUTING -i <LAN_IFACE_NAME> -p udp -m udp --dport 514 -j REDIRECT --to-ports 5514

Разрешаем 9000/tcp веб-интерфейс порт

iptables -t filter -I INPUT 1 -p tcp --dport 9000 -j ACCEPT

Флашим правила

iptables -t filter -F INPUT

iptables -t filter -F FORWARD

Сохраняем правила

service iptables save

Переходим на веб-интерфейс Graylog

http://192.168.1.1:9000

Настроим приемник логов

• System - Inputs:
  • Select Input: Raw/Plaintext UDP
  • port: udp/5514
  • Listen on: 0.0.0.0/0

На клиенте Syslog mikrotik

/system logging action

set 3 remote=<GRAYLOG_IP> remote-port=514 src-address=<MIKROTIK_IP>

/system logging

add action=remote topics=critical

add action=remote topics=error

add action=remote topics=warning

Совершаем какие либо действия с конфигом микротика

На Graylog сервере

Проверяем в веб-интерфейсе Graylog - Search. Здесь должны появиться записи логов микротика