Kubernetes. AlmaLinux 8. Kubernetes Cluster installation with the only one Master-Node and many Worker-Nodes: различия между версиями
Cless (обсуждение | вклад) |
Cless (обсуждение | вклад) м (Cless переименовал страницу Kubernetes. AlmaLinux 8. Kubernetes Cluster install with the only one Master-Node and many Worker-Nodes в Kubernetes. AlmaLinux 8. Kubernetes Cluster installation with the only one Master-Node and many Worker-Nodes без оставления перенаправления) |
(нет различий)
| |
Версия 21:11, 23 марта 2023
About
Install Kubernetes Master + Node at AlmaLinux 8
Install
MASTER kub-master-1
Selinux OFF
setenforce 0
sed -i 's/#\?\(SELINUX\=\).*$/\1disabled/' /etc/sysconfig/selinux
reboot
Set hostname
hostnamectl set-hostname kub-master-1
Add IP address and hostname resolve to hosts
cat <<EOF>> /etc/hosts 192.168.199.184 kub-master-1 192.168.199.186 kub-node-1 192.168.199.187 kub-node-2 EOF
Swap OFF
swapoff -a && sed -i '/ swap / s/^/#/' /etc/fstab
Bridge settings
modprobe br_netfilter
echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
Docker install
dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
dnf install -y containerd.io
dnf install -y --nobest docker-ce
systemctl enable docker
systemctl start docker
Kubernetes install
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg exclude=kubelet kubeadm kubectl EOF
yum repolist
yum update -y
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
Autocompletion kubadm
yum install bash-completion -y
curl https://raw.githubusercontent.com/docker/docker-ce/master/components/cli/contrib/completion/bash/docker -o /etc/bash_completion.d/docker.sh
echo 'source <(kubectl completion bash)' >>~/.bashrc
kubectl completion bash >/etc/bash_completion.d/kubectl
echo 'source <(kubeadm completion bash)' >>~/.bashrc
kubeadm completion bash >/etc/bash_completion.d/kubeadm
Relogin to terminal!
Docker setup
mkdir /etc/docker
cat <<EOF | sudo tee /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2"
}
EOF
systemctl daemon-reload
systemctl restart docker
kubeadm reset
systemctl status docker
# init
ERROR:
[ERROR CRI]: container runtime is not running: output: time="2022-05-20T02:06:28Z" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
SOLUTION:
sudo rm /etc/containerd/config.toml
sudo systemctl restart containerd
sudo kubeadm init
Kubernetes init
--apiserver-advertise-address=192.168.199.184 - LAN interface address, if it is not specified, the default address will be used
--pod-network-cidr=172.16.213.0/24 - container virtual network
kubeadm init --apiserver-advertise-address=192.168.199.184 --pod-network-cidr=172.16.213.0/24
... Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.199.184:6443 --token gn2gva.2h73qvdhk0bg6krn \ --discovery-token-ca-cert-hash sha256:418ca3a0003c8945a1cb9c108992c33aedcebbd1a4ba3960b43230a71eb8cda5 ...
systemctl enable kubelet
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
Setup cluster networking
We need to deploy the pod's internal network so that containers on different nodes can communicate with each other. POD network is the overlay network between the worker nodes.
An overlay network is a virtual network of nodes and logical links, which are built on top of an existing network.
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
Wait about 1 minute
kubectl get nodes
touch /etc/default/kubelet
echo "KUBELET_EXTRA_ARGS=--node-ip=192.168.199.184" > /etc/default/kubelet
If you somehow miss the output from the kubeadm init command, nothing to worry about as you can always run following command to generate the kubeadm join token again.
kubeadm token create --print-join-command
... kubeadm join 192.168.199.184:6443 --token 2qw1q0.hnmu7eggccf7172y --discovery-token-ca-cert-hash sha256:c7832b18f117af05346cf019aa7a9d3ca0029c65a69320f7cb807147ee975efb ...
WORKERS kub-node-1 (you can add additional nodes this way, for example "kub-node-2")
setenforce 0
sed -i 's/#\?\(SELINUX\=\).*$/\1disabled/' /etc/sysconfig/selinux
reboot
hostnamectl set-hostname kub-node-1
Add host and IP match to /etc/hosts
Also you need to add new node host to
/etc/hostsof all k8s nodes and masters in cluster. Or use a DNS server to match IP and hostname.
cat <<EOF>> /etc/hosts 192.168.199.184 kub-master-1 192.168.199.186 kub-node-1 192.168.199.187 kub-node-2 EOF
swapoff -a && sed -i '/ swap / s/^/#/' /etc/fstab
modprobe br_netfilter
echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
dnf install -y containerd.io
dnf install -y --nobest docker-ce
systemctl enable docker
systemctl start docker
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg exclude=kubelet kubeadm kubectl EOF
yum repolist
yum update -y
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
yum install bash-completion -y
curl https://raw.githubusercontent.com/docker/docker-ce/master/components/cli/contrib/completion/bash/docker -o /etc/bash_completion.d/docker.sh
echo 'source <(kubectl completion bash)' >>~/.bashrc
kubectl completion bash >/etc/bash_completion.d/kubectl
echo 'source <(kubeadm completion bash)' >>~/.bashrc
kubeadm completion bash >/etc/bash_completion.d/kubeadm
echo 'alias k=kubectl' >>~/.bashrc
echo 'complete -o default -F __start_kubectl k' >>~/.bashrc
Relogin to terminal!
touch /etc/default/kubelet
--node-ip=192.168.199.186 - kub-node-1 IP address
echo "KUBELET_EXTRA_ARGS=--node-ip=192.168.199.186" > /etc/default/kubelet
mkdir /etc/docker
cat <<EOF | sudo tee /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2"
}
EOF
systemctl enable docker
systemctl daemon-reload
systemctl restart docker
kubeadm reset
ERROR:
[ERROR CRI]: container runtime is not running: output: time="2022-05-20T02:06:28Z" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
SOLUTION:
sudo rm /etc/containerd/config.toml
sudo systemctl restart containerd
kubeadm join 192.168.199.184:6443 --token 2qw1q0.hnmu7eggccf7172y --discovery-token-ca-cert-hash sha256:c7832b18f117af05346cf019aa7a9d3ca0029c65a69320f7cb807147ee975efb
systemctl enable kubelet
Postinstall and info
MASTER kub-master-1
Set node(s) role "node"
kubectl label node kub-node-1 node-role.kubernetes.io/node=
Cluster info
kubectl cluster-info
Master or node info
kubectl describe node kub-master-1
kubectl describe node kub-node-1
Get nodes
kubectl get nodes -o wide
... NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME master Ready control-plane,master 65m v1.23.1 192.168.199.169 <none> CentOS Linux 8 4.18.0-348.7.1.el8_5.x86_64 docker://20.10.12 node-1 Ready <none> 2m28s v1.23.1 192.168.199.179 <none> CentOS Linux 8 4.18.0-305.25.1.el8_4.x86_64 docker://20.10.12 ...
Get pods
kubectl get pods --all-namespaces
... NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-85b5b5888d-lrsq8 1/1 Running 0 58m kube-system calico-node-gwtl6 1/1 Running 0 2m32s kube-system calico-node-lbbbv 1/1 Running 0 58m kube-system coredns-64897985d-6jv5w 1/1 Running 0 65m kube-system coredns-64897985d-n6c9l 1/1 Running 0 65m kube-system etcd-master 1/1 Running 1 (64m ago) 65m kube-system kube-apiserver-master 1/1 Running 1 (64m ago) 65m kube-system kube-controller-manager-master 1/1 Running 1 (64m ago) 65m kube-system kube-proxy-44l8g 1/1 Running 1 (64m ago) 65m kube-system kube-proxy-ljfsd 1/1 Running 0 2m32s kube-system kube-scheduler-master 1/1 Running 1 (64m ago) 65m ...
Certificates
Check certs expiration
kubeadm certs check-expiration
... [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Feb 20, 2023 19:11 UTC 364d ca no apiserver Feb 20, 2023 19:10 UTC 364d ca no apiserver-etcd-client Feb 20, 2023 19:11 UTC 364d etcd-ca no apiserver-kubelet-client Feb 20, 2023 19:10 UTC 364d ca no controller-manager.conf Feb 20, 2023 19:11 UTC 364d ca no etcd-healthcheck-client Feb 20, 2023 19:10 UTC 364d etcd-ca no etcd-peer Feb 20, 2023 19:10 UTC 364d etcd-ca no etcd-server Feb 20, 2023 19:10 UTC 364d etcd-ca no front-proxy-client Feb 20, 2023 19:10 UTC 364d front-proxy-ca no scheduler.conf Feb 20, 2023 19:11 UTC 364d ca no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Feb 18, 2032 19:10 UTC 9y no etcd-ca Feb 18, 2032 19:10 UTC 9y no front-proxy-ca Feb 18, 2032 19:10 UTC 9y no ...
Renew all certs
kubeadm certs renew all
reboot