Revisions
-
MarWeUMR revised this gist
Sep 7, 2023 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -281,5 +281,5 @@ echo "export KUBECONFIG=$HOME/.kube/rke2.yaml" >> $HOME/.bashrc Here are some possible things that might resolve issues because of old/faulty entries: - `iptables -F` - `rm -rf /var/lib/rancher/rke2/server/db` - `rm -rf /run/k3s/containerd/*` - `rm /etc/rancher/rke2/rke2.yaml` -
MarWeUMR revised this gist
Aug 31, 2023 . 1 changed file with 3 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -278,7 +278,8 @@ echo "export KUBECONFIG=$HOME/.kube/rke2.yaml" >> $HOME/.bashrc # Troubleshooting Here are some possible things that might resolve issues because of old/faulty entries: - `iptables -F` - `rm -rf /var/lib/rancher/rke2/server/db` - `rm /run/k3s/containerd/*` - `rm /etc/rancher/rke2/rke2.yaml` -
MarWeUMR revised this gist
Aug 31, 2023 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -280,5 +280,5 @@ echo "export KUBECONFIG=$HOME/.kube/rke2.yaml" >> $HOME/.bashrc Here are some possible things that might resolve issues: - `iptables -F` - `rm -rf /var/lib/rancher/rke2/server/db` - `rm /run/k3s/containerd/*` -
MarWeUMR revised this gist
Aug 31, 2023 . 1 changed file with 8 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -274,4 +274,11 @@ echo "export PATH=$PATH:/var/lib/rancher/rke2/bin" >> $HOME/.bashrc mkdir ~/.kube cat /etc/rancher/rke2/rke2.yaml > ~/.kube/rke2.yaml echo "export KUBECONFIG=$HOME/.kube/rke2.yaml" >> $HOME/.bashrc ``` # Troubleshooting Here are some possible things that might resolve issues: - `iptables -F` - `rm -rf /var/lib/rancher/rke/server/db` - `rm /run/k3s/containerd/*` -
MarWeUMR revised this gist
Aug 31, 2023 . 1 changed file with 3 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -100,7 +100,7 @@ upstream backend { least_conn; server CP_NODE_1:9345 max_fails=3 fail_timeout=5s; server CP_NODE_2:9345 max_fails=3 fail_timeout=5s; server CP_NODE_3:9345 max_fails=3 fail_timeout=5s; } # This server accepts all traffic to port 9345 and passes it to the upstream. @@ -113,7 +113,7 @@ upstream ef_cx_api { least_conn; server CP_NODE_1:6443 max_fails=3 fail_timeout=5s; server CP_NODE_2:6443 max_fails=3 fail_timeout=5s; server CP_NODE_3:6443 max_fails=3 fail_timeout=5s; } server { listen 6443; @@ -264,9 +264,8 @@ curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE=server sh - And start the service: ```bash systemctl enable rke2-server systemctl start rke2-server ``` Now add the kubeconfig and binaries path: -
MarWeUMR revised this gist
Aug 31, 2023 . 1 changed file with 4 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -7,6 +7,10 @@ Helpful links: We need all machines up and running. At first we will setup the loadbalancer. ##### Note: Keep in mind to use `rke2-killall.sh` if you need to restart the service. You also may need to delete `/var/lib/rancher/rke/server/db` before setting up the cluster again. ## Loadbalancer Nginx Setup ### Configure /etc/hosts -
MarWeUMR revised this gist
Aug 30, 2023 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -179,7 +179,7 @@ Now we install rke2: curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE=server sh - ``` And start the service (ONE BY ONE!): ```bash systemctl start rke2-server -
MarWeUMR revised this gist
Aug 30, 2023 . 1 changed file with 3 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -268,5 +268,7 @@ Now add the kubeconfig and binaries path: ```bash echo "export PATH=$PATH:/var/lib/rancher/rke2/bin" >> $HOME/.bashrc mkdir ~/.kube cat /etc/rancher/rke2/rke2.yaml > ~/.kube/rke2.yaml echo "export KUBECONFIG=$HOME/.kube/rke2.yaml" >> $HOME/.bashrc ``` -
MarWeUMR revised this gist
Aug 30, 2023 . 1 changed file with 147 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -123,4 +123,150 @@ And (re)start the server `systemctl restart nginx`. ## Prepare the k8s Cluster Servers Now, before we start our cluster, all machines must be prepared. As a first step, lets create the necessary config files and directories. ```bash mkdir -p /etc/rancher/rke2/ mkdir -p /var/lib/rancher/rke2/server/manifests/ ``` and now we create the rke config (remember to change ips and fqdn): ```bash cat<<EOF|tee /etc/rancher/rke2/config.yaml tls-san: # Loadbalancer - devops67.ef.com - 10.192.168.67 # Cluster - devops61.ef.com - 10.192.168.61 - devops62.ef.com - 10.192.168.62 - devops63.ef.com - 10.192.168.63 EOF ``` Next create the nginx ingress config: ```bash cat<<EOF| tee /var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx-config.yaml --- apiVersion: helm.cattle.io/v1 kind: HelmChartConfig metadata: name: rke2-ingress-nginx namespace: kube-system spec: valuesContent: |- controller: metrics: service: annotations: prometheus.io/scrape: "true" prometheus.io/port: "10254" config: use-forwarded-headers: "true" allowSnippetAnnotations: "true" EOF ``` Now we install rke2: ```bash curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE=server sh - ``` And start the service: ```bash systemctl start rke2-server systemctl enable rke2-server ``` Now add the kubeconfig and binaries path: ```bash echo "export PATH=$PATH:/var/lib/rancher/rke2/bin" >> $HOME/.bashrc echo "export KUBECONFIG=/etc/rancher/rke2/rke2.yaml" >> $HOME/.bashrc ``` And as the last step get the token for the other Cluster Nodes to join: ```bash cat /var/lib/rancher/rke2/server/node-token ``` ### Configuring the other Cluster Nodes We start again by creating the directories: ```bash mkdir -p /etc/rancher/rke2/ mkdir -p /var/lib/rancher/rke2/server/manifests/ ``` and creaing the rke config file: ```bash cat<<EOF|tee /etc/rancher/rke2/config.yaml server: https://10.192.168.67:9345 # Loadbalancer ip token: [token from /var/lib/rancher/rke2/server/node-token on server node 1] tls-san: - devops67.ef.com - 10.192.168.67 - devops61.ef.com - 10.192.168.61 - devops62.ef.com - 10.192.168.62 - devops63.ef.com - 10.192.168.63 EOF ``` And the nginx config again. ```bash cat<<EOF| tee /var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx-config.yaml --- apiVersion: helm.cattle.io/v1 kind: HelmChartConfig metadata: name: rke2-ingress-nginx namespace: kube-system spec: valuesContent: |- controller: metrics: service: annotations: prometheus.io/scrape: "true" prometheus.io/port: "10254" config: use-forwarded-headers: "true" allowSnippetAnnotations: "true" EOF ``` Now we install rke2: ```bash curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE=server sh - ``` And start the service: ```bash systemctl start rke2-server systemctl enable rke2-server ``` Now add the kubeconfig and binaries path: ```bash echo "export PATH=$PATH:/var/lib/rancher/rke2/bin" >> $HOME/.bashrc echo "export KUBECONFIG=/etc/rancher/rke2/rke2.yaml" >> $HOME/.bashrc ``` -
MarWeUMR revised this gist
Aug 30, 2023 . 1 changed file with 8 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,3 +1,7 @@ Helpful links: - https://docs.expertflow.com/cx/rke2-deployment-in-high-availability-with-nginx-haproxy-195855073.html - https://docs.rke2.io/install/ha # Preparations We need all machines up and running. @@ -116,3 +120,7 @@ server { And (re)start the server `systemctl restart nginx`. ## Prepare the k8s Cluster Servers Now, before we start our cluster, all machines must be prepared. As a first step, lets create the necessary config files and directories. -
MarWeUMR revised this gist
Aug 30, 2023 . 1 changed file with 97 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -5,6 +5,7 @@ At first we will setup the loadbalancer. ## Loadbalancer Nginx Setup ### Configure /etc/hosts ```bash $ cat /etc/hosts @@ -19,4 +20,99 @@ $ cat /etc/hosts <k8s server 1 ip> <k8s server 1 fqdn> CP_NODE_1 # <- We need the alias names in the nginx config <k8s server 2 ip> <k8s server 1 fqdn> CP_NODE_2 <k8s server 3 ip> <k8s server 1 fqdn> CP_NODE_3 ``` ### Configure /etc/nginx/nginx.conf Now lets setup the nginx config: ```bash user nginx; worker_processes 4; worker_rlimit_nofile 40000; error_log /var/log/nginx/error.log; pid /run/nginx.pid; include /etc/nginx/modules-enabled/*.conf; # Load dynamic modules. See /usr/share/doc/nginx/README.dynamic. include /usr/share/nginx/modules/*.conf; events { worker_connections 8192; } http { ## # Basic Settings ## sendfile on; tcp_nopush on; types_hash_max_size 2048; # server_tokens off; # server_names_hash_bucket_size 64; # server_name_in_redirect off; include /etc/nginx/mime.types; default_type application/octet-stream; ## # SSL Settings ## ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE ssl_prefer_server_ciphers on; ## # Logging Settings ## access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ## # Gzip Settings ## gzip on; ## # Virtual Host Configs ## include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; } stream { upstream backend { least_conn; server CP_NODE_1:9345 max_fails=3 fail_timeout=5s; server CP_NODE_2:9345 max_fails=3 fail_timeout=5s; #server CP_NODE_3:9345 max_fails=3 fail_timeout=5s; } # This server accepts all traffic to port 9345 and passes it to the upstream. # Notice that the upstream name and the proxy_pass need to match. server { listen 9345; proxy_pass backend; } upstream ef_cx_api { least_conn; server CP_NODE_1:6443 max_fails=3 fail_timeout=5s; server CP_NODE_2:6443 max_fails=3 fail_timeout=5s; #server CP_NODE_3:6443 max_fails=3 fail_timeout=5s; } server { listen 6443; proxy_pass ef_cx_api; } } ``` And (re)start the server `systemctl restart nginx`. -
MarWeUMR created this gist
Aug 30, 2023 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,22 @@ # Preparations We need all machines up and running. At first we will setup the loadbalancer. ## Loadbalancer Nginx Setup ```bash $ cat /etc/hosts #################################################################### # IP FQDN ALIASES #-------------- --------------------------- ------------------------ # Loadbalancer <lb ip> <lb fqdn> <lb alias> # K8s Server <k8s server 1 ip> <k8s server 1 fqdn> CP_NODE_1 # <- We need the alias names in the nginx config <k8s server 2 ip> <k8s server 1 fqdn> CP_NODE_2 <k8s server 3 ip> <k8s server 1 fqdn> CP_NODE_3 ```