Kubernetes multi node cluster with Vagrant (included Kubernetes Dashboards, Grafana, Prometheus)

The following Vagrantfile code(syntax is Ruby) is creating one master node and two worker nodes for Kubernetes.

Go to Vagrantfile directory in Terminal and run the following code to create virtual machines.

As you can see in the below picture, three virtual machines are obtained and running.

But, Kubernetes cluester is still not ready. We created just an environment to install the cluster.

Let’s connect via SSH to Vagrant machines. As you can see, three terminal window is open and connected to masternode, workernode1, and workernode2 respectively. What we did till now was preparing environment to install Kubernetes cluster.

Kubernetes cluster installation

You can reach official installation details from the following page. But, I will also include all the steps here. 🙂

https://kubernetes.io/docs/setup/independent/install-kubeadm/
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/

Installing Docker

apt-get update
apt-get install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository "deb https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(lsb_release -cs) stable"
apt-get update && apt-get install -y docker-ce=$(apt-cache madison docker-ce | grep 17.03 | head -1 | awk '{print $3}')

Installing kubeadm, kubelet and kubectl

apt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

Let’s check docker and kubectl is installed properly. Your installation should seem as below.

Initializing Master Node

Execute the following code in master node terminal. It will take several seconds.

kubeadm init --apiserver-advertise-address="192.168.33.10"

Copy the following code from output of previous command and run in terminal window as below

Let’s check the nodes in Kubernetes cluster with the following code. As you can see, there is only master node and status is NOT READY

It is not ready because, the installation didn’t finish yet. When we check the pods in cluster, there are pods which have pending status.

We need to install one pod network per cluster by running following codes.

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
sysctl net.bridge.bridge-nf-call-iptables=1

When I check the pods in cluster, status of all pods is running

We finished all the works for master node and now we need to create a token for work nodes to join master node. As I mentioned in previous lines, I didn’t use the default token because it is valid for 24 hours. You can see below picture is showing created tokens in list.

I created new token with the following code and TTL is forever.

kubeadm token create --print-join-command --ttl 0

Generated token is as below for me.

kubeadm join 192.168.33.10:6443 --token mgqbmz.1o7lvt39d7fu75xd --discovery-token-ca-cert-hash sha256:5a5c6eea1f2cb0e6e3f157f3ce45ab23cd98dcffe8c854b4948d1cd211fec8c7

Copy result of the code and paste in Worker nodes respectively.

run the following code in masternode terminal window and all worker nodes should appear Ready.

Checking kube config

I delete KUBECONFIG variable on purpose in order to show effect of kube config from terminal.

unset KUBECONFIG

echo "export KUBECONFIG=$HOME/admin.conf" >> $HOME/.bashrc

source $HOME/.bashrc

kubectl config view

You can see in the following terminal.

If you want to see kubeconfig, run the following code in terminal.

cat $HOME/admin.conf

Connecting kubernetes cluster with server

kubectl get nodes --server=https://192.168.33.10:6443

Connecting kubernetes cluster with kubeconfig

kubectl get nodes --kubeconfig $HOME/admin.conf

Including KUBECONFIG Environment Variable to .bashrc

If KUBECONFIG variable is not added in $HOME/.bashrc file, we will encounter with below error.

The connection to the server localhost:8080 was refused - did you specify the right host or port?

So run the following code in the terminal, so that KUBECONFIG variable will be added at the end of $HOME/.bashrc file.

Make sure to use >> and not >, as the latter will overwrite the file.

echo "export KUBECONFIG=$HOME/admin.conf" >> $HOME/.bashrc

Installing Kubernetes Dashboard

You can reach Kubernetes Dashboard project below link.

https://github.com/kubernetes/dashboard

Deploying the Dashboard UI

Run the following code in master node terminal to install Kubernetes Dashboard.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml

Accessing the Dashboard UI

Our master node ip is 192.168.33.10. so, I specified IP address

kubectl proxy --address=192.168.33.10

When you try below link from your local machine, you will see “unauthorized”
http://192.168.33.10:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login

So, run the following command, and you will see below screen shot.

kubectl proxy --address=192.168.33.10 --disable-filter=true

Accessing the Dashboard UI with SSH tunnel

If your vagrant machine doesn’t have private IP address, this means that it is working on localhost(vagrant with forwarded_port). If you are using vagrant with vm-driver=Virtualbox. Your virtual machine will appear as below.

let’s run the following code in vagrant virtual machine (You can connect with vagrant ssh in Vagrantfile location).

kubectl proxy

we have to open ssh tunnel from host machine to vagrant machine. So, change directory to Vagrantfile location and run the following code in your host machine (probably your laptop).

left side local port 8001 <-> right side Vagrant port 8001

ssh -L 8001:localhost:8001 vagrant@localhost -p 2222 -i .vagrant/machines/default/virtualbox/private_key -fNg

Now you can see Kubernetes Dashboard screen in your host machine browser as below. So, call this link.

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login

Kubernetes Dashboard Access Control

Bearer Token

There are many Service Accounts created in Kubernetes by default. All with different access permissions. In order to find any token, run the following code in Vagrant terminal.

kubectl -n tube-system get secret

As you can see, I choose default-token

kubectl -n kube-system describe secrets default-token-8nqck

Copy token value from vagrant terminal and paste browser token textbook in host machine as below.

If you want to login with different token value, click right top corner button and click logout as below.

Leave a Reply

Your email address will not be published. Required fields are marked *