Setting Up and Managing a Kubernetes Cluster Using Vagrant, AWS, and Kubernetes Tools

As a DevOps professional, you’re expected to handle complex infrastructure setups efficiently. This article will guide you through setting up a virtual machine (VM) using Vagrant, configuring AWS tools, and managing a Kubernetes cluster with eksctl, kubectl, and Helm. We’ll start by setting up a VM with Vagrant, then configure AWS CLI and Kubernetes tools on this VM.

Setting Up a Virtual Machine with Vagrant

First, ensure you have Vagrant and VirtualBox installed on your machine. Vagrant allows you to create and configure lightweight, reproducible, and portable development environments.

Vagrantfile Configuration:

Create a Vagrantfile to define the virtual machine.

Vagrant.configure("2") do |config| 

  config.vm.define "node1" do |node1|
    node1.vm.hostname = "node1"
    node1.vm.box = "bento/ubuntu-18.04"
    node1.vm.network "private_network", ip: "192.100.10.10"

    node1.vm.provider "virtualbox" do |vb|
      vb.memory = "2048"
      vb.cpus = 2
    end
  end
end

Explanation:

• Vagrant.configure("2"): Configures Vagrant to use version 2 of its configuration syntax.
• config.vm.define "node1": Defines a VM named node1.
• node1.vm.hostname: Sets the hostname of the VM.
• node1.vm.box: Specifies the base image for the VM.
• node1.vm.network: Configures a private network with a specified IP address.
• node1.vm.provider: Customizes VM resources such as memory and CPU.

To start the VM, run:

vagrant up

Installing Necessary Tools on the VM

Once the VM is running, SSH into it to install necessary software.

vagrant ssh node1

Update and Upgrade the System:

sudo apt-get update
sudo apt-get upgrade -y

Install Python and Pip:

sudo apt-get install -y python python3
sudo apt-get install -y python-pip python3-pip

Install Essential Tools:

sudo apt-get install -y curl git zip software-properties-common apt-transport-https

Install AWS CLI:

pip3 install awscli --upgrade --user
curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
unzip awscli-bundle.zip
sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

Configuring AWS CLI

Set up AWS CLI with your IAM user. Replace testuser with your actual IAM user credentials.

aws configure

Provide the following details when prompted:

• AWS Access Key ID
• AWS Secret Access Key
• Default region name
• Default output format

To verify the configuration:

aws configure list
aws --version
aws ec2 describe-instances

Installing and Configuring eksctl

eksctl is a CLI tool for creating and managing Kubernetes clusters on Amazon EKS.

curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin
eksctl version

Installing and Configuring kubectl

kubectl is the Kubernetes command-line tool used to interact with the cluster.

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl
kubectl version
kubectl version --short --client

Installing Helm

Helm is a package manager for Kubernetes, allowing you to define, install, and upgrade complex Kubernetes applications.

mkdir helm && cd helm
curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash
cd ..
helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm repo update
helm version
helm repo list
helm search repo
helm install redis-chart stable/redis
helm ls
helm uninstall redis-chart

Explanation of Commands:

• sudo apt-get update: Updates the list of available packages.
• sudo apt-get upgrade -y: Upgrades all the installed packages.
• sudo apt-get install -y python python3: Installs Python 2 and 3.
• sudo apt-get install -y python-pip python3-pip: Installs Pip for Python 2 and 3.
• pip3 install awscli --upgrade --user: Installs the AWS CLI using Pip.
• curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip": Downloads the AWS CLI bundle.
• unzip awscli-bundle.zip: Extracts the AWS CLI bundle.
• sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws: Installs the AWS CLI.
• aws configure: Configures the AWS CLI with your credentials.
• curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp: Downloads and extracts eksctl.
• sudo mv /tmp/eksctl /usr/local/bin: Moves eksctl to a directory in the system’s PATH.
• curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -: Adds Google’s apt key for Kubernetes packages.
• echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list: Adds Kubernetes repository to the system’s apt sources.
• sudo apt-get update: Updates the list of available packages.
• sudo apt-get install -y kubectl: Installs kubectl.
• mkdir helm && cd helm: Creates a directory for Helm and navigates into it.
• curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash: Downloads and installs Helm.
• helm repo add stable https://kubernetes-charts.storage.googleapis.com/: Adds the stable Helm chart repository.
• helm repo update: Updates the Helm chart repository.
• helm install redis-chart stable/redis: Installs a Redis chart from the stable repository.
• helm ls: Lists all Helm releases.
• helm uninstall redis-chart: Uninstalls the Redis chart.

This detailed guide covers setting up a Vagrant VM, configuring AWS CLI, and installing essential Kubernetes tools (eksctl, kubectl, and Helm). These steps provide a robust environment for managing Kubernetes clusters and deploying applications efficiently. As a DevOps professional, mastering these tools and workflows is crucial for effective infrastructure management and deployment automation.