Kubernetes Bare-Metal Install (on Packet.net)

Installing Kubernetes on bare-metal machines is dead simple, and a million times easier than installing OpenStack. The bulk of the instructions below involve setting up the bare-metal machines on packet.net.

What You’ll Get with These Instructions

One may use these instructions to create a basic Kubernetes cluster. In order to create a cluster environment equivalent to a hosted solution (GKE) or turn-key solutions (Kubernetes on AWS or GCE), you’ll need persistent volume and load-balancer support. A future post will cover how to setup persistent volume storage as a Ceph RBD cluster, and how to work around the need for external load-balancer integration by deploying a Kubernetes Ingress DaemonSet with DNS.

In the following guide, we’ll build a Kubernetes cluster with 1 master and 2 nodes.

Bare-Metal or Cloud VMs, All the Same

These instructions should also work cloud VMs, so long as the following criterial are met:

  • All machines are network reachable, without restriction
    • i.e. open iptables, open security groups
  • Root access through password-less SSH is enabled
    • i.e. configured /root/.ssh/authorized_keys

Because not many of us have multiple bare-metal machines laying around, we’ll rent them from packet.net.

Step Summary

  • Setup a Development Environment
  • Provision Bare-Metal Machines (on packet.net)
  • Configure and Run the Kubernetes contrib/ansible scripts
  • Test the Cluster by Creating a POD
  • Configure a Remote Kubernetes Client to Talk to the Cluster

Setup a Development Environment

The easiest method is to instantiate a vagrant machine with all of the necessary tools. Feel free to do this manually. The vagrant environment includes the following:

  • Docker
  • Kubectl
  • Ansible
  • Hashicorp Terraform

Fetch the vagrant machine

If you are on OSX, ensure and install the host dependencies.

Create your vagant environment

* From now on, do everything from within this vagrant machine.

Provision Bare-Metal Machines (on packet.net)

The contrib/ansible scripts are targeted to offical Redhat distributions including RHEL and Fedora. However, packet.net does not currently support these operating systems. Thus, we deploy Centos7 servers, which is the next-closest thing and happens to work.

If you enjoy clicking around Web UI’s, follow the manual instructions below. Otherwise, the only automated provisioning method supported by packet.net is to use Hashicorp Terraform. A CLI client does not yet exist.

Manual WebUI Instructions

  • Login to the web UI at packet.net
  • Create a project
  • Set an SSH key (which will be provisioned on new servers)
  • Create 3 servers with Centos7
  • Note the names and IPs of the newly created machines

Automated Instructions (using Terraform)

Instantiating servers via Hashicorp Terraform must happen in two steps if a “packet_project” has not yet been created.

This is because the “packet_device” (aka. bare metal machine) definitions require a “project_id” upon execution. However, one does not know the “project_id” until after the “packet_project” has been created. One might resolve this issue in the future by having the packet_device require a project_name instead of a project_id. Then we could “terraform apply” this file in one go.

See the Appendix for curl commands that will help you discover project_ids from the API.

Step 1: Create the Packet.net Project

First, create your auth token via the packet.net UI and note it down.

Create a cluster.tf, and tweak the {variables} according to your needs.

Run Terraform to create the project.

Step 2: Create the Packet.net Machines

Locate your PACKET_PROJECT_ID, so that you may embed it in the cluster.tf Terraform file.

Append to cluster.tf, and tweak the {variables} according to your needs.

Run Terraform

Note the names and IPs of the newly created machines

Create an ~/.ssh/config file with the hostname to IP mappings. This allows you to refer to the machines via hostnames from your Dev box, as well as Ansible inventory definitions.

Configure and Run the Kubernetes contrib/ansible Scripts

Fetch the kubernetes/contrib code

Create an Ansible inventory file (more info in the contrib/ansible/README.md file)

Example “inventory” file, which will work without modification if ~/.ssh/config is configured with host IP mappings.

Test the ansible connection, and accept host-keys into your knownhosts

Configure group_vars/all.yml to set the ansible_ssh_user to “root”

Run the ansible setup script to install Kubernetes on all machines

* Run this more than once until zero errors. It’s idempotent!

Test the Cluster by Creating a POD

The kubernetes master is auto-configured to talk with the cluster. Run some test commands

Create a POD, and test that it is running

Configure a Remote Kubernetes Client to Talk to the Cluster

Now that the kubernetes cluster has been setup, you may use “kubectl” directly after ssh’ing to the master. However, it would be nice to use “kubectl” from the vagrant dev environment.

  • Now you have a working kubernetes cluster on Packet.net
    • Excluding persistent storage and load-balancing

Appendix

Curl Commands

Example Settings

3 thoughts on “Kubernetes Bare-Metal Install (on Packet.net)”

  1. Good overview post. It seems a bit complex because of needing to provision the systems and then run the ansible playbooks but nice to see it done in a repeatable way with terraform.
    I’ll be really interested to see how you get around the need for an external load balancer with an ingress daemonset. I’m assuming you’ll be using the kube proxy to do routing.

    1. Totally agree. When compared to a turnkey kube-up.sh which does provisioning as well as running configuration management, this is more complicated. Thanks for the note.

Leave a Reply

Your email address will not be published. Required fields are marked *