Application Containers: Kubernetes and Docker from Scratch

Overview

In this article we will look at how to configure a Kubernetes cluster using the Docker container format on CentOS or RHEL 7.1. For a detailed overview on Kubernetes and Docker take a look at this article. A Kubernetes cluster is comprised of a master and N nodes. The master acts as a control plane for the cluster and in this case also exposes a private Docker registry. A Kubernetes node runs Docker container images.

Kubernetes_High_Level_Architecture

Requirements

In this article we will setup a master and one node. At minimum two hosts will be required. Kubernetes also has the following networking requirements:

  • all containers can communicate with all other containers without NAT
  • all nodes can communicate with all containers (and vice-versa) without NAT
  • the IP that a container sees itself as is the same IP that others see it as

In order to meet these networking requirements an overlay network  must be configured. Two commonly used overlay networks for Kuberentes are Flannel and Open vSwitch. In this article we will use Flannel.

Setup Kubernetes Master

Creating a Kubernetes master means configuring Kubernetes, Etcd, Flannel, Docker and a private Docker registry. The private Docker registry is used by the nodes in order to pull images.

Install packages and enable services

#yum update -y
#yum install -y docker docker-registry etcd kubernetes flannel
#for SERVICES in docker.service docker-registry etcd kube-apiserver kube-controller-manager kube-scheduler flanneld 
   do systemctl enable $SERVICES 
done

 Configure Private Docker Registry

#vi /etc/sysconfig/docker
INSECURE_REGISTRY='--insecure-registry kube-master.lab.com:5000'

 Configure Kuberentes API Server

#vi /etc/kubernetes/apiserver
KUBE_API_ADDRESS="--address=0.0.0.0"
KUBE_API_PORT="--port=8080"
KUBE_ETCD_SERVERS="--etcd_servers=http://kube-master.lab.com:4001"

 Configure Kubernetes Master

#vi /etc/kubernetes/config
KUBE_MASTER="--master=http://kube-master.lab.com:8080"

Configure Kubernetes Nodes (kubelets)

#vi /etc/kubernetes/controller-manager
KUBELET_ADDRESSES="--machines=kube-node1.lab.com"

Configure ETCD

#vi /etc/etcd/etcd.conf
ETCD_LISTEN_PEER_URLS="http://localhost:2380,http://localhost:7001"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:4001,http://0.0.0.0:2379"
#systemctl start etcd

Configure Overlay Network using Flannel

#vi /etc/sysconfig/flanneld
FLANNEL_ETCD="http://kube-master.lab.com:4001"
FLANNEL_ETCD_KEY="/flannel/network"
FLANNEL_OPTIONS="eth0"
#vi /root/flannel-config.json
{
   "Network": "10.100.0.0/16",
   "SubnetLen": 24,
   "SubnetMin": "10.100.50.0",
   "SubnetMax": "10.100.199.0",
   "Backend": {
   "Type": "vxlan",
   "VNI": 1
   }
 }
curl -L http://kube-master.lab.com:4001/v2/keys/flannel/network/config -XPUT --data-urlencode value@flannel-config.json

Download Docker Images to private registry

#systemctl start docker
#systemctl start docker-registry
#for IMAGE in rhel6 rhel7  kubernetes/kube2sky:1.1 kubernetes/pause:go
   do docker pull $IMAGE
   docker tag $IMAGE kube-master.lab.com:5000/$IMAGE
   docker push kube-master.lab.com:5000/$IMAGES
done
systemctl reboot

Setup Kubernetes Node

In this example we will setup a Kubernetes node from scratch. It is also possible to use a container OS like RHEL Atomic as a Kubernetes node. RHEL Atomic is an OS optimized for running containers. Choosing whether to use RHEL Atomic or a standard RHEL depends greatly on your specific requirements.

Install Packages and enable services

#yum update -y
#yum install -y docker docker-registry etcd kubernetes flannel
#for SERVICES in docker.service kubelet kube-proxy flanneld
   do  systemctl enable $SERVICES
done

Configure Kubernets Master

#vi /etc/kubernetes/config
KUBE_MASTER="--master=http://kube-master.lab.com:8080"

Configure Kubernetes Node (kubelet)

#vi /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=0.0.0.0"

KUBELET_PORT=”–port=10250″
KUBELET_HOSTNAME=””

KUBELET_API_SERVER="--api_servers=http://kube-master.lab.com:8080"

Configure Docker

#vi /etc/sysconfig/docker
ADD_REGISTRY='--add-registry registry.access.redhat.com'
ADD_REGISTRY='--add-registry kube-master.lab.com:5000'

Configure Flannel

#vi /etc/sysconfig/flanneld
FLANNEL_ETCD="http://kube-master.lab.com:4001"
FLANNEL_ETCD_KEY="/flannel/network"
FLANNEL_OPTIONS="eth0"
#systemctl reboot

Summary

In this article we went through the steps of building a Kubernetes cluster from scratch on RHEL or CentOS 7.1. As you have seen standing up a Kubernetes cluster can be done very easily. Hopefully you have found this article helpful, feedback is always greatly appreciated.

Happy Containerizing!

(c) 2015 Keith Tenzer

5 thoughts on “Application Containers: Kubernetes and Docker from Scratch

  1. Pingback: KubeWeekly: Issue #11 – KubeWeekly

  2. I have one question: What does “all containers can communicate with all other containers without NAT” mean? In my case, i have 2 instances Amazon EC2 with Centos 7 already installed and each instance has its own global IP, is that eligible to follow this tutorial?

    Like

    • It means the requirement for Kubernetes is that all containers can communicate with on another. If you have one container host this is easy, if multiple then you need to ensure networking allows this. If you using AMazon EC2 you need to setup private network and use that for communications else your two instances wont be able to communicate. Make sense?

      Keith

      Like

  3. When i was running command to start docker, it said that “”Job for docker.service failed because the control process exited with error code.” Why is that?

    Like

    • Not enough detail, can you provide journald logs? Usually docker fails to start if you start using flannel for overlay network and there is problem with flannel or etcd (requirements for Kubernetes).

      Regards,

      Keith

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s