In this article we will introduce the concept of operators and provide a hands-on guide to building your very first operator. I will show you how to create a development environment with everything needed. Using the Operator SDK we will learn how to create boilerplate code, build and deploy an operator. A followup article is also planned that will go deeper into operator development so stay tuned!
First, what exactly is an Operator?
An Operator is a method of packaging, deploying and managing a Kubernetes application.
Or simply, it is the application that can deploy itself, manage itself and update itself. Welcome to the brave new world, where we don’t spend time doing repetitive manually tasks, but rather put our knowledge into software so it can do it for us, better.
The Kubernetes Operator pattern was introduced by CoreOS engineers in 2016 as again, a way to implement and inject application specific domain knowledge into the running of containerized applications. The idea was to couple Service Reliability Engineering (SRE) output with the application and provide a standard programmable path to do so. SRE teams as you likely know, operate applications by creating software. Most organizations unfortunately don’t have the luxury of having highly paid software engineers to run their applications, nor the time to build such software. Now, with Operators and the Operator Framework, there is a way for vendors or customers to provide that domain knowledge using a standardized reusable approach. The result is applications that run themselves or come with their own pre-programmed built-in SRE team. This is obviously a huge game-changer and differentiator for operating applications. In my opinion it is the only way to deal with the increased complexity we see today. This is the reason eventually, that every application will likely be deployed in containers on Kubernetes. It is simply no longer, with today’s complexity, possible for application domain knowledge to exist in a few people’s heads, that in turn need to operate application lifecycle manually. Think of an operator as the new Linux “Package Manager” and Kubernetes as the new “Linux”.
Introducing the Operator Framework
The Operator Framework is a toolkit to make it easy to build, scale and run operators. It includes the SDK, Operator Lifecycle Manager and Metering.
Operator SDK – Provides tooling to build and package operators. It abstracts Kubernetes API.
Operator Lifecycle Manager – Management control plane for operators. It governs who can access/deploy a given operator, namespaces where operators can run and lifecycle management, such as updates to an operator.
Metering – Provides ability to record historical usage which in turn can be used for operator reporting.
For the purpose of this article we will focus on the SDK.
Setup Development Environment
There are some prerequisites needed to build the operator-sdk and eventually your own operators. In this case I am using Fedora Linux.
Install Docker 17.03+
Add the docker ce repositories. Note: you can also use podman and buildah instead of Docker for those that want a complete and clean divorce.
$ sudo dnf -y install dnf-plugins-core
sudo dnf config-manager \ --add-repo \ https://download.docker.com/linux/fedora/docker-ce.repo
$ sudo dnf -y install docker-ce docker-ce-cli
$ sudo systemctl start docker $ sudo systemctl enable docker
Install Go 1.13+
Download Go and extract it to /usr/local
$ curl -O https://dl.google.com/go/go1.13.6.linux-amd64.tar.gz
$ sudo tar -C /usr/local -xzf go1.13.6.linux-amd64.tar.gz
Update bash profile and set GOPATH/GOBIN
$ vi ~/.bash_profile --- # User specific environment and startup programs PATH=$PATH:$HOME/.local/bin:$HOME/bin:/usr/local/go/bin export GOPATH=/home/ktenzer/go export GOBIN=/home/ktenzer export PATH ---
$ source ~/.bash_profile
Check Go version
$ go version go version go1.13.6 linux/amd64
Install Mercurial 3.9+
$ sudo dnf -y install mercurial
Install Bazaar 2.7.0+
$ sudo dnf -y install -y bzr
Build Operator SDK
Now that the prerequisites are installed we will build the operator sdk from source.
$ go get -d github.com/operator-framework/operator-sdk
$ cd $GOPATH/src/github.com/operator-framework/operator-sdk
$ git checkout master
$ make tidy
$ make install
The Makefile will deploy the operator-sdk CLI into the home directory. Move it to /usr/local/bin.
$ sudo mv ~/operator-sdk /usr/local/bin
Check the operator-sdk version
$ operator-sdk version operator-sdk version: "v0.15.0-5-g6b4e1370", commit: "6b4e1370f18e5d028b283375b740026d2efb0a44", go version: "go1.13.6 linux/amd64"
Building Your First Operator
The SDK is able to generate not only boilerplate code but also the CRDs, controller and API endpoints. In this example we will create a cars operator. It will simply provide an endpoint allowing us to do CRUD on car objects. Each car object will spawn a pod with a base image.
Change directory to the github.com source directory.
$ cd $GOPATH/src/github.com
Note: create directory if it doesn’t exist
Using operatore-sdk cli create boilerplate code.
$ operator-sdk new cars-operator --repo github.com/cars-operator
Add new API CRD for our car operator
$ operator-sdk add api --api-version=cars.example.com/v1alpha1 \ --kind=Car
Add controller API that watches our cars
$ operator-sdk add controller --api-version=cars.example.com/v1alpha1 \ --kind=Car
Create User on Quay.io
We will be using quay to store operator images. Authenticate using Github or Gmail to https://quay.io. Once authenticated go to account settings and set a password.
Test Quay.io Credentials
$ sudo docker login quay.io
Using operator-sdk cli build the operator which will push the image to your local Docker registry.
$ sudo PATH=$PATH:/usr/local/go/bin operator-sdk build \ quay.io/ktenzer/cars-operator
Note: Substitute ktenzer for your username.
Push Operator to your Quay.io Account
$ sudo docker push quay.io/ktenzer/cars-operator:latest
Make Quay Repository Public
By default your Quay repository will be private. If we want to access it from OpenShift we need to make it public. Login to quay.io with your username/password
Select the cars-operator repository
Under Repository Settings enable visibility to be public
Update Image in operator.yaml
We need to point the image to the location in our Quay repository.
vi deploy/operator.yaml --- # Replace this with the built image name image: quay.io/ktenzer/cars-operator command: ---
Deploy Cars Operator on OpenShift
Now that the operator is built and pushed to our Quay repository we can deploy it on an OpenShift cluster.
Authenticate to OpenShift Cluster
$oc login https://api.ocp4.keithtenzer.com:6443
Create new project for our operator
$ oc new-project cars-operator
Setup service accounts and role bindings
$ oc create -f deploy/service_account.yaml
$ oc create -f deploy/role.yaml
$ oc create -f deploy/role_binding.yaml
Create the CRD for our operator
$ oc create -f deploy/crds/cars.example.com_cars_crd.yaml
Deploy our operator
$ oc create -f deploy/operator.yaml
Create the CR for our operator
$ oc create -f deploy/crds/cars.example.com_v1alpha1_car_cr.yaml
Using Cars Operator
The cars operator will automatically deploy an example-car. Whe can query our car object just like any other Kubernetes object. This is the beauty of CR/CRDs and operators. We can easily extend the Kubernetes API without needing to understand it’s complexity.
$ oc get car NAME AGE example-car 31m
Next we can get information about our example-car.
$ oc get car example-car -o yaml apiVersion: cars.example.com/v1alpha1 kind: Car metadata: creationTimestamp: "2020-01-25T12:15:45Z" generation: 1 name: example-car namespace: cars-operator resourceVersion: "2635723" selfLink: /apis/cars.example.com/v1alpha1/namespaces/cars-operator/cars/example-car uid: 6a424ef9-3f6c-11ea-a391-fa163e9f184b spec: size: 3
Looking at the running pods in our cars-operator project we see the operator and our example-car.
$ oc get pods -n cars-operator NAME READY STATUS RESTARTS AGE cars-operator-b98bff54d-t2465 1/1 Running 0 5m example-car-pod 1/1 Running 0 5m
Create a new Car
Lets now create a BMW car.
$ vi bmw.yaml kind: Car metadata: name: bmw spec: size: 1
$ oc create -f bmw.yaml
Here we can see we now have a BMW car.
$ oc get car NAME AGE bmw 11m example-car 31m
Of course we can get information about our BMW car.
$ oc get car bmw -o yaml apiVersion: cars.example.com/v1alpha1 kind: Car metadata: creationTimestamp: "2020-01-25T12:35:25Z" generation: 1 name: bmw namespace: cars-operator resourceVersion: "2644044" selfLink: /apis/cars.example.com/v1alpha1/namespaces/cars-operator/cars/bmw uid: 294bc47f-3f6f-11ea-b32c-fa163e3e8e24 spec: size: 1
Finally as with the example-car, the operator will start a new pod when the BMW car is created.
$ oc get pods -n cars-operator NAME READY STATUS RESTARTS AGE bmw-pod 1/1 Running 0 10m cars-operator-b98bff54d-t2465 1/1 Running 0 14m example-car-pod 1/1 Running 0 14m
Follow these steps to remove the operator cleanly.
$ oc delete -f deploy/crds/cars.example.com_v1alpha1_car_cr.yaml
$ oc delete -f deploy/operator.yaml
$ oc delete -f deploy/role.yaml
$ oc delete -f deploy/role_binding.yaml
$ oc delete -f deploy/service_account.yaml
$ oc delete -f deploy/crds/cars.example.com_cars_crd.yaml
$ oc delete project cars-operator
In this article we introduced the Operator Framework. Explained why it is such a gamer changer for future workloads. We saw the Operator Framework is much more than just an SDK. The lifecycle manager and metrics integration provides critical day 2 operations capability that is currently only available on OpenShift. Finally a step-by-step guide was provided to setup a development environment, generate boilerplate code and deploy our custom cars operator on OpenShift using the Operator Framework.
(c) 2020 Keith Tenzer