Storage for Containers using Gluster – Part II

Overview

This article is a collaboration between Daniel Messer (Technical Marketing Manager Storage @RedHat) and Keith Tenzer (Solutions Architect @RedHat).

• Storage for Containers Overview – Part I
• Storage for Containers using Gluster – Part II
• Storage for Containers using Container Native Storage – Part III
• Storage for Containers using Ceph – Part IV
• Storage for Containers using NetApp ONTAP NAS – Part V
• Storage for Containers using NetApp SolidFire – Part VI

Gluster as Container-Ready Storage (CRS)

In this article we will look at one of the first options of storage for containers and how to deploy it. Support for GlusterFS has been in Kubernetes and OpenShift for some time. GlusterFS is a good fit because it is available across all deployment options: bare-metal, virtual, on-premise and public cloud. The recent addition of GlusterFS running in a container will be discussed later in this series.

GlusterFS is a distributed filesystem at heart with a native protocol (GlusterFS) and various other protocols (NFS, SMB,...). For integration with OpenShift, nodes will use the native protocol via FUSE to mount GlusterFS volumes on the node itself and then have them bind-mount'ed into the target containers. OpenShift/Kubernetes has a native provisioner that implements requesting, releasing and (un-)mounting GlusterFS volumes.

CRS Overview

On the storage side there is an additional component managing the cluster at the request of OpenShift/Kubernetes called "heketi". This is effectively a REST API for GlusterFS and also ships a CLI version. In the following steps we will deploy heketi among 3 GlusterFS nodes, use it to deploy a GlusterFS storage pool, connect it to OpenShift and use it to provision storage for containers via PersistentVolumeClaims. In total we will deploy 4 Virtual Machines. One for OpenShift (lab setup) and three for GlusterFS.

Note: your system should have at least a quad-core CPU, 16GB RAM and 20 GB of free disk space.

Deploying OpenShift

At first you will need an OpenShift deployment. An All-in-One deployment in a VM is sufficient, instructions can be found in the "OpenShift Enterprise 3.4 all-in-one Lab Environment" article.

Make sure your OpenShift VM can resolve external domain names. Edit /etc/dnsmasq.conf and add the following line to use Google DNS:

server=8.8.8.8

Restart it:

# systemctl restart dnsmasq
# ping -c1 google.com

Deploying Gluster

For GlusterFS at least 3 VMs are required with the following specs:

• RHEL 7.3
• 2 CPUs
• 2 GB RAM
• 30 GB disk for OS
• 10 GB disk for GlusterFS bricks

It is necessary to provide local name resolution for the 3 VMs and the OpenShift VM via a common /etc/hosts file.

For example (feel free to adjust the domain and host names to your environment):

# cat /etc/hosts
127.0.0.1      localhost localhost.localdomain localhost4 localhost4.localdomain4
::1            localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.99.144  ocp-master.lab ocp-master
172.16.128.7   crs-node1.lab crs-node1
172.16.128.8   crs-node2.lab crs-node2
172.16.128.9   crs-node3.lab crs-node3

Execute the following steps on all 3 GlusterFS VMs:

# subscription-manager repos --disable="*"
# subscription-manager repos --enable=rhel-7-server-rpms

If you have a GlusterFS subscription you can use it and enable the rh-gluster-3-for-rhel-7-server-rpms repository.

If you don't you can use the unsupported GlusterFS community repositories for testing via EPEL:

# yum -y install http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# rpm --import http://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7

Create a file named glusterfs-3.10.repo in /etc/yum.repos.d/

[glusterfs-3.10]
name=glusterfs-3.10
description="GlusterFS 3.10 Community Version"
baseurl=https://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.10/
gpgcheck=0
enabled=1

Verify the repository is active:

# yum repolist

You should now be able to install GlusterFS

# yum -y install glusterfs-server

A couple of basic TCP ports need to be opened for GlusterFS peers to communicate and provide storage to OpenShift:

# firewall-cmd --add-port=24007-24008/tcp --add-port=49152-49664/tcp --add-port=2222/tcp
# firewall-cmd --runtime-to-permanent

Now we are ready to start the GlusterFS daemon:

# systemctl enable glusterd
# systemctl start glusterd

That's it. GlusterFS is up and running. The rest of the configuration will be done via heketi.

Install heketi on one of the GlusterFS VMs:

[root@crs-node1 ~]# yum -y install heketi heketi-client

---

Update for for EPEL users

If you don't have a Red Hat Gluster Storage subscription you will get heketi from EPEL. At the time of writing this is version 3.0.0-1.el7 from October 2016 which is not working with OpenShift 3.4. You will need to update to a more current version:

[root@crs-node1 ~]# yum -y install wget
[root@crs-node1 ~]# wget https://github.com/heketi/heketi/releases/download/v4.0.0/heketi-v4.0.0.linux.amd64.tar.gz
[root@crs-node1 ~]# tar -xzf heketi-v4.0.0.linux.amd64.tar.gz
[root@crs-node1 ~]# systemctl stop heketi
[root@crs-node1 ~]# cp heketi/heketi* /usr/bin/
[root@crs-node1 ~]# chown heketi:heketi /usr/bin/heketi*

Create a file in /etc/systemd/system/heketi.service for the updated syntax of the v4 heketi binary:

[Unit]
Description=Heketi Server

[Service]
Type=simple
WorkingDirectory=/var/lib/heketi
EnvironmentFile=-/etc/heketi/heketi.json
User=heketi
ExecStart=/usr/bin/heketi --config=/etc/heketi/heketi.json
Restart=on-failure
StandardOutput=syslog
StandardError=syslog

[Install]
WantedBy=multi-user.target

[root@crs-node1 ~]# systemctl daemon-reload
[root@crs-node1 ~]# systemctl start heketi

---

Heketi will use SSH to configure GlusterFS on all nodes. Create an SSH key pair and copy the public key to all 3 nodes (including the first node you are logged on):

[root@crs-node1 ~]# ssh-keygen -f /etc/heketi/heketi_key -t rsa -N ''
[root@crs-node1 ~]# ssh-copy-id -i /etc/heketi/heketi_key.pub root@crs-node1.lab
[root@crs-node1 ~]# ssh-copy-id -i /etc/heketi/heketi_key.pub root@crs-node2.lab
[root@crs-node1 ~]# ssh-copy-id -i /etc/heketi/heketi_key.pub root@crs-node3.lab
[root@crs-node1 ~]# chown heketi:heketi /etc/heketi/heketi_key*

The only thing left is to configure heketi to use SSH. Edit the /etc/heketi/heketi.json to look like the below (changed parts highlighted are underlined):

{
   "_port_comment":"Heketi Server Port Number",
   "port":"8080",
   "_use_auth":"Enable JWT authorization. Please enable for deployment",
   "use_auth":false,
   "_jwt":"Private keys for access",
   "jwt":{
      "_admin":"Admin has access to all APIs",
      "admin":{
         "key":"My Secret"
      },
      "_user":"User only has access to /volumes endpoint",
      "user":{
         "key":"My Secret"
      }
   },
   "_glusterfs_comment":"GlusterFS Configuration",
   "glusterfs":{
      "_executor_comment":[
         "Execute plugin. Possible choices: mock, ssh",
         "mock: This setting is used for testing and development.",
         " It will not send commands to any node.",
         "ssh: This setting will notify Heketi to ssh to the nodes.",
         " It will need the values in sshexec to be configured.",
         "kubernetes: Communicate with GlusterFS containers over",
         " Kubernetes exec api."
      ],
      "executor":"ssh",
      "_sshexec_comment":"SSH username and private key file information",
      "sshexec":{
         "keyfile":"/etc/heketi/heketi_key",
         "user":"root",
         "port":"22",
         "fstab":"/etc/fstab"
      },
      "_kubeexec_comment":"Kubernetes configuration",
      "kubeexec":{
         "host":"https://kubernetes.host:8443",
         "cert":"/path/to/crt.file",
         "insecure":false,
         "user":"kubernetes username",
         "password":"password for kubernetes user",
         "namespace":"OpenShift project or Kubernetes namespace",
         "fstab":"Optional: Specify fstab file on node. Default is /etc/fstab"
      },
      "_db_comment":"Database file name",
      "db":"/var/lib/heketi/heketi.db",
      "_loglevel_comment":[
         "Set log level. Choices are:",
         " none, critical, error, warning, info, debug",
         "Default is warning"
      ],
      "loglevel":"debug"
   }
}

Finished. heketi will listen on port 8080, let's make sure the firewall allows that:

# firewall-cmd --add-port=8080/tcp
# firewall-cmd --runtime-to-permanent

Now restart heketi:

# systemctl enable heketi
# systemctl restart heketi

Test if it's running:

# curl http://crs-node1.lab:8080/hello
Hello from Heketi

Good. Time to put heketi to work. We will use it to configure our GlusterFS storage pool. The software is already running on all our VMs but it's unconfigured. To change that to a functional storage system we will describe our desired GlusterFS storage pool in a topology file, like below:

# vi topology.json
{
  "clusters": [
    {
      "nodes": [
        {
          "node": {
            "hostnames": {
              "manage": [
                "crs-node1.lab"
              ],
              "storage": [
                "172.16.128.7"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/sdb"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
                "crs-node2.lab"
              ],
              "storage": [
                "172.16.128.8"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/sdb"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
                "crs-node3.lab"
              ],
              "storage": [
                "172.16.128.9"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/sdb"
          ]
        }
      ]
    }
  ]
}

Despite the formatting the file is relatively simple. It basically tells heketi to create a 3 node cluster with each node being known by a FQDN, an IP address and with at least one spare block device which will be used as a GlusterFS brick.

Now feed this file to heketi:

# export HEKETI_CLI_SERVER=http://crs-node1.lab:8080
# heketi-cli topology load --json=topology.json
Creating cluster ... ID: 78cdb57aa362f5284bc95b2549bc7e7d
 Creating node crs-node1.lab ... ID: ffd7671c0083d88aeda9fd1cb40b339b
 Adding device /dev/sdb ... OK
 Creating node crs-node2.lab ... ID: 8220975c0a4479792e684584153050a9
 Adding device /dev/sdb ... OK
 Creating node crs-node3.lab ... ID: b94f14c4dbd8850f6ac589ac3b39cc8e
 Adding device /dev/sdb ... OK

Now heketi has configured a 3 node GlusterFS storage pool. Easy! You can see that the 3 VMs have successfully formed what's called a Trusted Storage Pool in GlusterFS

[root@crs-node1 ~]# gluster peer status
Number of Peers: 2

Hostname: crs-node2.lab
Uuid: 93b34946-9571-46a8-983c-c9f128557c0e
State: Peer in Cluster (Connected)
Other names:
crs-node2.lab

Hostname: 172.16.128.9
Uuid: e3c1f9b0-be97-42e5-beda-f70fc05f47ea
State: Peer in Cluster (Connected)

Now back to OpenShift!

Integrating Gluster with OpenShift

For integration in OpenShift two things are needed: a dynamic Kubernetes Storage Provisioner and a StorageClass. The provisioner ships out of the box with OpenShift. It does the actual heavy lifting of attaching storage to containers. The StorageClass is an entity that users in OpenShift can make PersistentVolumeClaims against, which will in turn trigger a provisioner to implement the actual provisioning and represent the result as Kubernetes PersistentVolume (PV).

Like everything else in OpenShift the StorageClass is simply defined as a YAML file:

# cat crs-storageclass.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
 name: container-ready-storage
 annotations:
 storageclass.beta.kubernetes.io/is-default-class: "true"
provisioner: kubernetes.io/glusterfs
parameters:
 resturl: "http://crs-node1.lab:8080"
 restauthenabled: "false"

Our provisioner is kubernetes.io/glusterfs and we make it point to our heketi instance. We name the class "container-ready-storage" and at the same time make it the default StorageClass for all PersistentVolumeClaims that do not explicitly specify one.
Create the StorageClass for your GlusterFS pool:

# oc create -f crs-storageclass.yaml

Using Gluster with OpenShift

Let's look at how we would use GlusterFS in OpenShift. First create a playground project on the OpenShift VM:

# oc new-project crs-storage --display-name="Container-Ready Storage"

To request storage in Kubernetes/OpenShift a PersistentVolumeClaim (PVC) is issued. It's a simple object describing at the minimum how much capacity we need and in which access mode it should be supplied (non-shared, shared, read-only). It's usually part of an application template but let's just create a standalone PVC:

# cat crs-claim.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: my-crs-storage
 namespace: crs-storage
spec:
 accessModes:
 - ReadWriteOnce
 resources:
 requests:
 storage: 1Gi

Issue the claim:

# oc create -f crs-claim.yaml

Watch the PVC being processed and fulfilled with a dynamically created volume in OpenShift:

# oc get pvc
NAME             STATUS    VOLUME                                     CAPACITY   ACCESSMODES   AGE
my-crs-storage   Bound     pvc-41ad5adb-107c-11e7-afae-000c2949cce7   1Gi        RWO           58s

Great! You have now storage capacity available for use in OpenShift without any interaction with the storage system directly. Let's look at the volume that got created:

# oc get pv/pvc-41ad5adb-107c-11e7-afae-000c2949cce7
Name:		pvc-41ad5adb-107c-11e7-afae-000c2949cce7
Labels:		
StorageClass:	container-ready-storage
Status:		Bound
Claim:		crs-storage/my-crs-storage
Reclaim Policy:	Delete
Access Modes:	RWO
Capacity:	1Gi
Message:
Source:
    Type:		Glusterfs (a Glusterfs mount on the host that shares a pod's lifetime)
    EndpointsName:	gluster-dynamic-my-crs-storage
    Path:		vol_85e444ee3bc154de084976a9aef16025
    ReadOnly:		false

The volume has been created specifically according to the design specifications in the PVC. In the PVC we did not explicitly specify which StorageClass we wanted to use because the GlusterFS StorageClass using heketi was defined as the system-wide default.

What happened in the background was that when the PVC reached the system, our default StorageClass reached out to the GlusterFS Provisioner with the volume specs from the PVC. The provisioner in turn communicates with our heketi instance which facilitates the creation of the GlusterFS volume, which we can trace in it's log messages:

[root@crs-node1 ~]# journalctl -l -u heketi.service
...
Mar 24 11:25:52 crs-node1.lab heketi[2598]: [heketi] DEBUG 2017/03/24 11:25:52 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry.go:298: Volume to be created on cluster e
Mar 24 11:25:52 crs-node1.lab heketi[2598]: [heketi] INFO 2017/03/24 11:25:52 Creating brick 9e791b1daa12af783c9195941fe63103
Mar 24 11:25:52 crs-node1.lab heketi[2598]: [heketi] INFO 2017/03/24 11:25:52 Creating brick 3e06af2f855bef521a95ada91680d14b
Mar 24 11:25:52 crs-node1.lab heketi[2598]: [heketi] INFO 2017/03/24 11:25:52 Creating brick e4daa240f1359071e3f7ea22618cfbab
...
Mar 24 11:25:52 crs-node1.lab heketi[2598]: [sshexec] INFO 2017/03/24 11:25:52 Creating volume vol_85e444ee3bc154de084976a9aef16025 replica 3
...
Mar 24 11:25:53 crs-node1.lab heketi[2598]: Result: volume create: vol_85e444ee3bc154de084976a9aef16025: success: please start the volume to access data
...
Mar 24 11:25:55 crs-node1.lab heketi[2598]: Result: volume start: vol_85e444ee3bc154de084976a9aef16025: success
...
Mar 24 11:25:55 crs-node1.lab heketi[2598]: [asynchttp] INFO 2017/03/24 11:25:55 Completed job c3d6c4f9fc74796f4a5262647dc790fe in 3.176522702s
...

Success! In just about 3 seconds the GlusterFS pool was configured and provisioned a volume. The default as of today is replica 3, which means the data will be replicated across 3 bricks (GlusterFS speak for backend storage) of 3 distinct nodes. The process is orchestrated via heketi on behalf of OpenShift.

You can see this information on the volume from GlusterFS perspective:

[root@crs-node1 ~]# gluster volume list
vol_85e444ee3bc154de084976a9aef16025
[root@crs-node1 ~]# gluster volume info vol_85e444ee3bc154de084976a9aef16025

Volume Name: vol_85e444ee3bc154de084976a9aef16025
Type: Replicate
Volume ID: a32168c8-858e-472a-b145-08c20192082b
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 172.16.128.8:/var/lib/heketi/mounts/vg_147b43f6f6903be8b23209903b7172ae/brick_9e791b1daa12af783c9195941fe63103/brick
Brick2: 172.16.128.9:/var/lib/heketi/mounts/vg_72c0f520b0c57d807be21e9c90312f85/brick_3e06af2f855bef521a95ada91680d14b/brick
Brick3: 172.16.128.7:/var/lib/heketi/mounts/vg_67314f879686de975f9b8936ae43c5c5/brick_e4daa240f1359071e3f7ea22618cfbab/brick
Options Reconfigured:
transport.address-family: inet
nfs.disable: on

Notice how the volume name in GlusterFS corresponds to the "path" of the Kubernetes Persistent Volume in OpenShift.

Alternatively you can also use the OpenShift UI to provision storage, which allows you to conveniently select among all known StorageClasses in the system:

Let's make this a little more interesting and run a workload on OpenShift.

On our OpenShift VM still being in the crs-storage project:

# oc get templates -n openshift

You should see a nice list of application and database templates for easy consumption in OpenShift to get your app development project kickstarted.

We will use  MySQL to demonstrate how to host a stateful application on OpenShift with persistent and elastic storage. The mysql-persistent template includes a PVC of 1G for the MySQL database directory. For demonstration purposes all default values are fine.

# oc process mysql-persistent -n openshift | oc create -f -

Wait for the deployment to finish. You can observe the progress in the UI or via

# oc get pods
NAME            READY     STATUS    RESTARTS   AGE
mysql-1-h4afb   1/1       Running   0          2m

Nice. This template created a service, secrets, a PVC and a pod. Let's use it (your pod name will differ):

# oc rsh mysql-1-h4afb

You have successfully attached to the MySQL pod. Let's connect to the database:

sh-4.2$ mysql -u $MYSQL_USER -p$MYSQL_PASSWORD -h $HOSTNAME $MYSQL_DATABASE

Conveniently all vital configuration like MySQL credentials, database name, etc are part of environment variables in the pod template and hence available in the pod as shell environment variables too. Let's create some data:

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| sampledb           |
+--------------------+
2 rows in set (0.02 sec)

mysql> \u sampledb
Database changed
mysql> CREATE TABLE IF NOT EXISTS equipment (
    ->     equip_id int(5) NOT NULL AUTO_INCREMENT,
    ->     type varchar(50) DEFAULT NULL,
    ->     install_date DATE DEFAULT NULL,
    ->     color varchar(20) DEFAULT NULL,
    ->     working bool DEFAULT NULL,
    ->     location varchar(250) DEFAULT NULL,
    ->     PRIMARY KEY(equip_id)
    ->     );
Query OK, 0 rows affected (0.13 sec)

mysql> INSERT INTO equipment (type, install_date, color, working, location)
    -> VALUES
    -> ("Slide", Now(), "blue", 1, "Southwest Corner");
Query OK, 1 row affected, 1 warning (0.01 sec)

mysql> SELECT * FROM equipment;
+----------+-------+--------------+-------+---------+------------------+
| equip_id | type  | install_date | color | working | location         |
+----------+-------+--------------+-------+---------+------------------+
|        1 | Slide | 2017-03-24   | blue  |       1 | Southwest Corner |
+----------+-------+--------------+-------+---------+------------------+
1 row in set (0.00 sec)

This means the database is functional. Great!

Do you want to see where the data is stored? Easy! Look at the mysql volume that got created as part of the template:

# oc get pvc/mysql
NAME      STATUS    VOLUME                                     CAPACITY   ACCESSMODES   AGE
mysql     Bound     pvc-a678b583-1082-11e7-afae-000c2949cce7   1Gi        RWO           11m
# oc describe pv/pvc-a678b583-1082-11e7-afae-000c2949cce7
Name:		pvc-a678b583-1082-11e7-afae-000c2949cce7
Labels:		
StorageClass:	container-ready-storage
Status:		Bound
Claim:		crs-storage/mysql
Reclaim Policy:	Delete
Access Modes:	RWO
Capacity:	1Gi
Message:
Source:
    Type:		Glusterfs (a Glusterfs mount on the host that shares a pod's lifetime)
    EndpointsName:	gluster-dynamic-mysql
    Path:		vol_6299fc74eee513119dafd43f8a438db1
    ReadOnly:		false

Note the path to GlusterFS volume name vol_6299fc74eee513119dafd43f8a438db1.

Return to one of your GlusterFS VMs and issue:

# gluster volume info vol_6299fc74eee513119dafd43f8a438db

Volume Name: vol_6299fc74eee513119dafd43f8a438db1
Type: Replicate
Volume ID: 4115918f-28f7-4d4a-b3f5-4b9afe5b391f
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 172.16.128.7:/var/lib/heketi/mounts/vg_67314f879686de975f9b8936ae43c5c5/brick_f264a47aa32be5d595f83477572becf8/brick
Brick2: 172.16.128.8:/var/lib/heketi/mounts/vg_147b43f6f6903be8b23209903b7172ae/brick_f5731fe7175cbe6e6567e013c2591343/brick
Brick3: 172.16.128.9:/var/lib/heketi/mounts/vg_72c0f520b0c57d807be21e9c90312f85/brick_ac6add804a6a467cd81cd1404841bbf1/brick
Options Reconfigured:
transport.address-family: inet
nfs.disable: on

You can see how the data is replicated across 3 GlusterFS bricks. Let's pick one of them (ideally the host you are logged on to and look at the directory contents):

# ll /var/lib/heketi/mounts/vg_67314f879686de975f9b8936ae43c5c5/brick_f264a47aa32be5d595f83477572becf8/brick
total 180300
-rw-r-----. 2 1000070000 2001       56 Mar 24 12:11 auto.cnf
-rw-------. 2 1000070000 2001     1676 Mar 24 12:11 ca-key.pem
-rw-r--r--. 2 1000070000 2001     1075 Mar 24 12:11 ca.pem
-rw-r--r--. 2 1000070000 2001     1079 Mar 24 12:12 client-cert.pem
-rw-------. 2 1000070000 2001     1680 Mar 24 12:12 client-key.pem
-rw-r-----. 2 1000070000 2001      352 Mar 24 12:12 ib_buffer_pool
-rw-r-----. 2 1000070000 2001 12582912 Mar 24 12:20 ibdata1
-rw-r-----. 2 1000070000 2001 79691776 Mar 24 12:20 ib_logfile0
-rw-r-----. 2 1000070000 2001 79691776 Mar 24 12:11 ib_logfile1
-rw-r-----. 2 1000070000 2001 12582912 Mar 24 12:12 ibtmp1
drwxr-s---. 2 1000070000 2001     8192 Mar 24 12:12 mysql
-rw-r-----. 2 1000070000 2001        2 Mar 24 12:12 mysql-1-h4afb.pid
drwxr-s---. 2 1000070000 2001     8192 Mar 24 12:12 performance_schema
-rw-------. 2 1000070000 2001     1676 Mar 24 12:12 private_key.pem
-rw-r--r--. 2 1000070000 2001      452 Mar 24 12:12 public_key.pem
drwxr-s---. 2 1000070000 2001       62 Mar 24 12:20 sampledb
-rw-r--r--. 2 1000070000 2001     1079 Mar 24 12:11 server-cert.pem
-rw-------. 2 1000070000 2001     1676 Mar 24 12:11 server-key.pem
drwxr-s---. 2 1000070000 2001     8192 Mar 24 12:12 sys

You can see the MySQL database directory here. This is how it's stored in GlusterFS backend and presented to the MySQL container as a bind-mount. If you check your mount table on the OpenShift VM you will see the GlusterFS mount.

Summary

What we have done here is create a simple but functional GlusterFS storage pool outside of OpenShift. This pool can grow and shrink independently of the applications. The entire lifecycle of this pool is managed by a simple front-end known as heketi which only needs manual intervention when the deployment grows. For daily provisioning operations it's API is used via OpenShifts dynamic provisioner, eliminating the need for Developers to interact with Infrastructure teams directly.
This is how we bring storage into the DevOps world - painless, and available directly via developer tooling of the OpenShift PaaS system.
GlusterFS and OpenShift run across all foot-prints: bare-metal, virtual, private and public cloud (Azure, Google Cloud, AWS...) ensuring application portability and avoiding cloud provider lock-in.

Happy Glustering your Containers!

(c) 2017 Keith Tenzer