REX-Ray with Minio

rexray-minio.md

Getting Started

The following commands will install the latest version of REX-Ray to /usr/bin/rexray on Linux systems:

$ sudo apt install s3fs
$ curl -sSL https://dl.bintray.com/rexray/rexray/install | sh -s stable

Depending on the Linux distribution, REX-Ray will be registered as either a SystemD or SystemV service.

Configure

REX-Ray requires a configuration file for storing details used to communicate with storage providers. This can include authentication credentials and driver specific configuration options. After REX-Ray has been installed, copy and paste the contents below to a new file on the host at /etc/rexray/config.yml to configure s3fs storage driver.

libstorage:
  service: s3fs
s3fs:
  accessKey: Q3AM3UQ867SPQQA43P2F
  secretKey: zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG
  endpoint: https://play.minio.io:9000  
  region: us-east-1
  disablePathStyle: false 
  options:
  - url=https://play.minio.io:9000
  - use_path_request_style
  - nonempty

Start as service

Container platforms rely on REX-Ray to be running as a service to function properly. For instance, Docker communicates to the REX-Ray Volume Driver via a UNIX socket file.

$ sudo rexray service start
$ sudo systemctl enable rexray

Create a volume

You can create a volume with docker or with rexray.

$ docker volume create --driver=rexray TESTVOLUME
TESTVOLUME

or

$ rexray volume create test50
ID      Name    Status     Size
test50  test50  available  0

List volumes

Again here you have the choice of docker or rexray. Docker will list all volumes from all drivers, and rexray will list only rexray drivers but with more detail

$ docker volume ls
DRIVER              VOLUME NAME
local               pihole_dnsmasq_data
local               pihole_pihole_data
local               portainer_portainer_data
rexray              test10
rexray              test20
rexray              test50

or

$ rexray volume ls
ID      Name    Status    Size
test10  test10  attached  0
test20  test20  attached  0
test50  test50  attached  0

Delete a volume

Rexray will not allow you to delete an empty bucket. You must purge the contents first. I have tried the -f flag but it still won't work. For empty buckets, use:

$ rexray volume rm test20
test20

or

$ docker volume rm test50
test50

Mount a volume

After a volume has been created, the rexray volume mount command can be used to both attach and mount it to the current host:

$ sudo rexray volume mount testbucket
ID          Name        Status    Size  Path
testbucket  testbucket  attached  0     /var/lib/rexray/volumes/testbucket/data

Unmount a volume

Once a volume is no longer needed, the rexray volume unmount command will unmount the volume:

$ sudo rexray volume unmount
ID          Name        Status     Size
testbucket  testbucket  available  0

Using a volume in docker-compose.yml

Unfortunately, docker-compose volumes are created with underscores automatically, which conflicts with S3FS. There is a workaround to this by setting a name: value in the volumes section. This explicitly declares the volume name so that docker-compose doesn't append to it. Another option is simply to make the volume outside of docker-compose and reference it with external: true

version: '3.5'
services:
  alpinetest:
    image: arm32v7/busybox
    command: tail -F /etc/resolv.conf
    volumes:
      - "test10:/mnt/test10"
volumes:
  test10:
    # Or use "external: true"
    name: test10
    driver: rexray