MinIO-1: Installing And Using MinIO As An Object Storage

It functions almost the same as the S3 tool in Amazon Web Services. However, MinIO is a 100% open-source object storage solution, while AWS S3 is a commercial product offered by AWS. Also, considering the prices of S3 tools and legal regulations, especially in finance, healthcare, and similar sectors, you may not want to use cloud-based systems such as AWS S3, as it is not legal to store data abroad. At this point, you can create and use your Object Storage Servers with MinIO, in a way that fully meets your needs. In this article, we will practically install MinIO for Docker, Kubernetes, and Linux. Then, we will upload an object file into MinIO and access the object file through our Browser by using a file-sharing link.

12 min readFeb 21, 2024

Topics we will cover:

What Is MinIO And Why Do We Use It?
Installing Minio Server Using Docker Compose For Containers
2. a. Creating It Using Docker Compose
2. b. Get started With The Admin UI
MinIO Object Storage For Kubernetes
3. a. Installing MinIO For Google Kubernetes Engine
3. b. Installing MinIO to Kubernetes Using The “kubectl krew” Command
3. c. Installing MinIO To The Kubernetes Cluster Via Helm
3. d. Accessing The MinIO Console
Installing The MinIO Object Storage For Linux
4. a. Accessing The MinIO Console In Linux
Using MinIO As A Object Storage: Uploading And Sharing A File
As a result
Next post: “MinIO-2: Using the MinIO CLI Tool And AWS-SDK With MinIO”
References

If you like the article, I will be happy if you click the Medium Follow button to encourage me to write more, and not miss future articles.

Your clapping, following, or subscribing helps my articles to reach a broader audience. Thank you in advance for them.

1. What Is MinIO And Why Do We Use It?

MinIO is a High Performance Object Storage released under GNU AGPL v3.0. It is API compatible with Amazon S3 cloud storage service. Use MinIO to build high-performance infrastructure for machine learning, analytics, and application data workloads. (1)

Native to Kubernetes, MinIO is the only object storage suite available on every public cloud, every Kubernetes distribution, the private cloud and the edge. (2)

It functions almost the same as the S3 tool in Amazon Web Services. MinIO is a 100% open-source object storage solution, while AWS S3 is a commercial product offered by AWS. However, considering the prices of S3 tools and legal regulations, especially in finance, healthcare and similar sectors, you may not want to use cloud-based systems such as AWS S3, as it is not legal to store data abroad. At this point, you can create and use your Object Storage Servers with MinIO, in a way that fully meets your needs.

Advantages of MinIO

Scalability: MinIO’s architecture allows it to scale horizontally, making it an ideal choice for businesses of all sizes. Whether you’re a startup or a global enterprise, MinIO can grow with you.
Compatibility: MinIO is S3-compatible, which means it seamlessly integrates with existing S3 applications and tools. You can migrate your data to MinIO without the need for significant changes to your existing workflows.
High Performance: MinIO is optimized for speed, ensuring low-latency and high-throughput access to your data. This makes it perfect for applications where performance is crucial.
Data Protection: With built-in encryption (at rest and in transit), access controls, and versioning, MinIO ensures the security and integrity of your data.
Distributed Architecture: MinIO’s distributed architecture guarantees fault tolerance and high availability, minimizing the risk of data loss and downtime. (3)

2. Installing MinIO Server Using Docker Compose For Containers

Prerequisites

Docker and Docker Compose are installed on your machine.

2. a. Creating It Using Docker Compose

Create a new file named docker-compose.yaml and open it by using your text editor (mine is VSCode). Then, paste the following code into the docker-compose.yaml to install the MinIO service.

Modify the environment variables MINIO_ROOT_USER, MINIO_ROOT_PASSWORD and MINIO_DEFAULT_BUCKETS to set your desired username, password and bucket name for MinIO server access.

The MinIO Console port has 9001 as its default value. The API port, for connecting to and performing operations on the MinIO through APIs, which I set to a static port number 9000 in the docker-compose file. You can change port mappings in the ports section of the docker-compose file. (5)

version: '3'

services:
  minio:
    image: docker.io/bitnami/minio:2022
    ports:
      - '9000:9000'
      - '9001:9001'
    networks:
      - minionetwork
    volumes:
      - 'minio_data:/data'
    environment:
      - MINIO_ROOT_USER=cumhur
      - MINIO_ROOT_PASSWORD=Pp.12345
      - MINIO_DEFAULT_BUCKETS=cumhurminio

networks:
  minionetwork:
    driver: bridge

volumes:
  minio_data:
    driver: local

To download and start the MinIO container; navigate to the directory where the docker-compose.yml file is located, and run the following command:

docker-compose up

Note-1: You can use this command -d flag, then it will start the MinIO server as a Docker container in the background.

docker-compose up -d

Note-2: You can use thedocker-compose.yaml below too, it starts 4 docker containers running MinIO server instances with nginx reverse proxy and load balancing. (6)

version: '3.7'

# Settings and configurations that are common for all containers
x-minio-common: &minio-common
  image: quay.io/minio/minio:RELEASE.2024-02-17T01-15-57Z
  command: server --console-address ":9001" http://minio{1...4}/data{1...2}
  expose:
    - "9000"
    - "9001"
  # environment:
    # MINIO_ROOT_USER: miniocumhur
    # MINIO_ROOT_PASSWORD: miniocumhur
  healthcheck:
    test: ["CMD", "mc", "ready", "local"]
    interval: 5s
    timeout: 5s
    retries: 5

# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
  minio1:
    <<: *minio-common
    hostname: minio1
    volumes:
      - data1-1:/data1
      - data1-2:/data2

  minio2:
    <<: *minio-common
    hostname: minio2
    volumes:
      - data2-1:/data1
      - data2-2:/data2

  minio3:
    <<: *minio-common
    hostname: minio3
    volumes:
      - data3-1:/data1
      - data3-2:/data2

  minio4:
    <<: *minio-common
    hostname: minio4
    volumes:
      - data4-1:/data1
      - data4-2:/data2

  nginx:
    image: nginx:1.19.2-alpine
    hostname: nginx
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "9000:9000"
      - "9001:9001"
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4

## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
  data1-1:
  data1-2:
  data2-1:
  data2-2:
  data3-1:
  data3-2:
  data4-1:
  data4-2:

## For more information, see the link in references section in this article.

2. b. Get started With The Admin UI

Accessing The MinIO Console

Now you can access the MinIO Console. After installation, your MinIO admin UI is available under port 9001. Open your browser and go to http://your-public-IP:9001 to open the MinIO Console login page. Log in with the Root User and Root password from the previous step (The username and password you specified in the docker-compose.yml file to log in).

We will see the MinIO’s home page, where our bucket named “cumhurminio” is created, as seen below.

Note: You can access MinIO by using API.
In addition to the MinIO web console, you can use the API addresses to access the MinIO server and perform S3 operations. You can use different SDKs, such as boto3 and bulkboto3, to perform bucket and object operations to any Amazon S3-compatible object storage service. Applications can authenticate using the MINIO_ROOT_USER as aws_access_key_id and MINIO_ROOT_PASSWORD as aws_secret_access_key credentials. Please note that we need to communicate on port 9000 through MinIO APIs as we set in the docker-compose file. Here, provided a small Python code for creating and transferring a directory with its structure to the bucket using the bulkboto3 package; (7)

import logging
 
from bulkboto3 import BulkBoto3
 
logging.basicConfig(
    level="INFO",
    format="%(asctime)s — %(levelname)s — %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S",
)
logger = logging.getLogger(__name__)
 
TARGET_BUCKET = "test-bucket"
NUM_TRANSFER_THREADS = 50
TRANSFER_VERBOSITY = True
 
# instantiate a BulkBoto3 object
bulkboto_agent = BulkBoto3(
    resource_type="s3",
    endpoint_url="http://127.0.0.1:9000",
    aws_access_key_id="masoud",
    aws_secret_access_key="Strong#Pass#2022",
    max_pool_connections=300,
    verbose=TRANSFER_VERBOSITY,
)
 
# create a new bucket
bulkboto_agent.create_new_bucket(bucket_name=TARGET_BUCKET)
 
# upload a whole directory with its structure to an S3 bucket in multi thread mode
bulkboto_agent.upload_dir_to_storage(
    bucket_name=TARGET_BUCKET,
    local_dir="test_dir",
    storage_dir="my_storage_dir",
    n_threads=NUM_TRANSFER_THREADS,
)

3. MinIO Object Storage for Kubernetes

MinIO is built to deploy an anywhere, public or private cloud, baremetal infrastructure, orchestrated environments, and edge infrastructure. So, you can install the MinIO object storage for Kubernetes environment, AWS Elastic Kubernetes Service, Azure Kubernetes Service, and Google Kubernetes Engine. (8)

Prerequisites

A local kubectl installation configured to create and access resources on the target Kubernetes deployment.

3. a. Installing MinIO For Google Kubernetes Engine

Here I will show you how to install minio on the GKE cluster. If you want to install it on AWS or Azure clouds, you can follow the instructions in the following links; Amazon Elastic Kubernetes Service, Azure Kubernetes Service.

For the GKE cluster, download minio-dev.yaml to your host machine with the following command: (9)

curl https://raw.githubusercontent.com/minio/docs/master/source/extra/examples/minio-dev.yaml -O

The following command applies the minio-dev.yaml configuration and deploys the objects to Kubernetes:

kubectl apply -f minio-dev.yaml

The command output should resemble the following:

You can verify the state of the pod by running kubectl get pods:

kubectl get pods -n minio-dev

The output should resemble the following:

Note: If you have problems with the pods running, check the nodeSelector setting in the minio-dev.yaml. Use thekubectl get nodes --show-labels command to view all labels assigned to each node in the cluster.

You can also use the following commands to retrieve detailed information on the pod status:

kubectl describe pod/minio -n minio-dev,
kubectl logs pod/minio -n minio-dev

3. b. Installing MinIO to Kubernetes Using `Kubectl Krew` Command

Firstly, install "krew” . It is a plugin manager for kubectl. Run these commands to download and install krew as shown below. (10)

(
  set -x; cd "$(mktemp -d)" &&
  OS="$(uname | tr '[:upper:]' '[:lower:]')" &&
  ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&
  KREW="krew-${OS}_${ARCH}" &&
  curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" &&
  tar zxvf "${KREW}.tar.gz" &&
  ./"${KREW}" install krew
)

Add the $HOME/.krew/bin directory to your PATH environment variable. To do this, update your .bashrc file and append the following line, as shown below.

export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"

and restart your shell. Run kubectl krew to check the installation, as shown below.

Check the version of krew and its working using the command given below:

kubectl krew version

Installing the MinIO Server

Now, we can install the MinIO Server using the following commands as shown in the images (11)

kubectl krew install minio

kubectl minio init

kubectl minio tenant create tenant1 --servers 4 --volumes 16 --capacity 16Ti -n minio-operator

We can also see the resulting objects from the GCP Console as shown below.

3. c. Installing MinIO To The Kubernetes Cluster Via Helm

We can also install MinIO to the Kubernetes cluster via Helm.

This chart bootstraps the MinIO Cluster on Kubernetes using the Helm package manager. (12)

Add repository:

helm repo add minio-official https://charts.min.io

Install chart:

helm install my-minio minio-official/minio --version 5.0.15

For detailed configuration and important points on installation with Helm, see the address in the link and the link.

3. d. Accessing The MinIO Console

Use the kubectl port-forward command to temporarily forward traffic from the MinIO pod to the local machine:

kubectl port-forward pod/minio 9000 9090 -n minio-dev

The command forwards the pod ports 9000 and 9090 to the matching port on the local machine while active in the shell. (Note: The kubectl port-forward command only functions while active in the shell session. Terminating the session closes the ports on the local machine.)

Access the MinIO Console by opening a browser on the local machine and navigating to http://your-public-IP:9001. You can log in to the Console with the credentials minioadmin | minioadmin. These are the default root user credentials.

4. Installing The MinIO Object Storage For Linux

This procedure deploys a Standalone MinIO server onto Linux for early development and evaluation of MinIO Object Storage and its S3-compatible API layer. (13) As with Linux, downloads are also available for Mac and Windows.

Use the following commands to download the latest stable MinIO DEB (Debian/Ubuntu) and install it: (13)

wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio_20240214213602.0.0_amd64.deb -O minio.deb
sudo dpkg -i minio.deb

Launch the MinIO Server

mkdir ~/minio
minio server ~/minio --console-address :9001

4. a. Accessing The MinIO Console In Linux

Open http://127.0.0.1:9001 or http://your-server-publicIP:9001 in a web browser to access the MinIO Console. While the port 9000 is used for connecting to the API, MinIO automatically redirects browser access to the MinIO Console.

If you noticed, this start page below is different from the MinIO Server’s start page we created with Docker Compose above.

Log in to the Console with the RootUser and RootPass user credentials displayed in the output. These default to minioadmin | minioadmin.

After entering our username and password, we will see the MinIO Server’s home page, as seen below.

5. Using MinIO As A Object Storage: Uploading And Sharing A File

Upload a file to cumhurminio bucket, as seen below.

Note: Our bucket names become our “folder names”. When we create another bucket, another folder named the relevant bucket will be created under the “data” folder in Minio object storage.

Uploaded the file named “pets.png” to thecumhurminio bucket, as seen below. The cumhurminio bucket has “PRIVATE” access.

Click the “Share” button to get the file-sharing link, as seen below.

Copy the file-sharing link, as seen below.

Paste it to your browser. We can now access and view the object file we uploaded into the MinIO storage through our Browser, as seen below.

We can change the access policy from the menu below as shown in the picture. By navigating, “Bucket > Manage > Access Policy:private”

6. As a result

In this article, we practically installed MinIO for Docker, Kubernetes, and Linux. Then, we uploaded an object file into MinIO and accessed the object file through our Browser by using a file-sharing link. We saw that we could use MinIO instead of object storage solutions of cloud providers such as AWS, Azure, and GCP.

If you liked the article, I would be happy if you click on the Medium Following button to encourage me to write and not miss future articles.

Your clap, follow, or subscribe, they help my articles to reach the broader audience. Thank you in advance for them.

For more info and questions, don’t hesitate to get in touch with me on Linkedin or Medium.

7. Next post

In the next post, “MinIO-2: Using the MinIO CLI Tool And AWS-SDK With MinIO”. We will install and use the MinIO CLI tool. Then, we will use the MinIO Bucket instead of the AWS S3 Bucket to save video and picture recordigs, in an application that was written with the Django framework.