@ -18,9 +18,9 @@ For example, an 8-node distributed Minio setup, with 1 disk per node would stay
### Limits
As with Minio in stand-alone mode, distributed Minio has a per tenant limit of minimum 4 and maximum 16 drives (imposed by erasure code). This helps maintain simplicity and yet remain scalable. If you need a multiple tenant setup, you can easily spin multiple Minio instances managed by orchestration tools like Kubernetes.
As with Minio in stand-alone mode, distributed Minio has a per tenant limit of minimum 2 and maximum 32 servers. There are no limits on number of disks shared across these servers. If you need a multiple tenant setup, you can easily spin multiple Minio instances managed by orchestration tools like Kubernetes.
Note that with distributed Minio you can play around with the number of nodes and drives as long as the limits are adhered to. For example, you can have 2 nodes with 4 drives each, 4 nodes with 4 drives each, 8 nodes with 2 drives each, and so on.
Note that with distributed Minio you can play around with the number of nodes and drives as long as the limits are adhered to. For example, you can have 2 nodes with 4 drives each, 4 nodes with 4 drives each, 8 nodes with 2 drives each, 32 servers with 24 drives each and so on.
You can also use [storage classes](https://github.com/minio/minio/tree/master/docs/erasure/storage-class) to set custom data and parity distribution across total disks.
@ -112,6 +112,7 @@ minio.exe server http://192.168.1.11/C:/data1 http://192.168.1.11/C:/data2 ^
To test this setup, access the Minio server via browser or [`mc`](https://docs.minio.io/docs/minio-client-quickstart-guide). You’ll see the combined capacity of all the storage drives as the capacity of this drive.
## Explore Further
- [Minio Large Bucket Suppport Guide](https://docs.minio.io/docs/minio-large-bucket-support-quickstart-guide)
@ -20,7 +20,7 @@ Erasure code protects data from multiple drives failure, unlike RAID or replicat
Bit Rot, also known as data rot or silent data corruption is a data loss issue faced by disk drives today. Data on the drive may silently get corrupted without signalling an error has occurred, making bit rot more dangerous than a permanent hard drive failure.
Minio's erasure coded backend uses high speed [BLAKE2](https://blog.minio.io/accelerating-blake2b-by-4x-using-simd-in-go-assembly-33ef16c8a56b#.jrp1fdwer) hash based checksums to protect against Bit Rot.
Minio's erasure coded backend uses high speed [HighwayHash](https://blog.minio.io/highwayhash-fast-hashing-at-over-10-gb-s-per-core-in-golang-fee938b5218a) checksums to protect against Bit Rot.
# Large Bucket Support Quickstart Guide [](https://slack.minio.io) [](https://goreportcard.com/report/minio/minio) [](https://hub.docker.com/r/minio/minio/) [](https://codecov.io/gh/minio/minio)
Minio erasure code limits the number of disks in a deployment to 16. This allows enough storage space for a single deployment to hold a tenant's data. However, there are use cases which
may need larger number of disks/storage space upfront. Keeping this in consideration, we added large bucket support.
Erasure code in Minio is limited to 16 disks. This on its own allows storage space to hold a tenant's data. However, to cater to cases which may need larger number of disks or high storage space requirements, large bucket support was introduced.
The feature is called large bucket as it allows a single bucket to expand over multiple erasure code deployment sets, without any special configuration.
With large bucket support, you can use more than 16 disks upfront while deploying the Minio server. Internally Minio creates multiple smaller erasure coded sets, and these sets
are further combined into a single namespace. This document gives a brief introduction on how to get started with large bucket deployments. To explore further on advanced uses and
limitations, refer to the [design document](https://github.com/minio/minio/blob/master/docs/large-bucket/DESIGN.md).
We call it large-bucket as it allows a single Minio bucket to expand over multiple erasure code deployment sets. Without any special configuration it helps you create peta-scale storage systems. With large bucket support, you can use more than 16 disks upfront while deploying the Minio server. Internally Minio creates multiple smaller erasure coded sets, and these sets are further combined into a single namespace. This document gives a brief introduction on how to get started with large bucket deployments. To explore further on advanced uses and limitations, refer to the [design document](https://github.com/minio/minio/blob/master/docs/large-bucket/DESIGN.md).
## Get started
If you're aware of distributed Minio setup, the installation and running remains the same. Only new addition to the syntax is a`...` convention to abbreviate the directory arguments. Remote directories in a distributed setup are encoded as HTTP(s) URIs which can be similarly abbreviated as well.
If you're aware of distributed Minio setup, the installation and configuration remains the same. Only new addition to the input syntax is `...` convention to abbreviate the drive arguments. Remote drives in a distributed setup are encoded as HTTP(s) URIs which can be similarly abbreviated as well.
To run Minio large bucket instances, you need to start multiple Minio servers pointing to the same disks. We'll see examples on how to do this in the following sections.
### 2. Run Minio on many drives
We'll see examples on how to do this in the following sections.
*Note*
- All the nodes running distributed Minio need to have same access key and secret key. To achieve this, we export access key and secret key as environment variables on all the nodes before executing Minio server command.
- The drive paths below are for demonstration purposes only, you need to replace these with the actual drive paths/folders.
#### Minio large bucket on Ubuntu 16.04 LTS server (standalone)
#### Minio large bucket on multiple drives (standalone)
You'll need the path to the disks e.g. `/export1, /export2 .... /export24`. Then run the following commands on all the nodes you'd like to launch Minio.
#### Minio large bucket on Ubuntu 16.04 LTS servers (distributed)
#### Minio large bucket on multiple servers (distributed)
You'll need the path to the disks e.g. `http://host1/export1, http://host2/export2 .... http://host4/export16`. Then run the following commands on all the nodes you'd like to launch Minio.
Harshavardhana
7 years ago
committed by