returnnil,-1,fmt.Errorf("All zones should be of the same setup-type to maintain the original SLA expectations - expected %s, got %s",prevSetupType,setupType)
returnnil,-1,fmt.Errorf("All serverSets should be of the same setup-type to maintain the original SLA expectations - expected %s, got %s",prevSetupType,setupType)
@ -94,37 +94,37 @@ Input for the key is the object name specified in `PutObject()`, returns a uniqu
- MinIO does erasure coding at the object level not at the volume level, unlike other object storage vendors. This allows applications to choose different storage class by setting `x-amz-storage-class=STANDARD/REDUCED_REDUNDANCY` for each object uploads so effectively utilizing the capacity of the cluster. Additionally these can also be enforced using IAM policies to make sure the client uploads with correct HTTP headers.
- MinIO does erasure coding at the object level not at the volume level, unlike other object storage vendors. This allows applications to choose different storage class by setting `x-amz-storage-class=STANDARD/REDUCED_REDUNDANCY` for each object uploads so effectively utilizing the capacity of the cluster. Additionally these can also be enforced using IAM policies to make sure the client uploads with correct HTTP headers.
- MinIO also supports expansion of existing clusters in zones. Each zone is a self contained entity with same SLA's (read/write quorum) for each object as original cluster. By using the existing namespace for lookup validation MinIO ensures conflicting objects are not created. When no such object exists then MinIO simply uses the least used zone.
- MinIO also supports expansion of existing clusters in server sets. Each zone is a self contained entity with same SLA's (read/write quorum) for each object as original cluster. By using the existing namespace for lookup validation MinIO ensures conflicting objects are not created. When no such object exists then MinIO simply uses the least used zone.
__There are no limits on how many zones can be combined__
__There are no limits on how many server sets can be combined__
```
```
minio server http://host{1...32}/export{1...32} http://host{5...6}/export{1...8}
minio server http://host{1...32}/export{1...32} http://host{5...6}/export{1...8}
```
```
In above example there are two zones
In above example there are two server sets
- 32 * 32 = 1024 drives zone1
- 32 * 32 = 1024 drives zone1
- 2 * 8 = 16 drives zone2
- 2 * 8 = 16 drives zone2
> Notice the requirement of common SLA here original cluster had 1024 drives with 16 drives per erasure set, second zone is expected to have a minimum of 16 drives to match the original cluster SLA or it should be in multiples of 16.
> Notice the requirement of common SLA here original cluster had 1024 drives with 16 drives per erasure set, second zone is expected to have a minimum of 16 drives to match the original cluster SLA or it should be in multiples of 16.
MinIO places new objects in zones based on proportionate free space, per zone. Following pseudo code demonstrates this behavior.
MinIO places new objects in server sets based on proportionate free space, per zone. Following pseudo code demonstrates this behavior.
```go
```go
func getAvailableZoneIdx(ctx context.Context) int {
func getAvailableZoneIdx(ctx context.Context) int {
zones := z.getZonesAvailableSpace(ctx)
serverSets := z.getServerSetsAvailableSpace(ctx)
total := zones.TotalAvailable()
total := serverSets.TotalAvailable()
// choose when we reach this many
// choose when we reach this many
choose := rand.Uint64() % total
choose := rand.Uint64() % total
atTotal := uint64(0)
atTotal := uint64(0)
for _, zone := range zones {
for _, zone := range serverSets {
atTotal += zone.Available
atTotal += zone.Available
if atTotal > choose && zone.Available > 0 {
if atTotal > choose && zone.Available > 0 {
return zone.Index
return zone.Index
}
}
}
}
// Should not happen, but print values just in case.
// Should not happen, but print values just in case.
panic(fmt.Errorf("reached end of zones (total: %v, atTotal: %v, choose: %v)", total, atTotal, choose))
panic(fmt.Errorf("reached end of serverSets (total: %v, atTotal: %v, choose: %v)", total, atTotal, choose))
minio server http://host{1...4}/export{1...16} http://host{5...12}/export{1...16}
minio server http://host{1...4}/export{1...16} http://host{5...12}/export{1...16}
```
```
Now the server has expanded total storage by _(newly_added_servers\*m)_ more disks, taking the total count to _(existing_servers\*m)+(newly_added_servers\*m)_ disks. New object upload requests automatically start using the least used cluster. This expansion strategy works endlessly, so you can perpetually expand your clusters as needed. When you restart, it is immediate and non-disruptive to the applications. Each group of servers in the command-line is called a zone. There are 2 zones in this example. New objects are placed in zones in proportion to the amount of free space in each zone. Within each zone, the location of the erasure-set of drives is determined based on a deterministic hashing algorithm.
Now the server has expanded total storage by _(newly_added_servers\*m)_ more disks, taking the total count to _(existing_servers\*m)+(newly_added_servers\*m)_ disks. New object upload requests automatically start using the least used cluster. This expansion strategy works endlessly, so you can perpetually expand your clusters as needed. When you restart, it is immediate and non-disruptive to the applications. Each group of servers in the command-line is called a zone. There are 2 server sets in this example. New objects are placed in server sets in proportion to the amount of free space in each zone. Within each zone, the location of the erasure-set of drives is determined based on a deterministic hashing algorithm.
> __NOTE:__ __Each zone you add must have the same erasure coding set size as the original zone, so the same data redundancy SLA is maintained.__
> __NOTE:__ __Each zone you add must have the same erasure coding set size as the original zone, so the same data redundancy SLA is maintained.__
> For example, if your first zone was 8 drives, you could add further zones of 16, 32 or 1024 drives each. All you have to make sure is deployment SLA is multiples of original data redundancy SLA i.e 8.
> For example, if your first zone was 8 drives, you could add further server sets of 16, 32 or 1024 drives each. All you have to make sure is deployment SLA is multiples of original data redundancy SLA i.e 8.
## 3. Test your setup
## 3. Test your setup
To test this setup, access the MinIO server via browser or [`mc`](https://docs.min.io/docs/minio-client-quickstart-guide).
To test this setup, access the MinIO server via browser or [`mc`](https://docs.min.io/docs/minio-client-quickstart-guide).
Harshavardhana
4 years ago
committed by