oyd
/
minio
6
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity

fix: bucket replication docs (#10104)

Browse Source 
* fix: bucket replication docs

* Update docs/bucket/replication/README.md

Co-authored-by: kannappanr <30541348+kannappanr@users.noreply.github.com>

Co-authored-by: kannappanr <30541348+kannappanr@users.noreply.github.com>
master
Harshavardhana 4 years ago committed by GitHub
parent c43da3005a
commit 0b5d1bc91d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 49 additions and 16 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats Download Patch File Download Diff File
  1. 65
      docs/bucket/replication/README.md

65
docs/bucket/replication/README.md
Unescape View File

@ -1,35 +1,68 @@
# Bucket Replication Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
Bucket replication is designed to replicate specific objects in buckets
that are configured for replication.
Bucket replication is designed to replicate specific objects in buckets that are configured for replication.
To replicate objects in a bucket to a destination bucket on a target site either on the same cluster or a different cluster, start by creating version enabled buckets on both `source` and `dest` buckets. Next, the target site and destination bucket need to be configured on MinIO server by setting
`
$ mc admin bucket replication set myminio/source https://accessKey:secretKey@replica-endpoint:9000/dest --path-style "auto" --api "s3v2"
`
## Highlights
- Supports source and destination buckets to have the same name unlike AWS S3, addresses variety of usecases such as *Splunk*, *Veeam* site to site DR.
- Supports object locking/retention across source and destination buckets natively out of the box, unlike AWS S3.
- Simpler implementation than [AWS S3 Bucket Replication Config](https://docs.aws.amazon.com/AmazonS3/latest/dev/replication-add-config.html) with requirements such as IAM Role, AccessControlTranslation, Metrics and SourceSelectionCriteria are not needed with MinIO.
Note that the admin needs "s3:GetReplicationConfigurationAction" permission on source in addition to "s3:ReplicateObject" permission on destination side.
## How to use?
Create a replication target on the source cluster as shown below:
Next, the replication ARN associated with the replica URL https://replica-endpoint:9000 can be fetched from the MinIO server by using the command
`mc admin bucket remote myminio/source https://replica-endpoint:9000`
```
mc admin bucket replication set myminio/srcbucket https://accessKey:secretKey@replica-endpoint:9000/destbucket --path ON --api s3v4
Replication ARN = 'arn:minio:s3::dadddae7-f1d7-440f-b5d6-651aa9a8c8a7:*'
```
Note that the admin needs *s3:GetReplicationConfigurationAction* permission on source cluster. The credential used at the destination requires *s3:ReplicateObject* permission. Once successfully created and authorized this generates a replication target ARN. The command below lists all the currently authorized replication targets:
```
mc admin bucket remote myminio/srcbucket https://replica-endpoint:9000
Replication ARN = 'arn:minio:s3::dadddae7-f1d7-440f-b5d6-651aa9a8c8a7:*'
```
The replication configuration can now be added to the source bucket by applying the json file with replication configuration. The ReplicationArn is passed in as a json element in the configuration.
The replication configuration follows [S3 Spec](https://docs.aws.amazon.com/AmazonS3/latest/dev/replication-add-config.html). Any
objects uploaded to the source bucket that meet replication criteria will now automatically be replicated by the MinIO server. Replication can be stopped at any time by disabling specific rules in the configuration or deleting the replication configuration.
```json
{
"ReplicationArn" : "arn:minio:s3::dadddae7-f1d7-440f-b5d6-651aa9a8c8a7:*",
"Rules": [
{
"Status": "Enabled",
"Priority": 1,
"DeleteMarkerReplication": { "Status": "Disabled" },
"Filter" : { "Prefix": "Tax"},
"Destination": {
"Bucket": "arn:aws:s3:::destbucket",
"StorageClass": "STANDARD"
}
}
]
}
```
```
mc bucket replicate myminio/srcbucket --config replicate-config.json
Replication configuration applied successfully to myminio/srcbucket.
```
Apart from *ReplicationArn* , rest of the configuration follows [AWS S3 Spec](https://docs.aws.amazon.com/AmazonS3/latest/dev/replication-add-config.html). Any objects uploaded to the source bucket that meet replication criteria will now be automatically replicated by the MinIO server to the remote destination bucket. Replication can be disabled at any time by disabling specific rules in the configuration or deleting the replication configuration entirely.
When an object is deleted from the source bucket, the replica will not be deleted as per S3 spec.
![delete](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/replication/DELETE_bucket_replication.png)
When object locking is used in conjunction with replication, both source and destination buckets needs to have object locking enabled. Similarly objects encrypted on the server side, will be replicated if destination also supports
encryption.
When object locking is used in conjunction with replication, both source and destination buckets needs to have object locking enabled. Similarly objects encrypted on the server side, will be replicated if destination also supports encryption.
Replication status can be seen in the metadata on the source and destination objects. On the source side, the `X-Amz-Replication-Status` changes from `PENDING` to `COMPLETE` or `FAILED` after replication attempt either succeeded or failed respectively. On the destination side, a `X-Amz-Replication-Status` status of `REPLICA` indicates that the object was replicated successfully. Any replication failures are automatically re-attempted during a periodic disk crawl cycle.
Replication status can be seen in the metadata on the source and destination objects. On the source side, the `X-Amz-Replication-Status` changes from `PENDING` to `COMPLETE` or `FAILED` after replication attempt is made. On the destination side, a `X-Amz-Replication-Status` status of `REPLICA` indicates that the object
was replicated successfully. Any replication failures are automatically reattempted during a periodic disk usage crawl.
![put](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/replication/PUT_bucket_replication.png)
![head](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/replication/HEAD_bucket_replication.png)
## Additional notes
The [S3 Spec]([S3 Spec](https://docs.aws.amazon.com/AmazonS3/latest/dev/replication-add-config.html) has configuration such as IAM Role,AccessControlTranslation, Metrics and SourceSelectionCriteria which are not required for MinIO.
## Explore Further
- [MinIO Bucket Versioning Implementation](https://docs.minio.io/docs/minio-bucket-versioning-guide.html)
- [MinIO Client Quickstart Guide](https://docs.minio.io/docs/minio-client-quickstart-guide.html)

Write Preview
Loading…
Cancel
Save