add bucket versioning zh_CN document (#10281)
飞雪无情
4 years ago
committed by
GitHub
parent
b43d376a87
commit
2eb5f934d8
@ -0,0 +1,79 @@ |
||||
# Bucket Versioning Design Guide [](https://slack.min.io) [](https://hub.docker.com/r/minio/minio/) |
||||
|
||||
## Description of `xl.meta` |
||||
|
||||
`xl.meta` is a new self describing backend format used by MinIO to support AWS S3 compatible versioning. This file is the source of truth for each `version` at rest. `xl.meta` is a msgpack file serialized from a well defined data structure. To understand `xl.meta` here are the few things to start with |
||||
|
||||
`xl.meta` carries first 8 bytes an XL header which describes the current format and the format version, allowing the unmarshaller's to automatically use the right data structures to parse the subsequent content in the stream. |
||||
|
||||
These are the current entries |
||||
```go |
||||
var ( |
||||
// XL header specifies the format |
||||
// one extra byte left for future use |
||||
xlHeader = [4]byte{'X', 'L', '2', ' '} |
||||
|
||||
// XLv2 version 1 specifies the current |
||||
// version of the XLv2 format, 3 extra bytes |
||||
// left for future use. |
||||
xlVersionV1 = [4]byte{'1', ' ', ' ', ' '} |
||||
) |
||||
``` |
||||
|
||||
Once the header is validated, we proceed to the actual data structure of the `xl.meta` format. `xl.meta` carries three types of object entries which designate the type of version object stored. |
||||
|
||||
- ObjectType (default) |
||||
- LegacyObjectType (preserves existing deployments and older xl.json format) |
||||
- DeleteMarker (a versionId to capture the DELETE sequences implemented primarily for AWS spec compatibility) |
||||
|
||||
A sample msgpack-JSON `xl.meta`, you can debug the content inside `xl.meta` using [xl-meta-to-json.go](https://github.com/minio/minio/blob/master/docs/bucket/versioning/xl-meta-to-json.go) program. |
||||
```json |
||||
{ |
||||
"Versions": [ |
||||
{ |
||||
"Type": 1, |
||||
"V2Obj": { |
||||
"ID": "KWUs8S+8RZq4Vp5TWy6KFg==", |
||||
"DDir": "X3pDAFu8Rjyft7QD6t7W5g==", |
||||
"EcAlgo": 1, |
||||
"EcM": 2, |
||||
"EcN": 2, |
||||
"EcBSize": 10485760, |
||||
"EcIndex": 3, |
||||
"EcDist": [ |
||||
3, |
||||
4, |
||||
1, |
||||
2 |
||||
], |
||||
"CSumAlgo": 1, |
||||
"PartNums": [ |
||||
1 |
||||
], |
||||
"PartETags": [ |
||||
"" |
||||
], |
||||
"PartSizes": [ |
||||
314 |
||||
], |
||||
"PartASizes": [ |
||||
282 |
||||
], |
||||
"Size": 314, |
||||
"MTime": 1591820730, |
||||
"MetaSys": { |
||||
"X-Minio-Internal-Server-Side-Encryption-S3-Kms-Key-Id": "bXktbWluaW8ta2V5", |
||||
"X-Minio-Internal-Server-Side-Encryption-S3-Kms-Sealed-Key": "ZXlKaFpXRmtJam9pUVVWVExUSTFOaTFIUTAwdFNFMUJReTFUU0VFdE1qVTJJaXdpYVhZaU9pSkJMMVZzZFVnelZYVjZSR2N6UkhGWUwycEViRmRCUFQwaUxDSnViMjVqWlNJNklpdE9lbkJXVWtseFlWSlNVa2t2UVhNaUxDSmllWFJsY3lJNklrNDBabVZsZG5WU1NWVnRLMFoyUWpBMVlYTk9aMU41YVhoU1RrNUpkMDlhTkdKa2RuaGpLMjFuVDNnMFFYbFJhbE15V0hkU1pEZzNRMk54ZUN0SFFuSWlmUT09", |
||||
"X-Minio-Internal-Server-Side-Encryption-Seal-Algorithm": "REFSRXYyLUhNQUMtU0hBMjU2", |
||||
"X-Minio-Internal-Server-Side-Encryption-Iv": "bW5YRDhRUGczMVhkc2pJT1V1UVlnbWJBcndIQVhpTUN1dnVBS0QwNUVpaz0=", |
||||
"X-Minio-Internal-Server-Side-Encryption-S3-Sealed-Key": "SUFBZkFPeUo5ZHVVSEkxYXFLU0NSRkJTTnM0QkVJNk9JWU1QcFVTSXFhK2dHVThXeE9oSHJCZWwwdnRvTldUNE8zS1BtcWluR0cydmlNNFRWa0N0Mmc9PQ==" |
||||
}, |
||||
"MetaUsr": { |
||||
"content-type": "application/octet-stream", |
||||
"etag": "20000f00f58c508b40720270929bd90e9f07b9bd78fb605e5432a67635fc34722e4fc53b1d5fab9ff8400eb9ded4fba2" |
||||
} |
||||
} |
||||
} |
||||
] |
||||
} |
||||
``` |
||||
@ -0,0 +1,166 @@ |
||||
# 存储桶版本控制指南 [](https://slack.min.io) [](https://hub.docker.com/r/minio/minio/) |
||||
|
||||
MinIO的版本控制，可以让一个存储通里的某个对象具有多个版本。例如，你可以存储`spark.csv` (版本为 `ede336f2`) 和 `spark.csv` (版本为 `fae684da`)在一个存储通中。版本控制保护你免受意外覆盖、删除、应用保留策略和存档对象的影响。 |
||||
|
||||
要自定义数据保留和存储使用情况，请在对象生命周期管理中使用对象版本控制。如果你在不受版本控制的存储桶中具有对象到期生命周期策略，并且希望在启用版本控制时保持相同的永久删除行为，则必须添加非当前版本到期策略。非当前版本到期生命周期策略将管理在受版本控制的存储桶中删除非当前对象版本的行为。(启用版本控制的存储桶会维护一个当前对象版本，以及零个或零个以上非当前对象版本。) |
||||
|
||||
必须在存储桶上显式启用版本控制，默认情况下不启用版本控制。启用对象锁定的存储桶会自动启用版本控制。启用和暂停版本控制是在存储桶级别完成的。 |
||||
|
||||
只有MinIO能生成版本ID，并且无法对其进行编辑。版本ID是`DCE 1.1 v4 UUID 4`(基于随机数据)，UUID是128位数字，旨在在空间和时间上具有很高的唯一性，并且难以通过计算猜测。它们是全局唯一的，可以在不联网一个全局注册服务器的情况下在本地生成。UUID旨在用作生命周期非常短的大量标记对象的唯一标识符，并可以可靠地标识整个网络上非常持久的对象。 |
||||
|
||||
当您在启用版本控制的存储桶中PUT一个对象时，非当前版本不会被覆盖。下图显示，当将新版本的`spark.csv`放入已经包含相同名称对象的存储桶中时，原始对象（ID = `ede336f2`）保留在存储桶中，MinIO生成新版本（ID = `fae684da`），并将新版本添加到存储桶中。 |
||||
|
||||
 |
||||
|
||||
这意味着对对象的意外覆盖或删除进行了保护，允许检索对象的先前版本。 |
||||
|
||||
删除对象时，所有版本都保留在存储桶中，MinIO添加删除标记，如下所示： |
||||
|
||||
 |
||||
|
||||
现在，删除标记成为对象的当前版本。默认情况下，GET请求始终检索最新的存储版本。因此，当当前版本为删除标记时，执行简单的GET对象请求将返回`404` `The specified key does not exist`，如下所示： |
||||
|
||||
 |
||||
|
||||
通过指定如下所示的版本ID进行GET请求，你可以检索特定的对象版本`fae684da`。 |
||||
|
||||
 |
||||
|
||||
要永久删除对象，你需要指定要删除的版本，只有具有适当权限的用户才能永久删除版本。如下所示，使用特定版本ID调用的DELETE请求从存储桶中永久删除一个对象。带版本id的DELETE请求不会添加删除标记。 |
||||
|
||||
 |
||||
|
||||
## 概念 |
||||
- MinIO上的所有存储桶始终处于以下状态之一：无版本控制（默认），启用版本控制或暂停版本控制。 |
||||
- 版本控制状态应用于启用版本控制的存储桶中的所有对象。首次启用存储桶版本控制后，将始终对存储桶中的对象进行版本控制并为其指定唯一的版本ID。 |
||||
- 现存或者新建的存储通都能启用版本控制，最终也可以将其暂停。 对象的现有版本保持不变，并且仍可以使用版本ID进行访问。 |
||||
- 在删除存储桶之前，应删除所有版本，包括删除标记。 |
||||
- **版本控制功能仅在纠删码和分布式纠删码模式下可用**。 |
||||
|
||||
## 如何在存储桶上配置版本控制 |
||||
创建的每个存储桶都有与其关联的版本控制配置。默认情况下，存储桶是无版本控制的，如下所示 |
||||
``` |
||||
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> |
||||
</VersioningConfiguration> |
||||
``` |
||||
|
||||
要启用版本控制，你可以发送一个Status为`Enabled`的版本控制配置请求到MinIO。 |
||||
``` |
||||
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> |
||||
<Status>Enabled</Status> |
||||
</VersioningConfiguration> |
||||
``` |
||||
|
||||
同样的，要暂停版本控制，把Status配置设置为`Suspended`即可。 |
||||
``` |
||||
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> |
||||
<Status>Suspended</Status> |
||||
</VersioningConfiguration> |
||||
``` |
||||
|
||||
只有具有相应的权限或者root用户才能配置任何存储桶的版本控制状态。 |
||||
|
||||
## 使用MinIO Java SDK启用存储通版本控制的示例 |
||||
|
||||
### EnableVersioning() API |
||||
|
||||
``` |
||||
import io.minio.EnableVersioningArgs; |
||||
import io.minio.MinioClient; |
||||
import io.minio.errors.MinioException; |
||||
import java.io.IOException; |
||||
import java.security.InvalidKeyException; |
||||
import java.security.NoSuchAlgorithmException; |
||||
|
||||
public class EnableVersioning { |
||||
/** MinioClient.enableVersioning() example. */ |
||||
public static void main(String[] args) |
||||
throws IOException, NoSuchAlgorithmException, InvalidKeyException { |
||||
try { |
||||
/* play.min.io for test and development. */ |
||||
MinioClient minioClient = |
||||
MinioClient.builder() |
||||
.endpoint("https://play.min.io") |
||||
.credentials("Q3AM3UQ867SPQQA43P2F", "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG") |
||||
.build(); |
||||
|
||||
/* Amazon S3: */ |
||||
// MinioClient minioClient = |
||||
// MinioClient.builder() |
||||
// .endpoint("https://s3.amazonaws.com") |
||||
// .credentials("YOUR-ACCESSKEY", "YOUR-SECRETACCESSKEY") |
||||
// .build(); |
||||
|
||||
// Enable versioning on 'my-bucketname'. |
||||
minioClient.enableVersioning(EnableVersioningArgs.builder().bucket("my-bucketname").build()); |
||||
|
||||
System.out.println("Bucket versioning is enabled successfully"); |
||||
|
||||
} catch (MinioException e) { |
||||
System.out.println("Error occurred: " + e); |
||||
} |
||||
} |
||||
} |
||||
``` |
||||
|
||||
### isVersioningEnabled() API |
||||
|
||||
``` |
||||
public class IsVersioningEnabled { |
||||
/** MinioClient.isVersioningEnabled() example. */ |
||||
public static void main(String[] args) |
||||
throws IOException, NoSuchAlgorithmException, InvalidKeyException { |
||||
try { |
||||
/* play.min.io for test and development. */ |
||||
MinioClient minioClient = |
||||
MinioClient.builder() |
||||
.endpoint("https://play.min.io") |
||||
.credentials("Q3AM3UQ867SPQQA43P2F", "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG") |
||||
.build(); |
||||
|
||||
/* Amazon S3: */ |
||||
// MinioClient minioClient = |
||||
// MinioClient.builder() |
||||
// .endpoint("https://s3.amazonaws.com") |
||||
// .credentials("YOUR-ACCESSKEY", "YOUR-SECRETACCESSKEY") |
||||
// .build(); |
||||
|
||||
// Create bucket 'my-bucketname' if it doesn`t exist. |
||||
if (!minioClient.bucketExists(BucketExistsArgs.builder().bucket("my-bucketname").build())) { |
||||
minioClient.makeBucket(MakeBucketArgs.builder().bucket("my-bucketname").build()); |
||||
System.out.println("my-bucketname is created successfully"); |
||||
} |
||||
|
||||
boolean isVersioningEnabled = |
||||
minioClient.isVersioningEnabled( |
||||
IsVersioningEnabledArgs.builder().bucket("my-bucketname").build()); |
||||
if (isVersioningEnabled) { |
||||
System.out.println("Bucket versioning is enabled"); |
||||
} else { |
||||
System.out.println("Bucket versioning is disabled"); |
||||
} |
||||
// Enable versioning on 'my-bucketname'. |
||||
minioClient.enableVersioning(EnableVersioningArgs.builder().bucket("my-bucketname").build()); |
||||
System.out.println("Bucket versioning is enabled successfully"); |
||||
|
||||
isVersioningEnabled = |
||||
minioClient.isVersioningEnabled( |
||||
IsVersioningEnabledArgs.builder().bucket("my-bucketname").build()); |
||||
if (isVersioningEnabled) { |
||||
System.out.println("Bucket versioning is enabled"); |
||||
} else { |
||||
System.out.println("Bucket versioning is disabled"); |
||||
} |
||||
|
||||
} catch (MinioException e) { |
||||
System.out.println("Error occurred: " + e); |
||||
} |
||||
} |
||||
} |
||||
``` |
||||
|
||||
## 进一步探索 |
||||
- [使用`minio-java` SDK](https://docs.minio.io/cn/java-client-quickstart-guide.html) |
||||
- [对象锁定指南](https://docs.minio.io/docs/minio-bucket-object-lock-guide.html) |
||||
- [MinIO管理指南](https://docs.min.io/docs/minio-admin-complete-guide.html) |
||||
- [MinIO官方文档](https://docs.min.io/cn/) |
||||
|
After Width: | Height: | Size: 144 KiB |
|
After Width: | Height: | Size: 123 KiB |
|
After Width: | Height: | Size: 134 KiB |
|
After Width: | Height: | Size: 169 KiB |
|
After Width: | Height: | Size: 103 KiB |
@ -0,0 +1,121 @@ |
||||
/* |
||||
* MinIO Cloud Storage, (C) 2020 MinIO, Inc. |
||||
* |
||||
* Licensed under the Apache License, Version 2.0 (the "License"); |
||||
* you may not use this file except in compliance with the License. |
||||
* You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package main |
||||
|
||||
import ( |
||||
"bytes" |
||||
"encoding/json" |
||||
"fmt" |
||||
"io" |
||||
"log" |
||||
"os" |
||||
|
||||
"github.com/minio/cli" |
||||
"github.com/tinylib/msgp/msgp" |
||||
) |
||||
|
||||
var xlHeader = [4]byte{'X', 'L', '2', ' '} |
||||
|
||||
func main() { |
||||
app := cli.NewApp() |
||||
app.Copyright = "MinIO, Inc." |
||||
app.Usage = "xl.meta to JSON" |
||||
app.HideVersion = true |
||||
app.CustomAppHelpTemplate = `NAME: |
||||
{{.Name}} - {{.Usage}} |
||||
|
||||
USAGE: |
||||
{{.Name}} {{if .VisibleFlags}}[FLAGS]{{end}} METAFILES... |
||||
{{if .VisibleFlags}} |
||||
GLOBAL FLAGS: |
||||
{{range .VisibleFlags}}{{.}} |
||||
{{end}}{{end}} |
||||
` |
||||
|
||||
app.HideHelpCommand = true |
||||
|
||||
app.Flags = []cli.Flag{ |
||||
cli.BoolFlag{ |
||||
Usage: "Print each file as a separate line without formatting", |
||||
Name: "ndjson", |
||||
}, |
||||
} |
||||
|
||||
app.Action = func(c *cli.Context) error { |
||||
if !c.Args().Present() { |
||||
cli.ShowAppHelp(c) |
||||
return nil |
||||
} |
||||
for _, file := range c.Args() { |
||||
var r io.Reader |
||||
switch file { |
||||
case "-": |
||||
r = os.Stdin |
||||
default: |
||||
f, err := os.Open(file) |
||||
if err != nil { |
||||
return err |
||||
} |
||||
defer f.Close() |
||||
r = f |
||||
} |
||||
|
||||
// Read header
|
||||
var tmp [4]byte |
||||
_, err := io.ReadFull(r, tmp[:]) |
||||
if err != nil { |
||||
return err |
||||
} |
||||
if !bytes.Equal(tmp[:], xlHeader[:]) { |
||||
return fmt.Errorf("xlMeta: unknown XLv2 header, expected %v, got %v", xlHeader[:4], tmp[:4]) |
||||
} |
||||
// Skip version check for now
|
||||
_, err = io.ReadFull(r, tmp[:]) |
||||
if err != nil { |
||||
return err |
||||
} |
||||
|
||||
var buf bytes.Buffer |
||||
_, err = msgp.CopyToJSON(&buf, r) |
||||
if err != nil { |
||||
return err |
||||
} |
||||
if c.Bool("ndjson") { |
||||
fmt.Println(buf.String()) |
||||
continue |
||||
} |
||||
var msi map[string]interface{} |
||||
dec := json.NewDecoder(&buf) |
||||
// Use number to preserve integers.
|
||||
dec.UseNumber() |
||||
err = dec.Decode(&msi) |
||||
if err != nil { |
||||
return err |
||||
} |
||||
b, err := json.MarshalIndent(msi, "", " ") |
||||
if err != nil { |
||||
return err |
||||
} |
||||
fmt.Println(string(b)) |
||||
} |
||||
return nil |
||||
} |
||||
err := app.Run(os.Args) |
||||
if err != nil { |
||||
log.Fatal(err) |
||||
} |
||||
} |
||||
Loading…
Reference in new issue