feb337098d
XL: bring in new storage API. ( #1780 )
...
Fixes #1771
9 years ago
293d246f95
XL/FS: Rewrite in new format.
9 years ago
7d6ed50fc2
objects: Save all the incoming metadata properly. ( #1688 )
...
For both multipart and single put operation
9 years ago
d267696110
Validation: Reject object names with trailing "/". ( #1619 )
...
Fixes #1616
9 years ago
75320f70d0
multipart: reject part upload if size is less than 5MB. ( #1518 )
9 years ago
ac2933c799
windows: Enable erasure test for windows. ( #1432 )
...
Fixes #1363
9 years ago
4e34e03dd4
xl/fs: Split object layer into interface. ( #1415 )
9 years ago
1813e9c070
Cleanup - Comments and readability fixes ( #1386 )
9 years ago
4333e529e6
xl/ListFiles: return as many objects as requested. ( #1383 )
...
* xl/ListFiles: return as many objects as requested and take care of eof (#1361 )
* xl/ListFiles: fix review comments.
* xl/ListFiles: Add windows filepath translation.
* xl/ListFiles: Use slashSeparator instead of "/". Remove filepath.FromSlash() as golang-windows takes care of it automatically.
9 years ago
57f35c2bcc
xl: Introduce new blocking writer to make CreateFile atomic. ( #1362 )
...
Creates a new write closer that must be released
by the read consumer. This is necessary so that
while commiting the underlying writers in erasure
coding we need to make sure we reply success only if
we have committed to disk.
This in turn also fixes plethora of bugs related to
subsequent PutObject() races with namespace locking.
This patch also enables most of the tests, other than
ListObjects paging which has some issues still.
Fixes #1358 , #1360
9 years ago
cab6805f09
xl: Enable a subset of tests for XL branch. ( #1359 )
9 years ago
8c85815106
xl: refactor functions to xl-v1-common.go xl-v1-utils.go. ( #1357 )
9 years ago
6e372f83b4
Tests: object api multipart tests and bug fixes.
9 years ago
d63d17012d
tests: Add API suite tests back for object api. ( #1352 )
9 years ago
be002ac01e
fs/object: Fix issues from review comments.
9 years ago
30b0b4deba
storage/server/client: Enable storage server, enable client storage.
9 years ago
01a439f95b
refactor: add multipart code to the object layer.
9 years ago
3c48537f20
refactor: refactor code to separate fs into object-layer and fs layer. ( #1305 )
9 years ago
33cd910d3a
backend/fs: More cleanup and start using checkBuckeArg. ( #1306 )
...
backend/fs: More cleanup and start using checkBucketArg.
9 years ago
9632c94e7a
Fix list objects test and remove all the old unnecessary files.
...
- Fix tests for new changes.
- Change Golang err as 'e' for the time being, before we bring in
probe removal change.
- Remove old structs and temporary files.
9 years ago
0479d4976b
objectAPI: Fix object API interface, remove unnecessary structs.
...
ObjectAPI changes.
```
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsInfo, *probe.Error)
ListMultipartUploads(bucket, objectPrefix, keyMarker, uploadIDMarker, delimiter string, maxUploads int) (ListMultipartsInfo, *probe.Error)
ListObjectParts(bucket, object, uploadID string, partNumberMarker, maxParts int) (ListPartsInfo, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []completePart) (ObjectInfo, *probe.Error)
```
9 years ago
efc80343e3
fs: Break fs package to top-level and introduce ObjectAPI interface.
...
ObjectAPI interface brings in changes needed for XL ObjectAPI layer.
The new interface for any ObjectAPI layer is as below
```
// ObjectAPI interface.
type ObjectAPI interface {
// Bucket resource API.
DeleteBucket(bucket string) *probe.Error
ListBuckets() ([]BucketInfo, *probe.Error)
MakeBucket(bucket string) *probe.Error
GetBucketInfo(bucket string) (BucketInfo, *probe.Error)
// Bucket query API.
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsResult, *probe.Error)
ListMultipartUploads(bucket string, resources BucketMultipartResourcesMetadata) (BucketMultipartResourcesMetadata, *probe.Error)
// Object resource API.
GetObject(bucket, object string, startOffset int64) (io.ReadCloser, *probe.Error)
GetObjectInfo(bucket, object string) (ObjectInfo, *probe.Error)
PutObject(bucket string, object string, size int64, data io.Reader, metadata map[string]string) (ObjectInfo, *probe.Error)
DeleteObject(bucket, object string) *probe.Error
// Object query API.
NewMultipartUpload(bucket, object string) (string, *probe.Error)
PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Hex string) (string, *probe.Error)
ListObjectParts(bucket, object string, resources ObjectResourcesMetadata) (ObjectResourcesMetadata, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []CompletePart) (ObjectInfo, *probe.Error)
AbortMultipartUpload(bucket, object, uploadID string) *probe.Error
}
```
9 years ago
9dca46e156
signature: Use a layered approach for signature verification.
...
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.
In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
9 years ago
b55922effe
Fix for Istruncated set to true under certain conditions.
...
Optimizing List Objects by using binary sort to discard entries in cases
where prefix or marker is set.
Adding test coverage to ListObjects.
Adding benchmark to ListObjects.
9 years ago
5282a79eda
cleanup: Remove unecessary packages and tests. Simplify.
9 years ago
52751d81cb
cleanup: Rename ObjectMetadata as ObjectInfo.
...
Fixes #1215
9 years ago
d5057b3c51
accessPolicy: Implement Put, Get, Delete access policy.
...
This patch implements Get,Put,Delete bucket policies
Supporting - http://docs.aws.amazon.com/AmazonS3/latest/dev/access-policy-language-overview.html
Currently supports following actions.
"*": true,
"s3:*": true,
"s3:GetObject": true,
"s3:ListBucket": true,
"s3:PutObject": true,
"s3:CreateBucket": true,
"s3:GetBucketLocation": true,
"s3:DeleteBucket": true,
"s3:DeleteObject": true,
"s3:AbortMultipartUpload": true,
"s3:ListBucketMultipartUploads": true,
"s3:ListMultipartUploadParts": true,
following conditions for "StringEquals" and "StringNotEquals"
"s3:prefix", "s3:max-keys"
9 years ago
f111997184
multipart: remove proper MD5, rather create MD5 based on parts to be s3 compatible.
...
This increases the performance phenominally.
9 years ago
c7021f6a95
list: Fix handling of maxKeys and prefixes.
...
This fixes a problem of requeuing the same request
and also fixes a major problem of sending truncated
for full key prefixes.
Fixes #1177
9 years ago
a4c005ce30
multipart: Code cleanup
...
- More locking cleanup. Fix naming convention.
- Simplify concatenation and blocking calls.
9 years ago
5934a00058
listObjects: ListObjects should have idempotent behavior.
...
listObjects was returning inconsistent results, i.e missing
entries during recursive and non-recursive listing. This led
to 'mc mirror' copying contents repeatedly consisdering
these files to be missing on the destination.
This patch addresses this problem - fixes #1056
9 years ago
682020ef2f
listObjects: Channel based changes.
...
Supports:
- prefixes
- marker
9 years ago
5b2fa33bdb
Implementing min-free-disk
9 years ago
762b798767
Migrate this project to minio micro services code
9 years ago
Harshavardhana