Krishnan Parthasarathi
967c2b2940
Handled possible short writes to httpResponseWriter ( #1804 )
...
* XL: Handled possible short writes to httpResponseWriter
* Added tests for Range Header combinations
8 years ago
Harshavardhana
5a4b074ca0
XL: PutObject incorrectly returned after deleting multipart object. ( #1715 )
...
Fixes #1714
8 years ago
Harshavardhana
9fdb69563d
handler: CopyObject should save metadata. ( #1698 )
...
- Content-Type
- Content-Encoding
- ETag
Fixes #1682
8 years ago
Harshavardhana
00d0558131
XL: Enable tests for content-type for Head and Get
...
Fixes #1674
8 years ago
Harshavardhana
498ce1e9bb
handler: Add a waitgroup to avoid expect100Continue crash. ( #1623 )
...
This waitgroup allows for safe blocking operation where we can cleanly
control the flow of the writes and the underlying pipe altogether.
Fixes #1553
8 years ago
Bala FA
adbcafefad
xl/CreateFile: handle errFileNameTooLong error properly ( #1523 )
...
When errFileNameTooLong error is returned from posix, xl.CreateFile()
treats the error specially by returning the same error immediately.
Fixes #1501
8 years ago
Krishna Srinivas
04a5b25929
Multipart: Minimum part size limit does not apply to the last part during CompleteMultipartUpload. ( #1518 ) ( #1538 )
8 years ago
Krishna Srinivas
75320f70d0
multipart: reject part upload if size is less than 5MB. ( #1518 )
8 years ago
Harshavardhana
0625c050e6
xl/tests: Enable server handler tests over XL. ( #1512 )
...
Fixes #1513
8 years ago
Harshavardhana
82fbe908a3
object: DeleteBucket should return proper error for BucketNotEmpty. ( #1489 )
...
Fixes #1488
8 years ago
Yurii
bba5468368
minio: Replace 'bucket already exists' error by 'bucket already owned by you'. ( #1465 )
...
S3 API returns BucketAlreadyExists error when some another user has such bucket.
If user that creates the bucket already has it, s3 returns BucketAlreadyOwnedByYou.
As minio has only one user, it should behave accordingly.
Otherwise it causes failures in the applications that ignore creation of already existing bucket in the account, but fail when bucket name is used by someone else.
8 years ago
Harshavardhana
ac7a7cec20
bucket-policy: Delete policy should remove policy properly. ( #1456 )
8 years ago
Harshavardhana
984903cce1
server: Add global namespace lock. ( #1398 )
...
Fixes #1393
8 years ago
Harshavardhana
8457af5708
fs: Add proper volume and path validation.
9 years ago
Harshavardhana
30b0b4deba
storage/server/client: Enable storage server, enable client storage.
9 years ago
Krishna Srinivas
01a439f95b
refactor: add multipart code to the object layer.
9 years ago
Krishna Srinivas
3c48537f20
refactor: refactor code to separate fs into object-layer and fs layer. ( #1305 )
9 years ago
Harshavardhana
33cd910d3a
backend/fs: More cleanup and start using checkBuckeArg. ( #1306 )
...
backend/fs: More cleanup and start using checkBucketArg.
9 years ago
Harshavardhana
0479d4976b
objectAPI: Fix object API interface, remove unnecessary structs.
...
ObjectAPI changes.
```
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsInfo, *probe.Error)
ListMultipartUploads(bucket, objectPrefix, keyMarker, uploadIDMarker, delimiter string, maxUploads int) (ListMultipartsInfo, *probe.Error)
ListObjectParts(bucket, object, uploadID string, partNumberMarker, maxParts int) (ListPartsInfo, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []completePart) (ObjectInfo, *probe.Error)
```
9 years ago
Harshavardhana
484ba91b08
config: Migrate to the new version. Remove backend details.
...
Migrate to new config format v4.
```
{
"version": "4",
"credential": {
"accessKey": "WLGDGYAQYIGI833EV05A",
"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
},
"region": "us-east-1",
"logger": {
"console": {
"enable": true,
"level": "fatal"
},
"file": {
"enable": false,
"fileName": "",
"level": "error"
},
"syslog": {
"enable": false,
"address": "",
"level": "debug"
}
}
}
```
This patch also updates [minio cli spec](./minio.md)
9 years ago
Harshavardhana
efc80343e3
fs: Break fs package to top-level and introduce ObjectAPI interface.
...
ObjectAPI interface brings in changes needed for XL ObjectAPI layer.
The new interface for any ObjectAPI layer is as below
```
// ObjectAPI interface.
type ObjectAPI interface {
// Bucket resource API.
DeleteBucket(bucket string) *probe.Error
ListBuckets() ([]BucketInfo, *probe.Error)
MakeBucket(bucket string) *probe.Error
GetBucketInfo(bucket string) (BucketInfo, *probe.Error)
// Bucket query API.
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsResult, *probe.Error)
ListMultipartUploads(bucket string, resources BucketMultipartResourcesMetadata) (BucketMultipartResourcesMetadata, *probe.Error)
// Object resource API.
GetObject(bucket, object string, startOffset int64) (io.ReadCloser, *probe.Error)
GetObjectInfo(bucket, object string) (ObjectInfo, *probe.Error)
PutObject(bucket string, object string, size int64, data io.Reader, metadata map[string]string) (ObjectInfo, *probe.Error)
DeleteObject(bucket, object string) *probe.Error
// Object query API.
NewMultipartUpload(bucket, object string) (string, *probe.Error)
PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Hex string) (string, *probe.Error)
ListObjectParts(bucket, object string, resources ObjectResourcesMetadata) (ObjectResourcesMetadata, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []CompletePart) (ObjectInfo, *probe.Error)
AbortMultipartUpload(bucket, object, uploadID string) *probe.Error
}
```
9 years ago
Karthic Rao
30fc970eab
Changes to TestPutBucket to catch the race
9 years ago
Harshavardhana
9dca46e156
signature: Use a layered approach for signature verification.
...
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.
In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
9 years ago
Harshavardhana
aaf97ea02c
config/main: Re-write config files - add to new config v3
...
- New config format.
```
{
"version": "3",
"address": ":9000",
"backend": {
"type": "fs",
"disk": "/path"
},
"credential": {
"accessKey": "WLGDGYAQYIGI833EV05A",
"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
},
"region": "us-east-1",
"logger": {
"file": {
"enable": false,
"fileName": "",
"level": "error"
},
"syslog": {
"enable": false,
"address": "",
"level": "debug"
},
"console": {
"enable": true,
"level": "fatal"
}
}
}
```
New command lines in lieu of supporting XL.
Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~
Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~
For 'fs' backend it starts the server.
~~~
$ minio server
~~~
For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~
Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
9 years ago
Harshavardhana
902aa05021
main: Remove all the dead/unused code.
...
This patch removes some dead and unused code.
9 years ago
Harshavardhana
fc72a0362f
api: ListMultipartUploads and ListParts responded more entries.
...
Issue is empty entries were added since allocating an array was
followed by an append. Keep the index and copy the right entries
precisely.
Fixes an issue reported at - https://github.com/minio/mc/issues/1642
9 years ago
Harshavardhana
88714e7c8e
bucketpolicy: Improve bucket policy validation, avoid nested rules.
...
Bucket policy validation is more stricter now, to avoid nested
rules. The reason to do this is keep the rules simpler and more
meaningful avoiding conflicts.
This patch implements stricter checks.
Example policy to be generally avoided.
```
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:DeleteObject"
],
"Effect": "Allow",
"Principal": {
"AWS": [
"*"
]
},
"Resource": [
"arn:aws:s3:::jarjarbing/*"
]
},
{
"Action": [
"s3:GetObject",
"s3:DeleteObject"
],
"Effect": "Deny",
"Principal": {
"AWS": [
"*"
]
},
"Resource": [
"arn:aws:s3:::jarjarbing/restic/key/*"
]
}
]
}
```
9 years ago
Harshavardhana
d5057b3c51
accessPolicy: Implement Put, Get, Delete access policy.
...
This patch implements Get,Put,Delete bucket policies
Supporting - http://docs.aws.amazon.com/AmazonS3/latest/dev/access-policy-language-overview.html
Currently supports following actions.
"*": true,
"s3:*": true,
"s3:GetObject": true,
"s3:ListBucket": true,
"s3:PutObject": true,
"s3:CreateBucket": true,
"s3:GetBucketLocation": true,
"s3:DeleteBucket": true,
"s3:DeleteObject": true,
"s3:AbortMultipartUpload": true,
"s3:ListBucketMultipartUploads": true,
"s3:ListMultipartUploadParts": true,
following conditions for "StringEquals" and "StringNotEquals"
"s3:prefix", "s3:max-keys"
9 years ago
Harshavardhana
aed62788d9
api: Implement multiple objects Delete api - fixes #956
...
This API takes input XML input in following form.
```
<?xml version="1.0" encoding="UTF-8"?>
<Delete>
<Quiet>true</Quiet>
<Object>
<Key>Key</Key>
</Object>
<Object>
<Key>Key</Key>
</Object>
...
</Delete>
```
and responds the list of successful deletes, list of errors
for all the deleted objects.
```
<?xml version="1.0" encoding="UTF-8"?>
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/ ">
<Deleted>
<Key>sample1.txt</Key>
</Deleted>
<Error>
<Key>sample2.txt</Key>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
</Error>
</DeleteResult>
```
9 years ago
Harshavardhana
ee1b86e517
api: Implement support for additional request headers.
...
Now GetObject and HeadObject both support
- If-Modified-Since, If-Unmodified-Since
- If-Match, If-None-Match
request headers.
These headers are used to further handle the responses for GetObject
and HeadObject API.
Fixes #1098
9 years ago
Harshavardhana
3ff8a1b719
api: Implement CopyObject s3 API, doing server side copy.
...
Fixes #1172
9 years ago
Harshavardhana
ae6e774377
api: Return NotImplemented for MultiDelete and CopyObject APIs
9 years ago
Harshavardhana
91a092792a
presigned: Fix a bug in presigned request verification.
...
Additionally add Docker proxy configuration.
9 years ago
Harshavardhana
dd9aaa855c
web/rpc: Merge ports with API server.
...
Fixes #1081 and #1130
9 years ago
Harshavardhana
5a9333a67b
signature: Rewrite signature handling and move it into a library.
9 years ago
Harshavardhana
8df201ef30
multipart: Multipart session map now is based on uploadID.
...
- Fixes initiating parallel uploads, and configs being quickly
re-written by another incoming request.
- Parallel uploads work smoothly now and return expected behavior.
9 years ago
Harshavardhana
7a3409c309
fs: Cleanup Golang errors to be called 'e' and probe to be called as 'err'
...
- Replace the ACL checks back, remove them when bucket
policy is implemented.
- Move FTW (File Tree Walk) into ioutils package.
9 years ago
Harshavardhana
df91661ec6
flags: Remove anonymous, ratelimit, json and web-address flags.
...
- Web address now uses the port + 1 from the API address port directly.
- Remove ratelimiting, ratelimiting will be achieved if necessary through
iptables.
- Remove json flag, not needed anymore.
- Remove anonymous flag, server will be no more anonymous for play.minio.io
we will use demo credentials.
9 years ago
Harshavardhana
2e311168ee
multipart: NewMultipartUpload shouldn't return empty UploadID
...
Existing code
```
{
if os.IsNotExist(e) {
e = os.MkdirAll(objectDir, 0700)
if e != nil {
return "", probe.NewError(e)
}
}
return "", probe.NewError(e) ---> Error was here.
}
```
For a successful 'MkdirAll' it would still return an empty uploadID,
but the 'error' would be nil. This would succeed the request but
client would fail.
Fix is to re-arrange the logic. Thanks to Alexander Neumann @fd0 , for
reporting this problem.
9 years ago
Harshavardhana
682020ef2f
listObjects: Channel based changes.
...
Supports:
- prefixes
- marker
9 years ago
Harshavardhana
836f5204af
minio: Add config-folder option.
...
Fixes #997
9 years ago
Harshavardhana
7c91a8495f
signature/region: Remove 'milkyway' and use 'us-east-1' as default.
...
Fixes #980
9 years ago
Kanai Masumi
84de2e33c4
Fix: permit trailing slash for compatible with S3.
...
ex.
s3cmd requests to path:`/<bucket>/` for PutBucket.
9 years ago
Harshavardhana
ab15f56a61
fs/bucket: Move bucket metadata into buckets.json
9 years ago
Harshavardhana
56003fded7
Add logger command - also migrate from old config to newer config
9 years ago
Harshavardhana
5b2fa33bdb
Implementing min-free-disk
9 years ago
Harshavardhana
1256ca86d0
Add fs separator
9 years ago
Harshavardhana
704fa420a3
Reply back proper statuses for DeleteBucket/DeleteObject
9 years ago
Harshavardhana
762b798767
Migrate this project to minio micro services code
9 years ago
Harshavardhana
7fea9cb550
Add rpc signature handler
9 years ago