723153951c
Test api responses ( #2427 )
...
* Test List Multipart Uploads with correct max-keys
* Test List Objects V1 V2 with valid parameters
8 years ago
70fd38818e
clean up: ineffassign fixes ( #2411 )
8 years ago
d1bb8a5b21
api: refactor the bucket policy reading and writing. ( #2395 )
...
Policies are read once during server startup and subsequently
managed through in memory map. In-memory map is updated as
and when there are new changes coming in.
8 years ago
064c51162d
api: Add new ListenBucketNotificationHandler. ( #2336 )
...
This API is precursor before implementing `minio lambda` and `mc` continous replication.
This new api is an extention to BucketNofication APIs.
// Request
```
GET /bucket?notificationARN=arn:minio:lambda:us-east-1:10:minio HTTP/1.1
...
...
```
// Response
```
{"Records": ...}
...
...
...
{"Records": ...}
```
8 years ago
2f7358a8a6
XL: erasure Index should have its corresponding distribution order. ( #2300 )
8 years ago
f248089523
api: Implement bucket notification. ( #2271 )
...
* Implement basic S3 notifications through queues
Supports multiple queues and three basic queue types:
1. NilQueue -- messages don't get sent anywhere
2. LogQueue -- messages get logged
3. AmqpQueue -- messages are sent to an AMQP queue
* api: Implement bucket notification.
Supports two different queue types
- AMQP
- ElasticSearch.
* Add support for redis
8 years ago
303f216150
tests: xl-v1-metadata.go, xl-v1-multipart-common.go - remove unused methods, add enhance tests to improve code coverage. ( #2260 )
8 years ago
aaf7803831
api: Requests should be differentiated if possible based on http router. ( #2219 )
...
In current master ListObjectsV2 was merged into ListObjectsHandler
which also implements V1 API as well.
Move the detection of ListObject types to its rightful place
in http router.
8 years ago
0793237d94
tests: Move signature calculation in separate function. ( #2160 )
...
Previously newTestRequest() creates request object and returns
signature v4 signed request. In TestCopyObject(), its required to add
headers later to the request and sign the request.
This patch introduces two new functions
* signRequest(): signs request using given access/secret keys.
* newTestSignedRequest(): returns new request object signed with given
access/secret keys.
Fixes #2097
8 years ago
bc8720406d
Added specific error for InvalidObjectName ( #2157 )
8 years ago
8c767218a4
URL Encode X-Amz-Copy-Source as per the spec ( #2114 )
...
The documents for COPY state that the X-Amz-Copy-Source must be URL encoded.
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html
8 years ago
a51bb1d728
http: handle request range like Amazon S3. ( #2112 )
...
Fixes #2098
8 years ago
8ddf52021a
server: Bring in s3 compatibility fixes. ( #2099 )
...
This patch fixes majority of discrepant messages and responses
previously reported.
There are few discrepancies observed
- S3 is not honoring 'If-Modified-Since' header.
- We do not implement object policy, S3 returns a different response in this category.
- Adding new headers causes signature mismatch, but Minio server is fine for example
TestCopyObject() to be fixed by moving the signature logic out.
Relevant bug - https://github.com/minio/minio/issues/2097
Fixes #1955
8 years ago
8a028a9efb
handler/PUT: Handle signature verification through a custom reader. ( #2066 )
...
Change brings in a new signVerifyReader which provides a io.Reader
compatible reader, additionally implements Verify() function.
Verify() function validates the signature present in the incoming
request. This approach is choosen to avoid complexities involved
in using io.Pipe().
Thanks to Krishna for his inputs on this.
Fixes #2058
Fixes #2054
Fixes #2087
8 years ago
1ec86dac2c
server-tests: unify XL and FS tests into common code. server_test.go contains common test code. server_xl_test.go contains XL tests specific to XL. ( #2089 )
8 years ago
d64c3fd464
posix: Return errDiskNotWritable during disk initialization. ( #2048 )
...
It can happen that minio server might not have
writable permissions on the export paths command line.
Fixes #2035
8 years ago
8e8f6f90a4
adding detailed comments to server_test ( #2032 )
9 years ago
ed2fdd90b0
fs: Fix GetObject failure to read large blocks. ( #1982 )
...
Add relevant test cases as well for verifying this
part of the codebase.
Fixes #1979
9 years ago
ba5bb4a127
TestServer introduction and revamp of functional tests. ( #1940 )
...
Allows for easy creation of Test server with temp backend.
changes
9 years ago
ad779a8ba4
XL: Enable tests for large GetObject. ( #1947 )
...
Ref #1946
9 years ago
8559e89494
XL: fix getBlockInfo() to return correct end block ( #1941 )
...
If requested offset/length of an object is equal to
erasureInfo.BlockSize, getBlockInfo() returns one more block added to
actual end block. This patch fixes the issue.
This patch also adds unit test for get objects with big files.
9 years ago
4ee2136b28
Unit tests for PUT object when object already exists ( #1904 )
...
* fs/xl tests for multiple put object requests
* xl fix for put object on directory
* Unit tests fix windows test issue.
9 years ago
c41bf26712
Unit tests: add unit tests for listv1/v2 for list bucket handler. ( #1933 )
...
fixes #1818
9 years ago
365f80efa3
XL/DeleteObject: delete call on a prefix should not delete the entire tree structure. ( #1916 )
...
fixes #1915
9 years ago
f51d34cedd
Do not guess content-type for objects with no extension ( #1918 )
9 years ago
ed4fe689b4
posix: Support UNC paths on windows. ( #1887 )
...
This allows us to now use 32K paths names on windows.
Fixes #1620
9 years ago
feb337098d
XL: bring in new storage API. ( #1780 )
...
Fixes #1771
9 years ago
498ce1e9bb
handler: Add a waitgroup to avoid expect100Continue crash. ( #1623 )
...
This waitgroup allows for safe blocking operation where we can cleanly
control the flow of the writes and the underlying pipe altogether.
Fixes #1553
9 years ago
5b29cefd40
api: DeleteObject should always return 204. ( #1645 )
...
Fixes #1643
9 years ago
d4745c7d6a
object: PutObjectHandler should set the md5Sum properly. ( #1604 )
...
Additionally add a test case as well for validating for us
to reply BadDigest properly.
Fixes #1603
9 years ago
04a5b25929
Multipart: Minimum part size limit does not apply to the last part during CompleteMultipartUpload. ( #1518 ) ( #1538 )
9 years ago
0625c050e6
xl/tests: Enable server handler tests over XL. ( #1512 )
...
Fixes #1513
9 years ago
82fbe908a3
object: DeleteBucket should return proper error for BucketNotEmpty. ( #1489 )
...
Fixes #1488
9 years ago
bba5468368
minio: Replace 'bucket already exists' error by 'bucket already owned by you'. ( #1465 )
...
S3 API returns BucketAlreadyExists error when some another user has such bucket.
If user that creates the bucket already has it, s3 returns BucketAlreadyOwnedByYou.
As minio has only one user, it should behave accordingly.
Otherwise it causes failures in the applications that ignore creation of already existing bucket in the account, but fail when bucket name is used by someone else.
9 years ago
ac7a7cec20
bucket-policy: Delete policy should remove policy properly. ( #1456 )
9 years ago
984903cce1
server: Add global namespace lock. ( #1398 )
...
Fixes #1393
9 years ago
8457af5708
fs: Add proper volume and path validation.
9 years ago
30b0b4deba
storage/server/client: Enable storage server, enable client storage.
9 years ago
01a439f95b
refactor: add multipart code to the object layer.
9 years ago
3c48537f20
refactor: refactor code to separate fs into object-layer and fs layer. ( #1305 )
9 years ago
33cd910d3a
backend/fs: More cleanup and start using checkBuckeArg. ( #1306 )
...
backend/fs: More cleanup and start using checkBucketArg.
9 years ago
0479d4976b
objectAPI: Fix object API interface, remove unnecessary structs.
...
ObjectAPI changes.
```
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsInfo, *probe.Error)
ListMultipartUploads(bucket, objectPrefix, keyMarker, uploadIDMarker, delimiter string, maxUploads int) (ListMultipartsInfo, *probe.Error)
ListObjectParts(bucket, object, uploadID string, partNumberMarker, maxParts int) (ListPartsInfo, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []completePart) (ObjectInfo, *probe.Error)
```
9 years ago
484ba91b08
config: Migrate to the new version. Remove backend details.
...
Migrate to new config format v4.
```
{
"version": "4",
"credential": {
"accessKey": "WLGDGYAQYIGI833EV05A",
"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
},
"region": "us-east-1",
"logger": {
"console": {
"enable": true,
"level": "fatal"
},
"file": {
"enable": false,
"fileName": "",
"level": "error"
},
"syslog": {
"enable": false,
"address": "",
"level": "debug"
}
}
}
```
This patch also updates [minio cli spec](./minio.md)
9 years ago
efc80343e3
fs: Break fs package to top-level and introduce ObjectAPI interface.
...
ObjectAPI interface brings in changes needed for XL ObjectAPI layer.
The new interface for any ObjectAPI layer is as below
```
// ObjectAPI interface.
type ObjectAPI interface {
// Bucket resource API.
DeleteBucket(bucket string) *probe.Error
ListBuckets() ([]BucketInfo, *probe.Error)
MakeBucket(bucket string) *probe.Error
GetBucketInfo(bucket string) (BucketInfo, *probe.Error)
// Bucket query API.
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsResult, *probe.Error)
ListMultipartUploads(bucket string, resources BucketMultipartResourcesMetadata) (BucketMultipartResourcesMetadata, *probe.Error)
// Object resource API.
GetObject(bucket, object string, startOffset int64) (io.ReadCloser, *probe.Error)
GetObjectInfo(bucket, object string) (ObjectInfo, *probe.Error)
PutObject(bucket string, object string, size int64, data io.Reader, metadata map[string]string) (ObjectInfo, *probe.Error)
DeleteObject(bucket, object string) *probe.Error
// Object query API.
NewMultipartUpload(bucket, object string) (string, *probe.Error)
PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Hex string) (string, *probe.Error)
ListObjectParts(bucket, object string, resources ObjectResourcesMetadata) (ObjectResourcesMetadata, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []CompletePart) (ObjectInfo, *probe.Error)
AbortMultipartUpload(bucket, object, uploadID string) *probe.Error
}
```
9 years ago
30fc970eab
Changes to TestPutBucket to catch the race
9 years ago
9dca46e156
signature: Use a layered approach for signature verification.
...
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.
In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
9 years ago
aaf97ea02c
config/main: Re-write config files - add to new config v3
...
- New config format.
```
{
"version": "3",
"address": ":9000",
"backend": {
"type": "fs",
"disk": "/path"
},
"credential": {
"accessKey": "WLGDGYAQYIGI833EV05A",
"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
},
"region": "us-east-1",
"logger": {
"file": {
"enable": false,
"fileName": "",
"level": "error"
},
"syslog": {
"enable": false,
"address": "",
"level": "debug"
},
"console": {
"enable": true,
"level": "fatal"
}
}
}
```
New command lines in lieu of supporting XL.
Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~
Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~
For 'fs' backend it starts the server.
~~~
$ minio server
~~~
For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~
Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
9 years ago
902aa05021
main: Remove all the dead/unused code.
...
This patch removes some dead and unused code.
9 years ago
fc72a0362f
api: ListMultipartUploads and ListParts responded more entries.
...
Issue is empty entries were added since allocating an array was
followed by an append. Keep the index and copy the right entries
precisely.
Fixes an issue reported at - https://github.com/minio/mc/issues/1642
9 years ago
88714e7c8e
bucketpolicy: Improve bucket policy validation, avoid nested rules.
...
Bucket policy validation is more stricter now, to avoid nested
rules. The reason to do this is keep the rules simpler and more
meaningful avoiding conflicts.
This patch implements stricter checks.
Example policy to be generally avoided.
```
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:DeleteObject"
],
"Effect": "Allow",
"Principal": {
"AWS": [
"*"
]
},
"Resource": [
"arn:aws:s3:::jarjarbing/*"
]
},
{
"Action": [
"s3:GetObject",
"s3:DeleteObject"
],
"Effect": "Deny",
"Principal": {
"AWS": [
"*"
]
},
"Resource": [
"arn:aws:s3:::jarjarbing/restic/key/*"
]
}
]
}
```
9 years ago
Anis Elleuch