3a539ce660
Implement gateway S3 support ( #3940 )
7 years ago
05e53f1b34
api: CopyObjectPart was copying wrong offsets due to shadowing. ( #3838 )
...
startOffset was re-assigned to '0' so it would end up
copying wrong content ignoring the requested startOffset.
This also fixes the corruption issue we observed while
using docker registry.
Fixes https://github.com/docker/distribution/issues/2205
Also fixes #3842 - incorrect routing.
8 years ago
1c699d8d3f
fs: Re-implement object layer to remember the fd ( #3509 )
...
This patch re-writes FS backend to support shared backend sharing locks for safe concurrent access across multiple servers.
8 years ago
ff4ce0ee14
fs/xl: Combine input checks into re-usable functions. ( #3383 )
...
Repeated code around both object layers are moved
and combined into simple re-usable functions.
8 years ago
12c1abed98
Vendorize with bug fixes from minio browser. ( #3341 )
...
This patch brings in changes from miniobrowser repo.
- Bucket policy UI and functionality fixes by @krishnasrinivas
- Bucket policy implementation by @balamurugana
- UI changes and new functionality changing password etc. @rushenn
- UI and new functionality for sharing URLs, deleting files
@rushenn and @krishnasrinivas .
- Other misc fixes by @vadmeste @brendanashworth
8 years ago
1c47365445
xl/bootup: Upon bootup handle errors loading bucket and event configs. ( #3287 )
...
In a situation when we have lots of buckets the bootup time
might have slowed down a bit but during this situation the
servers quickly going up and down would be an in-transit state.
Certain calls which do not use quorum like `readXLMetaStat`
might return an error saying `errDiskNotFound` this is returned
in place of expected `errFileNotFound` which leads to an issue
where server doesn't start.
To avoid this situation we need to ignore them as safe values
to be ignored, for the most part these are network related errors.
Fixes #3275
8 years ago
c91d3791f9
heal: Add healing support for bucket, bucket metadata files. ( #3252 )
...
This patch implements healing in general but it is only used
as part of quickHeal().
Fixes #3237
8 years ago
35e541e0b1
content-length-range policy should be honored for the uploaded object sizes. ( #3076 )
8 years ago
5999a23d3e
When object whose size is greater than 5G is uploaded using presigned POST we should return error. ( #3033 )
...
fixes #2961
8 years ago
41f9ab1c69
Translate storage access denied error to S3 Access Denied response ( #3015 )
8 years ago
61a18ed48f
sha256: Verify sha256 along with md5sum, signature is verified on the request early. ( #2813 )
8 years ago
9358ee011b
logging: Print stack trace in case of errors.
...
fixes #1827
8 years ago
bccf549463
server: Move all the top level files into cmd folder. ( #2490 )
...
This change brings a change which was done for the 'mc'
package to allow for clean repo and have a cleaner
github drop in experience.
8 years ago
e2498edb45
contoller: Implement controlled healing and trigger ( #2381 )
...
This patch introduces new command line 'control'
- minio control
TO manage minio server connecting through GoRPC API frontend.
- minio control heal
Is implemented for healing objects.
8 years ago
778b870b77
placing the http range error in objct-api-errors. ( #2150 )
8 years ago
4cfbdb1bf0
server: Remove deadcode/deprecated code. ( #2088 )
8 years ago
748dc80047
API: add writePartTooSmallErrorResponse to extend standard error responses. ( #2005 )
...
This function is added to extend the standard error responses.
Which is needed in some cases for example CompleteMultipartUpload
should respond with ErrPartTooSmall error when parts uploaded are
lesser than 5MB (i.e minimum allowed size per part).
Fixes #1536
8 years ago
fb95c1fad3
XL: Bring in some modularity into format verification and healing. ( #1832 )
8 years ago
adbcafefad
xl/CreateFile: handle errFileNameTooLong error properly ( #1523 )
...
When errFileNameTooLong error is returned from posix, xl.CreateFile()
treats the error specially by returning the same error immediately.
Fixes #1501
8 years ago
76c511c9fe
api: Extend S3 errors with Minio errors. ( #1533 )
...
Fixes #1530
8 years ago
75320f70d0
multipart: reject part upload if size is less than 5MB. ( #1518 )
8 years ago
937d68202d
server: Deadcode removal. ( #1517 )
8 years ago
17868ccd7f
handlers: overhaul entire writErrorResponse, simplify. ( #1472 )
8 years ago
82fbe908a3
object: DeleteBucket should return proper error for BucketNotEmpty. ( #1489 )
...
Fixes #1488
8 years ago
dd417e5476
fs: Handle cases of PutObject for an existing prefix. ( #1478 )
8 years ago
1813e9c070
Cleanup - Comments and readability fixes ( #1386 )
8 years ago
cab6805f09
xl: Enable a subset of tests for XL branch. ( #1359 )
8 years ago
91588209fa
obj: Object api handle all errors in common location. ( #1343 )
8 years ago
6e372f83b4
Tests: object api multipart tests and bug fixes.
8 years ago
e0f8fed011
object: handle Error responses and handle errDiskFull. ( #1331 )
9 years ago
01a439f95b
refactor: add multipart code to the object layer.
9 years ago
3c48537f20
refactor: refactor code to separate fs into object-layer and fs layer. ( #1305 )
9 years ago
efc80343e3
fs: Break fs package to top-level and introduce ObjectAPI interface.
...
ObjectAPI interface brings in changes needed for XL ObjectAPI layer.
The new interface for any ObjectAPI layer is as below
```
// ObjectAPI interface.
type ObjectAPI interface {
// Bucket resource API.
DeleteBucket(bucket string) *probe.Error
ListBuckets() ([]BucketInfo, *probe.Error)
MakeBucket(bucket string) *probe.Error
GetBucketInfo(bucket string) (BucketInfo, *probe.Error)
// Bucket query API.
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsResult, *probe.Error)
ListMultipartUploads(bucket string, resources BucketMultipartResourcesMetadata) (BucketMultipartResourcesMetadata, *probe.Error)
// Object resource API.
GetObject(bucket, object string, startOffset int64) (io.ReadCloser, *probe.Error)
GetObjectInfo(bucket, object string) (ObjectInfo, *probe.Error)
PutObject(bucket string, object string, size int64, data io.Reader, metadata map[string]string) (ObjectInfo, *probe.Error)
DeleteObject(bucket, object string) *probe.Error
// Object query API.
NewMultipartUpload(bucket, object string) (string, *probe.Error)
PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Hex string) (string, *probe.Error)
ListObjectParts(bucket, object string, resources ObjectResourcesMetadata) (ObjectResourcesMetadata, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []CompletePart) (ObjectInfo, *probe.Error)
AbortMultipartUpload(bucket, object, uploadID string) *probe.Error
}
```
9 years ago
9dca46e156
signature: Use a layered approach for signature verification.
...
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.
In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
9 years ago
d5057b3c51
accessPolicy: Implement Put, Get, Delete access policy.
...
This patch implements Get,Put,Delete bucket policies
Supporting - http://docs.aws.amazon.com/AmazonS3/latest/dev/access-policy-language-overview.html
Currently supports following actions.
"*": true,
"s3:*": true,
"s3:GetObject": true,
"s3:ListBucket": true,
"s3:PutObject": true,
"s3:CreateBucket": true,
"s3:GetBucketLocation": true,
"s3:DeleteBucket": true,
"s3:DeleteObject": true,
"s3:AbortMultipartUpload": true,
"s3:ListBucketMultipartUploads": true,
"s3:ListMultipartUploadParts": true,
following conditions for "StringEquals" and "StringNotEquals"
"s3:prefix", "s3:max-keys"
9 years ago
800b19d8e5
cleanup: Remove definitions and move them to its relative places accordingly
...
- Move fs-definitions.go and break them into fs-datatypes.go, fs-bucket-acl.go
and fs-utils.go
- Move api-definitions.go to api-response.go, where they should be.
- Move web-definitions to its related handlers.
9 years ago
5a9333a67b
signature: Rewrite signature handling and move it into a library.
9 years ago
62f6ffb6db
xl: Moved to minio/minio - fixes #1112
9 years ago
a4c005ce30
multipart: Code cleanup
...
- More locking cleanup. Fix naming convention.
- Simplify concatenation and blocking calls.
9 years ago
835b297ba7
fs: Fail createObject with appropriate message.
...
Fail createObject() if a file already exists and one attempts
to create a prefix/directory by same name.
Send an approriate error back to the client as 409 Conflict.
9 years ago
5b2fa33bdb
Implementing min-free-disk
9 years ago
762b798767
Migrate this project to minio micro services code
9 years ago
d5ce2f6944
Make erasure matrix type not optional choose automatically
...
Remove option of providing Technique and handling errors based on that
choose a matrix type automatically based on number of data blocks.
INTEL recommends on using cauchy for consistent invertible matrices,
while vandermonde is faster we should default to cauchy for large
data blocks.
9 years ago
c8de5bad2f
Implement presigned policy
9 years ago
81cc017f91
Implement presigned signature v4 support
9 years ago
63c9cf0c4b
Move from Minimalist Object Storage to Minio Cloud Storage
9 years ago
7ae60a6d10
For missing parts reply back as InvalidPart{}
9 years ago
00890c254e
CompleteMultipartUpload and CreateObjectPart now fully support signature v4
9 years ago
375860077d
HTTP header Content-Length signifies body length of the request, if its smaller reply appropriately
...
This patch also handles large individual part sizes > 5MB by using less memory copies.
9 years ago
ec33d79d57
Add server side signaturev4 check, not wired up to the readers yet.
9 years ago
Remco Verhoef