79ba4d3f33
refactor ObjectLayer PutObject and PutObjectPart ( #4925 )
...
This change refactor the ObjectLayer PutObject and PutObjectPart
functions. Instead of passing an io.Reader and a size to PUT operations
ObejectLayer expects an HashReader.
A HashReader verifies the MD5 sum (and SHA256 sum if required) of the object.
This change updates all all PutObject(Part) calls and removes unnecessary code
in all ObjectLayer implementations.
Fixes #4923
7 years ago
155a90403a
fs/erasure: Rename meta 'md5Sum' as 'etag'. ( #4319 )
...
This PR also does backend format change to 1.0.1
from 1.0.0. Backward compatible changes are still
kept to read the 'md5Sum' key. But all new objects
will be stored with the same details under 'etag'.
Fixes #4312
8 years ago
298b470f69
fs/erasure: Ignore objects with / even for DeleteObject() ( #4303 )
...
Additionally GetObject() also returns errFileNotFound similar
to HeadObject().
Fixes #4302
8 years ago
48aa2ac392
server: Validate path for bad components in a handler. ( #4170 )
8 years ago
6df7bc42b8
Fix check for bucket name: ( #3832 )
...
* Do not allow bucket names with adjacent hypen and periods.
* Improve performance by eliminating the usage of regular expressions.
8 years ago
e5d4e7aa9d
web: Validate if bucket names are reserved ( #3841 )
...
Both '.minio.sys' and 'minio' should be never allowed
to be created from web-ui and then fail to list it
by filtering them out.
Fixes #3840
8 years ago
7ea1de8245
copyObject: Be case sensitive for windows only server. ( #3766 )
...
For case sensitive platforms we should honor case.
Fixes #3765
```
1) python s3cmd -c s3cfg_localminio put logo.png s3://testbucket/xyz/etc2/logo.PNG
2) python s3cmd -c s3cfg_localminio ls s3://testbucket/xyz/etc2/
2017-02-18 10:58 22059 s3://testbucket/xyz/etc2/logo.PNG
3) python s3cmd -c s3cfg_localminio cp s3://testbucket/xyz/etc2/logo.PNG s3://testbucket/xyz/etc2/logo.png
remote copy: 's3://testbucket/xyz/etc2/logo.PNG' -> 's3://testbucket/xyz/etc2/logo.png'
4) python s3cmd -c s3cfg_localminio ls s3://testbucket/xyz/etc2/
2017-02-18 10:58 22059 s3://testbucket/xyz/etc2/logo.PNG
2017-02-18 11:10 22059 s3://testbucket/xyz/etc2/logo.png
```
8 years ago
50b4e54a75
fs: Do not return reservedBucket names in ListBuckets() ( #3754 )
...
Make sure to skip reserved bucket names in `ListBuckets()`
current code didn't skip this properly and also generalize
this behavior for both XL and FS.
8 years ago
533338bdeb
all/windows: Be case in-sensitive about pattern matching. ( #3682 )
...
Resource strings and paths are case insensitive on windows
deployments but if user happens to use upper case instead of
lower case for certain configuration params like bucket
policies and bucket notification config. We might not honor
them which leads to a wrong behavior on windows.
This is windows only behavior, for all other platforms case
is still kept sensitive.
8 years ago
c194b9f5f1
Implement mgmt REST APIs for heal subcommands ( #3533 )
...
The heal APIs supported in this change are,
- listing of objects to be healed.
- healing a bucket.
- healing an object.
8 years ago
c8f57133a4
Implement list, clear locks REST API w/ pkg/madmin support ( #3491 )
...
* Filter lock info based on bucket, prefix and time since lock was held
* Implement list and clear locks REST API
* madmin: Add list and clear locks API
* locks: Clear locks matching bucket, prefix, relTime.
* Gather lock information across nodes for both list and clear locks admin REST API.
* docs: Add lock API to management APIs
8 years ago
ff4ce0ee14
fs/xl: Combine input checks into re-usable functions. ( #3383 )
...
Repeated code around both object layers are moved
and combined into simple re-usable functions.
8 years ago
1d4ac4b084
Rename getUUID() into mustGetUUID() ( #3320 )
...
In case of UUID generation failure mustGetUUID() will panic than
infinitely trying in for loop.
8 years ago
339c9019b9
Protect multipart directory from removing when it is empty ( #3315 )
8 years ago
bef0a50bc1
Cleanup and fixes ( #3273 )
...
* newRequestID() (previously generateUploadID()) returns string than byte array.
* Remove unclear comments and added appropriate comments.
* SHA-256, MD5 Hash functions return Hex/Base64 encoded string than byte array.
* Remove duplicate MD5 hasher functions.
* Rename listObjectsValidateArgs() into validateListObjectsArgs()
* Remove repeated auth check code in all bucket request handlers.
* Remove abbreviated names in bucket-metadata
* Avoid nested if in bucketPolicyMatchStatement()
* Use ioutil.ReadFile() instead of os.Open() and ioutil.ReadAll()
* Set crossDomainXML as constant.
8 years ago
aa98702908
api: Handle content-length-range policy properly. ( #3297 )
...
content-length-range policy in postPolicy API was
not working properly handle it. The reflection
strategy used has changed in recent version of Go.
Any free form interface{} of any integer is treated
as `float64` this caused a bug where content-length-range
parsing failed to provide any value.
Fixes #3295
8 years ago
ffbee70e04
Avoid removing 'tmp' directory inside '.minio.sys' ( #3294 )
8 years ago
51d1e6f75b
Fix missing CompleteMultipartUpload Etag. ( #3227 )
...
Fixes #3224
8 years ago
35e541e0b1
content-length-range policy should be honored for the uploaded object sizes. ( #3076 )
8 years ago
36639b65a9
rename completeMultipartMD5() into getCompleteMultipartMD5(). ( #3051 )
8 years ago
5999a23d3e
When object whose size is greater than 5G is uploaded using presigned POST we should return error. ( #3033 )
...
fixes #2961
8 years ago
5ecba587f7
api: Relax object name validation. ( #2814 )
...
Fixes #2812
8 years ago
9358ee011b
logging: Print stack trace in case of errors.
...
fixes #1827
8 years ago
bccf549463
server: Move all the top level files into cmd folder. ( #2490 )
...
This change brings a change which was done for the 'mc'
package to allow for clean repo and have a cleaner
github drop in experience.
8 years ago
73d1a46f3e
Allow asterisk character in key names ( #2487 )
8 years ago
f248089523
api: Implement bucket notification. ( #2271 )
...
* Implement basic S3 notifications through queues
Supports multiple queues and three basic queue types:
1. NilQueue -- messages don't get sent anywhere
2. LogQueue -- messages get logged
3. AmqpQueue -- messages are sent to an AMQP queue
* api: Implement bucket notification.
Supports two different queue types
- AMQP
- ElasticSearch.
* Add support for redis
8 years ago
2d38046a5a
utils: BucketNames with double periods and ip address should be rejected. ( #2213 )
...
Fixes #2212
8 years ago
0fcfb5df3b
XL/fs: Change minioMetaBucket different than '.minio' config dir. ( #2190 )
...
This fixes corruption of config directory seen when minio server
exports 'home' directory.
```
minio server ~
```
8 years ago
feb337098d
XL: bring in new storage API. ( #1780 )
...
Fixes #1771
9 years ago
51bb613fdf
pkg/safe: remove temporary file on failure ( #1774 )
9 years ago
293d246f95
XL/FS: Rewrite in new format.
9 years ago
d267696110
Validation: Reject object names with trailing "/". ( #1619 )
...
Fixes #1616
9 years ago
722abe2d0f
xl/fs: pathJoin now takes variadic inputs. ( #1550 )
...
Retains slash for the last element.
Fixes #1546
9 years ago
90ea494338
erasure: waitCloser should implement CloseWithError. ( #1543 )
...
This is needed so that the other end of the pipe receives
and error, cleanups temporary files.
9 years ago
a205aca6d2
init: Cleanup .minio/tmp directories recursively. Also takes care of cleaning up of parts directory during abortMultipartUpload. ( #1532 )
9 years ago
3f51dd4fd4
xl: CompleteMultipartUpload should rename files in a routine. ( #1527 )
...
This solves the client timeout while renaming 9000+ parts.
Fixes #1526
9 years ago
56b7df90e1
xl/fs: ListObjectParts should set nextPartNumberMarker properly. ( #1528 )
...
For list requests on parts more than 1000, would lead to an infinite
loop.
Fixes #1522
9 years ago
d0e854afb7
xl/fs: Bring in ".minio/tmp" directory support. ( #1464 )
...
All transactions happen through this directory inside ".minio/temp".
Adding this allows us to remove any temporary files which were not
committed before.
Fixes #1462
Fixes #1444
9 years ago
7caa82f32f
xl/fs: Rename minioMetaVolume to minioMetaBucket. ( #1442 )
9 years ago
27c50a70cc
obj: support object names with curly braces. ( #1429 )
...
Example files like
```
/usr/share/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/ubufox@ubuntu.com.xpi
Song's Son.ogg
```
Should be supported.
9 years ago
4e34e03dd4
xl/fs: Split object layer into interface. ( #1415 )
9 years ago
8deddb82f4
Cleanup: Moving IsValidLocationContraint to handler utils
9 years ago
eed756777b
object: Allow '[' and ']' as part of object names.
9 years ago
1284ecc6f2
api: Fix verification of checkLeafDirectory. ( #1347 )
...
This fixes a problem where leaf directory has more than 1000
entries, also resulting in listing issues, leading to an infinite
loop.
Fixes #1334
9 years ago
cb1116725b
api: verify Location constraint for make bucket. ( #1342 )
9 years ago
be002ac01e
fs/object: Fix issues from review comments.
9 years ago
149c6ca094
listMultipart: bugfixes. ( #1318 )
9 years ago
8457af5708
fs: Add proper volume and path validation.
9 years ago
3c48537f20
refactor: refactor code to separate fs into object-layer and fs layer. ( #1305 )
9 years ago
efc80343e3
fs: Break fs package to top-level and introduce ObjectAPI interface.
...
ObjectAPI interface brings in changes needed for XL ObjectAPI layer.
The new interface for any ObjectAPI layer is as below
```
// ObjectAPI interface.
type ObjectAPI interface {
// Bucket resource API.
DeleteBucket(bucket string) *probe.Error
ListBuckets() ([]BucketInfo, *probe.Error)
MakeBucket(bucket string) *probe.Error
GetBucketInfo(bucket string) (BucketInfo, *probe.Error)
// Bucket query API.
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsResult, *probe.Error)
ListMultipartUploads(bucket string, resources BucketMultipartResourcesMetadata) (BucketMultipartResourcesMetadata, *probe.Error)
// Object resource API.
GetObject(bucket, object string, startOffset int64) (io.ReadCloser, *probe.Error)
GetObjectInfo(bucket, object string) (ObjectInfo, *probe.Error)
PutObject(bucket string, object string, size int64, data io.Reader, metadata map[string]string) (ObjectInfo, *probe.Error)
DeleteObject(bucket, object string) *probe.Error
// Object query API.
NewMultipartUpload(bucket, object string) (string, *probe.Error)
PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Hex string) (string, *probe.Error)
ListObjectParts(bucket, object string, resources ObjectResourcesMetadata) (ObjectResourcesMetadata, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []CompletePart) (ObjectInfo, *probe.Error)
AbortMultipartUpload(bucket, object, uploadID string) *probe.Error
}
```
9 years ago
Andreas Auernhammer