083e4e9479
backend/fs: Refactor multipart upload
...
This patch modifies multipart upload related functions as below
* New multipart upload call creates file
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.uploadid
* Put object part call creates file
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.PART_NUMBER.MD5SUM_STRING
* Abort multipart call removes all files matching
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.*
* Complete multipart call does
1. creates a staging file
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete.TEMP_NAME
then renames to
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete
2. rename staging file
EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete
to EXPORT_DIR/BUCKET/PATH/TO/OBJECT
9 years ago
c69fdf0cf2
listObjects: Cleanup and naming conventions.
...
- Marker should be escaped outside in handlers.
- Delimiter should be handled outside in handlers.
- Add missing comments and change the function names.
- Handle case of 'maxKeys' when its set to '0', its a valid
case and should be treated as such.
9 years ago
85ab1df5a8
listObjects: do not do stat during readdir()
...
* listObjects: improve response time by not doing stat during readDir() operation.
* listObjects: Add windows support.
* listObjects: Readdir() in batches to conserve memory. Add solaris build.
* listObjects: cleanup code.
9 years ago
7623e0f8e8
docker: Fix bug in start.sh arguments.
9 years ago
9843aa1f7a
Merge pull request #1284 from harshavardhana/list
...
objectAPI: Fix object API interface, remove unnecessary structs.
9 years ago
0479d4976b
objectAPI: Fix object API interface, remove unnecessary structs.
...
ObjectAPI changes.
```
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsInfo, *probe.Error)
ListMultipartUploads(bucket, objectPrefix, keyMarker, uploadIDMarker, delimiter string, maxUploads int) (ListMultipartsInfo, *probe.Error)
ListObjectParts(bucket, object, uploadID string, partNumberMarker, maxParts int) (ListPartsInfo, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []completePart) (ObjectInfo, *probe.Error)
```
9 years ago
12515eabe2
Merge pull request #1280 from harshavardhana/region
...
signature: No need to validate region for getBucketLocation and listBuckets
9 years ago
a6a4e7e297
signature: No need to validate region for getBucketLocation and listBuckets.
...
This type of check is added for making sure that we can support
custom regions.
ListBuckets and GetBucketLocation are always "us-east-1" rest
should look for the configured region.
Fixes #1278
9 years ago
2c793a2ea7
Merge pull request #1282 from harshavardhana/remove-old-code
...
cleanup: Remove old donut/xl code and erasure implementation.
9 years ago
2bb262cc56
Merge pull request #1279 from harshavardhana/backend
...
config: Migrate to the new version. Remove backend details.
9 years ago
379e0abf03
cleanup: Remove old donut/xl code and erasure implementation.
...
This is a change to bring in 'klauspost/reedsolomon' library
in #1270 patch.
9 years ago
484ba91b08
config: Migrate to the new version. Remove backend details.
...
Migrate to new config format v4.
```
{
"version": "4",
"credential": {
"accessKey": "WLGDGYAQYIGI833EV05A",
"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
},
"region": "us-east-1",
"logger": {
"console": {
"enable": true,
"level": "fatal"
},
"file": {
"enable": false,
"fileName": "",
"level": "error"
},
"syslog": {
"enable": false,
"address": "",
"level": "debug"
}
}
}
```
This patch also updates [minio cli spec](./minio.md)
9 years ago
6037fe66e9
minio: Simplify for `gosimple` tool complaints.
9 years ago
ba3a5805c1
vendorize: Add updated ui-assets.go.
9 years ago
33830bfcae
Merge pull request #1273 from harshavardhana/fs-linux
...
fs: Break fs package to top-level and introduce ObjectAPI interface.
9 years ago
efc80343e3
fs: Break fs package to top-level and introduce ObjectAPI interface.
...
ObjectAPI interface brings in changes needed for XL ObjectAPI layer.
The new interface for any ObjectAPI layer is as below
```
// ObjectAPI interface.
type ObjectAPI interface {
// Bucket resource API.
DeleteBucket(bucket string) *probe.Error
ListBuckets() ([]BucketInfo, *probe.Error)
MakeBucket(bucket string) *probe.Error
GetBucketInfo(bucket string) (BucketInfo, *probe.Error)
// Bucket query API.
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsResult, *probe.Error)
ListMultipartUploads(bucket string, resources BucketMultipartResourcesMetadata) (BucketMultipartResourcesMetadata, *probe.Error)
// Object resource API.
GetObject(bucket, object string, startOffset int64) (io.ReadCloser, *probe.Error)
GetObjectInfo(bucket, object string) (ObjectInfo, *probe.Error)
PutObject(bucket string, object string, size int64, data io.Reader, metadata map[string]string) (ObjectInfo, *probe.Error)
DeleteObject(bucket, object string) *probe.Error
// Object query API.
NewMultipartUpload(bucket, object string) (string, *probe.Error)
PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Hex string) (string, *probe.Error)
ListObjectParts(bucket, object string, resources ObjectResourcesMetadata) (ObjectResourcesMetadata, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []CompletePart) (ObjectInfo, *probe.Error)
AbortMultipartUpload(bucket, object, uploadID string) *probe.Error
}
```
9 years ago
272c5165aa
Merge pull request #1272 from krishnasrinivas/get-auth
...
GetAuth implementation. min/max check for accessKey and secretKey.
9 years ago
e318925f62
credentials: min/max length check for credentials.
9 years ago
2395c42fb5
Merge pull request #1277 from krishnasrinivas/remove-minio-go2
...
UI-handler: remove minio-go dependancy.
9 years ago
9333dc3294
Merge pull request #1204 from hackintoshrao/test-bucket
...
Test: Changes to TestPutBucket to catch the race
9 years ago
30fc970eab
Changes to TestPutBucket to catch the race
9 years ago
331890c4c8
UI-handler: remove minio-go dependancy.
9 years ago
ae5c65d3c6
Merge pull request #1275 from harshavardhana/signature
...
error: Signature errors should be returned with APIErrorCode.
9 years ago
02ad48466d
error: Signature errors should be returned with APIErrorCode.
...
The reasoning is that we can reply back with wide range of
S3 error responses, which would provide more richer context
to S3 client.
Fixes #1267
9 years ago
a84c466a40
Merge pull request #1251 from harshavardhana/release-fixes
...
release: gz doesn't preserve permissions use tar.gz
9 years ago
956142be37
Merge pull request #1271 from krishnasrinivas/set-auth2
...
UI: implement SetAuth/GenerateAuth handlers for changing credentials.
9 years ago
5201905ad0
UI: implement SetAuth/GenerateAuth handlers for changing credentials.
9 years ago
186998ad99
Merge pull request #1266 from harshavardhana/cleanup
...
routers: Move API and Web routers into their own files.
9 years ago
aa8c9bad54
routers: Move API and Web routers into their own files.
...
This is done to ensure we have a clean way to add new routers such as
- diskRouter
- configRouter
- lockRouter
9 years ago
59ee5a547c
release: gz doesn't preserve permissions use tar.gz
...
And fix various other issues with release script.
9 years ago
1502e2f29f
Merge pull request #1265 from vadmeste/add_fbsd_support
...
Add simple FreeBSD support, make the minio project compilable
9 years ago
90a46faf31
Merge pull request #1228 from harshavardhana/signature-cleanup
...
signature: Move signature outside, use a layered approach for signature verification
9 years ago
9dca46e156
signature: Use a layered approach for signature verification.
...
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.
In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
9 years ago
663f24064b
Add simple FreeBSD support, make the minio project compilable
9 years ago
1b0bc814c4
docker: Fix docker Makefile.
9 years ago
cd5992c6db
Merge pull request #1262 from harshavardhana/docker-file
...
docker: Fix docker command entry.
9 years ago
1ef5ab3c28
docker: Fix docker command entry.
9 years ago
5bd47861d6
Merge pull request #1261 from harshavardhana/update-message
...
minio: Server upon start displays a message if update is available.
9 years ago
3538c9f598
minio: Server upon start displays a message if update is available.
...
This code also handles to turn itself off when network is not
available and if request fails. Also prints only when the update
is available.
9 years ago
24ae5467c8
Merge pull request #1260 from harshavardhana/minio
...
server: Print a message if not backends are configured.
9 years ago
36267eb6e2
server: Print a message if not backends are configured.
9 years ago
8255590b3c
config/main: set the missing value.
9 years ago
4f6cf5a6b2
Merge pull request #1123 from harshavardhana/rewrite-v1
...
config/main: Re-write config files - add to new config v3
9 years ago
aaf97ea02c
config/main: Re-write config files - add to new config v3
...
- New config format.
```
{
"version": "3",
"address": ":9000",
"backend": {
"type": "fs",
"disk": "/path"
},
"credential": {
"accessKey": "WLGDGYAQYIGI833EV05A",
"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
},
"region": "us-east-1",
"logger": {
"file": {
"enable": false,
"fileName": "",
"level": "error"
},
"syslog": {
"enable": false,
"address": "",
"level": "debug"
},
"console": {
"enable": true,
"level": "fatal"
}
}
}
```
New command lines in lieu of supporting XL.
Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~
Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~
For 'fs' backend it starts the server.
~~~
$ minio server
~~~
For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~
Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
9 years ago
85e50f2bb9
Merge pull request #1258 from hackintoshrao/list-object-prefix-response
...
api: GetObjectInfo fix
9 years ago
c8570edaab
Issue of 'mc ls' when prefix is a directory fixed, tests added for GetObjectInfo
9 years ago
f1161d830f
Merge pull request #1257 from harshavardhana/content-md5
...
routers: Fix order of PostPolicyHandlers and headers.
9 years ago
4aa098ede9
Merge pull request #1256 from harshavardhana/resources
...
bucketpolicy: checkBucketPolicy should keep resources in map.
9 years ago
76bda0d8f1
routers: Fix order of PostPolicyHandlers and headers.
9 years ago
996d2e2a10
bucketpolicy: checkBucketPolicy should keep resources in map.
...
This is done to make sure to avoid appending duplicates for
resources for each actions.
9 years ago
Bala.FA