c0c8a8430e
XL/PutObject: Add single putObject and multipart caching. ( #2115 )
...
- Additionally adds test cases as well for object cache.
- Adds auto-expiry with expiration and cleanup time interval.
Fixes #2080
Fixes #2091
8 years ago
6266328a85
XL/metadata: use new hashOrder algorithm for newXLMeta. ( #2147 )
8 years ago
ca1b1921c4
XL: Implement ignore errors. ( #2136 )
...
Each metadata ops have a list of errors which can be
ignored, this is essentially needed when
- disks are not found
- disks are found but cannot be accessed (permission denied)
- disks are there but fresh disks were added
This is needed since we don't have healing code in place where
it would have healed the fresh disks added.
Fixes #2072
8 years ago
f55093cdd6
multipart: During multipart list the listing go-routine should be saved to the List-pool. ( #2130 )
8 years ago
8a028a9efb
handler/PUT: Handle signature verification through a custom reader. ( #2066 )
...
Change brings in a new signVerifyReader which provides a io.Reader
compatible reader, additionally implements Verify() function.
Verify() function validates the signature present in the incoming
request. This approach is choosen to avoid complexities involved
in using io.Pipe().
Thanks to Krishna for his inputs on this.
Fixes #2058
Fixes #2054
Fixes #2087
8 years ago
7a8b8cd0a1
tree-walk: unify FS and XL tree-walk with functional approach. ( #2027 )
8 years ago
734e779b19
XL/erasureCreate: Create a limit reader if size is specified. ( #2059 )
...
This is needed so that we only write data which was requested
for, using a limit reader avoids spurious reads on the incoming
client data. Additionally using limit reader provides server
safety from rogue clients sending copious amounts of data (for
example a denial of service attack).
This patch also caters for size == -1 when content encoding from
a client is set as chunked, we happily read till io.EOF
8 years ago
64899e5197
xl: Used unique tmp file to update xl.json in putObjectPart ( #2043 )
...
An in-place update to xl.json amidst concurrent PutObjectPart operations
lead to racy updates to xl.json making it un-parseable. To avoid this,
we create a unique tmp file where updates to xl.json are staged before
renaming it to the final location.
8 years ago
ae936a0147
XL: Relax write quorum further to N/2 + 1. ( #2018 )
...
This changes behavior in some parts of the code
as well address it.
Fixes #2016
8 years ago
748dc80047
API: add writePartTooSmallErrorResponse to extend standard error responses. ( #2005 )
...
This function is added to extend the standard error responses.
Which is needed in some cases for example CompleteMultipartUpload
should respond with ErrPartTooSmall error when parts uploaded are
lesser than 5MB (i.e minimum allowed size per part).
Fixes #1536
8 years ago
4db2b03312
XL: Rename objectN to part.N ( #2019 )
...
Fixes #2015
8 years ago
2e1f66c37d
XL: Handle quorum situations properly for write operations. ( #1986 )
...
Adds two test cases one for
- PutObject when write quorum is not available.
- PutObjectPart when write quorum is not available.
Fixes #1951
8 years ago
393c504de0
Renaming a part from tmp namespace needs to be handled different from… ( #1944 )
...
* Renaming a part from tmp namespace needs to be handled different from renaming of an object
* Made argument passing in xl.rename and xl.undoRename explicit
8 years ago
4ee2136b28
Unit tests for PUT object when object already exists ( #1904 )
...
* fs/xl tests for multiple put object requests
* xl fix for put object on directory
* Unit tests fix windows test issue.
8 years ago
e1aad066c6
XL: CompleteMultipart should ignore last part is 0bytes. ( #1931 )
...
Fixes #1917
8 years ago
8c0942bf0d
XL: Remove usage of reduceErr and make it isQuorum verification. ( #1909 )
...
Fixes #1908
8 years ago
de1c7d33eb
XL: appendFile should return error if quorum is not met. ( #1898 )
...
Fixes #1890
8 years ago
276282957e
Test for Complete Multipart Upload. ( #1888 )
8 years ago
61598ed02f
posix: return errFaultyDisk on I/O errors. ( #1885 )
...
When I/O error is occured more than allowed limit, posix returns
errFaultyDisk.
Fixes #1884
8 years ago
51f3d4e0ca
XL/multipart: statPart should ignore errDiskNotFound. ( #1862 )
...
startPart should also take uploadId and partName as arguments.
8 years ago
1cf1532ca3
XL: Implement ListObjects channel and pool management.
8 years ago
82fd907933
XL/PutObject: Handle all pending cases of DiskNotFound.
8 years ago
b00ac40c35
XL/PutObject: Calculate size if not provided by the client and update xl.json with the correct size. ( #1844 )
8 years ago
fb95c1fad3
XL: Bring in some modularity into format verification and healing. ( #1832 )
8 years ago
aa1d769b1e
FS/Multipart: remove uploads.json on complete-multipart if no more uploadIDs are present for the object. ( #1843 )
...
Fixes #1835
8 years ago
ae311aa53b
XL: Cleanup, comments and all the updated functions. ( #1830 )
8 years ago
c493ab5d0d
XL: Bring in sha512 checksum support. ( #1797 )
8 years ago
445dc22118
XL: Cleanup and add more comments. ( #1807 )
8 years ago
b466f27705
Nslock fixes ( #1803 )
...
* XL/Multipart: Support parallel upload of parts by doing NS locking appropriately.
* XL/Multipart: hold lock on the multipart upload while aborting.
8 years ago
a4a0ea605b
XL: Fix GetObject erasure decode issues. ( #1793 )
8 years ago
5e8de786b3
XL: Truly use unique id's in temp directory. ( #1790 )
...
This also helps in avoiding cleaning up directories after.
Additionally this patch also fixes the problem of Range offsets.
8 years ago
feb337098d
XL: bring in new storage API. ( #1780 )
...
Fixes #1771
8 years ago
c87f259820
Remove parts that are missing in CompleteMultipartUpload ( #1786 )
...
* Remove parts that are missing in CompleteMultipartUpload
* Moved isUploadIDExists under proper namespace locks
* Moved code that deletes part files to a function
8 years ago
41a5b3908b
XL/ListParts: take the size from xl.json instead of backend file size as it will be different. ( #1781 )
...
Fixes #1779
8 years ago
3fb0b5e455
XL/Multipart: check existance upload uploadID after lock. ( #1778 )
...
Fixes #1767
8 years ago
302ec27fa2
Fixed race during parallel PutObjectPart requests ( #1775 )
...
The race is between two parallel PutObjectPart requests updating partsInfo in xl.json.
Previously, it was being updated under a RLock().
8 years ago
d65101a8c8
XL: Implement strided erasure distribution. ( #1772 )
...
Strided erasure distribution uses a new randomized
block distribution for each Put operation. This
information is captured inside `xl.json` for subsequent
Get operations.
8 years ago
6dc8323684
FS/ListMultipart: Fix FS list-multipart to work for unit test cases.
8 years ago
b2293c2bf4
XL: Rename, cleanup and add more comments. ( #1769 )
...
- xl-v1-bucket.go - removes a whole bunch of code.
- {xl-v1,fs-v1}-metadata.go - add a lot of comments and rename functions
appropriately.
8 years ago
553fdb9211
XL: Bring in support for object versions written during writeQuorum. ( #1762 )
...
Erasure is initialized as needed depending on the quorum and onlineDisks.
This way we can manage the quorum at the object layer.
8 years ago
cae4782973
XL: explicit deleteObject is not needed after rename failure. ( #1760 )
...
Reason is renameObject() does deleteObject() upon writeQuorum
failure if not keeps the successfully renamed parts if we have
reached readQuorum.
8 years ago
a4771265cf
XL: Abortmultipart should update `uploads.json` properly. ( #1757 )
8 years ago
a9e778f460
XL/fs: initObjectLayer should cleanup tmpMetaPrefix in parallel. ( #1752 )
...
Fixes #1747
8 years ago
ee6645f421
XL: Add additional PartNumber variable as part of `xl.json` ( #1750 )
...
This is needed for verification of incoming parts and to
support variadic part uploads. Which should be sorted
properly.
Fixes #1740
8 years ago
6d84e84b3c
XL/mutltipart: fix partnumber to partname association. ( #1739 )
...
Fixes #1738
8 years ago
a00a5c6e7e
XL: Multipart update uploads.json properly. ( #1741 )
8 years ago
293d246f95
XL/FS: Rewrite in new format.
8 years ago
Harshavardhana