cef26fd6ea
XL: Refactor usage of reduceErrs and consistent behavior. ( #2240 )
...
This refactor is also needed in lieu of our quorum
requirement change for the newly understood logic behind
klauspost/reedsolom implementation.
8 years ago
5cc9e4e214
fs/XL: Return IncompleteBody{} error for short writes ( #2228 )
8 years ago
8cc163e51a
Refactor xl.GetObject and erasureReadFile. ( #2211 )
...
* XL: Refactor xl.GetObject and erasureReadFile. erasureReadFile() responsible for just erasure coding, it takes ordered disks and checkSum slice.
* move getOrderedPartsMetadata and getOrderedDisks to xl-v1-utils.go
* Review fixes.
8 years ago
b090c7112e
Refactor of xl.PutObjectPart and erasureCreateFile. ( #2193 )
...
* XL: Refactor of xl.PutObjectPart and erasureCreateFile.
* GetCheckSum and AddCheckSum methods for xlMetaV1
* Simple unit test case for erasureCreateFile()
8 years ago
cdf1373f8e
XL: Ignore and continue for cases when bucket does not exist. ( #2205 )
...
Fixes #2201
Fixes #2204
8 years ago
45240f158d
xl: Make namespace locking granular for PutObject ( #2199 )
8 years ago
dc3bafb194
XL: isQuorum rename as isDiskQuorum, word it properly. ( #2196 )
8 years ago
623e0f9243
XL: listOnlineDisks should use modTime instead of version. ( #2166 )
...
This change is needed to make reading from objects future proof
in-terms of handling online disks. Our current counter is not
based on affirmative knowledge and relies on arithmetic sequence
which can lead to bugs.
Using modTime simplifies the understanding of `xl.json` and future
tooling / debugging of the format.
8 years ago
e5cd35aad0
XL: GetObjectInfo() store and retrieve user-defined object metadata. ( #2189 )
8 years ago
1c82b81408
Rename parts/objects only on onlineDisks ( #2185 )
8 years ago
bef72f26db
xl: Make locking more granular for PutObjectPart requests ( #2168 )
8 years ago
ae80f8ca35
ObjectLayer/GetObject: Should return the right error value. Fix done in FS and XL. ( #2133 )
...
fixes #2117
8 years ago
c0c8a8430e
XL/PutObject: Add single putObject and multipart caching. ( #2115 )
...
- Additionally adds test cases as well for object cache.
- Adds auto-expiry with expiration and cleanup time interval.
Fixes #2080
Fixes #2091
8 years ago
6266328a85
XL/metadata: use new hashOrder algorithm for newXLMeta. ( #2147 )
8 years ago
ec35330ebb
XL/fs: GetObject should validate all its inputs. ( #2142 )
...
Fixes #2141
Fixes #2139
8 years ago
ddf3245677
xl/fs: offset and length cannot be negative. ( #2121 )
...
Fixes #2119
8 years ago
26b7d79a85
XL/ObjectCache: DeleteObject() should delete the object from the object cache. ( #2113 )
...
fixes #2103
8 years ago
01cbacd803
object-cache: use golang bytes.Buffer and bytes.NewReader instead of custom implementation. ( #2108 )
8 years ago
a35341448f
XL: Cache: Purging partially cached content upon erasureReadFile failure ( #2104 )
8 years ago
8a028a9efb
handler/PUT: Handle signature verification through a custom reader. ( #2066 )
...
Change brings in a new signVerifyReader which provides a io.Reader
compatible reader, additionally implements Verify() function.
Verify() function validates the signature present in the incoming
request. This approach is choosen to avoid complexities involved
in using io.Pipe().
Thanks to Krishna for his inputs on this.
Fixes #2058
Fixes #2054
Fixes #2087
8 years ago
d2557bb538
XL: GetObject caching implemented for XL. ( #2017 )
...
The object cache implementation is XL cache, which defaults
to 8GB worth of read cache. Currently GetObject() transparently
writes to this cache upon first client read and then subsequently
serves reads from the same cache.
Currently expiration is not implemented.
8 years ago
734e779b19
XL/erasureCreate: Create a limit reader if size is specified. ( #2059 )
...
This is needed so that we only write data which was requested
for, using a limit reader avoids spurious reads on the incoming
client data. Additionally using limit reader provides server
safety from rogue clients sending copious amounts of data (for
example a denial of service attack).
This patch also caters for size == -1 when content encoding from
a client is set as chunked, we happily read till io.EOF
9 years ago
4c1a11aae6
XL: allow meta bucket name appended with tmp meta prefix. ( #2007 )
9 years ago
4db2b03312
XL: Rename objectN to part.N ( #2019 )
...
Fixes #2015
9 years ago
2e1f66c37d
XL: Handle quorum situations properly for write operations. ( #1986 )
...
Adds two test cases one for
- PutObject when write quorum is not available.
- PutObjectPart when write quorum is not available.
Fixes #1951
9 years ago
d4bea5fbf8
XL/erasure-read: Add Comments and enable bitrot detection.
9 years ago
17efaaa902
XL/erasure-read: Support parallel reads from disks.
9 years ago
393c504de0
Renaming a part from tmp namespace needs to be handled different from… ( #1944 )
...
* Renaming a part from tmp namespace needs to be handled different from renaming of an object
* Made argument passing in xl.rename and xl.undoRename explicit
9 years ago
4ee2136b28
Unit tests for PUT object when object already exists ( #1904 )
...
* fs/xl tests for multiple put object requests
* xl fix for put object on directory
* Unit tests fix windows test issue.
9 years ago
1ea1dba528
erasure-readfile: write to given Writer than returning buffer. ( #1910 )
...
Fixes #1889
9 years ago
8c0942bf0d
XL: Remove usage of reduceErr and make it isQuorum verification. ( #1909 )
...
Fixes #1908
9 years ago
d31b38aac8
XL/GetObject: pick the xl.json with highest version for metadata information. ( #1914 )
...
fixes #1913
9 years ago
365f80efa3
XL/DeleteObject: delete call on a prefix should not delete the entire tree structure. ( #1916 )
...
fixes #1915
9 years ago
f51d34cedd
Do not guess content-type for objects with no extension ( #1918 )
9 years ago
de1c7d33eb
XL: appendFile should return error if quorum is not met. ( #1898 )
...
Fixes #1890
9 years ago
d13e6e7156
XL: return error if DeleteObject() fails. ( #1878 )
...
Previously DeleteObject() does not return any error if write quorum is
not available. This patch fixes the issue by returning errors.
9 years ago
82fd907933
XL/PutObject: Handle all pending cases of DiskNotFound.
9 years ago
b00ac40c35
XL/PutObject: Calculate size if not provided by the client and update xl.json with the correct size. ( #1844 )
9 years ago
fb95c1fad3
XL: Bring in some modularity into format verification and healing. ( #1832 )
9 years ago
ae311aa53b
XL: Cleanup, comments and all the updated functions. ( #1830 )
9 years ago
c493ab5d0d
XL: Bring in sha512 checksum support. ( #1797 )
9 years ago
967c2b2940
Handled possible short writes to httpResponseWriter ( #1804 )
...
* XL: Handled possible short writes to httpResponseWriter
* Added tests for Range Header combinations
9 years ago
a4a0ea605b
XL: Fix GetObject erasure decode issues. ( #1793 )
9 years ago
5e8de786b3
XL: Truly use unique id's in temp directory. ( #1790 )
...
This also helps in avoiding cleaning up directories after.
Additionally this patch also fixes the problem of Range offsets.
9 years ago
feb337098d
XL: bring in new storage API. ( #1780 )
...
Fixes #1771
9 years ago
5f679d9d1e
Rename back multipart objects if read/write Quorum was unavailable ( #1773 )
9 years ago
d65101a8c8
XL: Implement strided erasure distribution. ( #1772 )
...
Strided erasure distribution uses a new randomized
block distribution for each Put operation. This
information is captured inside `xl.json` for subsequent
Get operations.
9 years ago
b2293c2bf4
XL: Rename, cleanup and add more comments. ( #1769 )
...
- xl-v1-bucket.go - removes a whole bunch of code.
- {xl-v1,fs-v1}-metadata.go - add a lot of comments and rename functions
appropriately.
9 years ago
553fdb9211
XL: Bring in support for object versions written during writeQuorum. ( #1762 )
...
Erasure is initialized as needed depending on the quorum and onlineDisks.
This way we can manage the quorum at the object layer.
9 years ago
cae4782973
XL: explicit deleteObject is not needed after rename failure. ( #1760 )
...
Reason is renameObject() does deleteObject() upon writeQuorum
failure if not keeps the successfully renamed parts if we have
reached readQuorum.
9 years ago
Harshavardhana