8cc163e51a
Refactor xl.GetObject and erasureReadFile. ( #2211 )
...
* XL: Refactor xl.GetObject and erasureReadFile. erasureReadFile() responsible for just erasure coding, it takes ordered disks and checkSum slice.
* move getOrderedPartsMetadata and getOrderedDisks to xl-v1-utils.go
* Review fixes.
9 years ago
b090c7112e
Refactor of xl.PutObjectPart and erasureCreateFile. ( #2193 )
...
* XL: Refactor of xl.PutObjectPart and erasureCreateFile.
* GetCheckSum and AddCheckSum methods for xlMetaV1
* Simple unit test case for erasureCreateFile()
9 years ago
ddf3245677
xl/fs: offset and length cannot be negative. ( #2121 )
...
Fixes #2119
9 years ago
5291db60c6
XL/erasure: refactor erasureReadFile. Move parallelRead into a separate function. ( #2008 )
9 years ago
ce7d5eddbc
Misspell warnings fix ( #2001 )
9 years ago
9add048f3c
erasure: Add erasure encode/decode unit tests. ( #1911 )
...
Fixes #1819
9 years ago
cb1200a66d
XL/erasure-read: read disks in parallel. ( #1975 )
...
On read failure, fallback to reads from other
disks also happen in parallel.
9 years ago
e8990e42c2
XL: Make allocations simpler avoid redundant allocs. ( #1961 )
...
- Reduce 10MiB buffers for loopy calls to use 128KiB.
- start using 128KiB buffer where needed.
9 years ago
9b82e64a11
XL/erasure-read: Avoid memory copy, write to writer directly all the dataBlocks.
9 years ago
d4bea5fbf8
XL/erasure-read: Add Comments and enable bitrot detection.
9 years ago
17efaaa902
XL/erasure-read: Support parallel reads from disks.
9 years ago
7d757033f2
erasure-readfile: Use chunk size to read from each disk for a block. ( #1949 )
...
A block of data is split into data chunk and each data chunk is
written to each disk. Previously block size was used to read data
chunk which returns corrupted data.
This patch fixes the issue by reading chunk sized data from each disk
and assembles a block.
Fixes #1939
9 years ago
1ea1dba528
erasure-readfile: write to given Writer than returning buffer. ( #1910 )
...
Fixes #1889
9 years ago
c6ac3fa6db
erasure: Fix block index matching.
...
This patch fixes an important issue of block reconstruction upon block corruption.
9 years ago
fb95c1fad3
XL: Bring in some modularity into format verification and healing. ( #1832 )
9 years ago
de21126f7e
XL: Re-align the code again.
9 years ago
ae311aa53b
XL: Cleanup, comments and all the updated functions. ( #1830 )
9 years ago
c493ab5d0d
XL: Bring in sha512 checksum support. ( #1797 )
9 years ago
db2fdbf38d
erasure: allocate buffer only for non-nil disk ( #1811 )
9 years ago
967c2b2940
Handled possible short writes to httpResponseWriter ( #1804 )
...
* XL: Handled possible short writes to httpResponseWriter
* Added tests for Range Header combinations
9 years ago
a4a0ea605b
XL: Fix GetObject erasure decode issues. ( #1793 )
9 years ago
feb337098d
XL: bring in new storage API. ( #1780 )
...
Fixes #1771
9 years ago
27cc8a6529
erasure: read only dataBlocks if we have enough. ( #1776 )
...
Reconstruct with parity blocks if we don't have enough data blocks.
9 years ago
d65101a8c8
XL: Implement strided erasure distribution. ( #1772 )
...
Strided erasure distribution uses a new randomized
block distribution for each Put operation. This
information is captured inside `xl.json` for subsequent
Get operations.
9 years ago
553fdb9211
XL: Bring in support for object versions written during writeQuorum. ( #1762 )
...
Erasure is initialized as needed depending on the quorum and onlineDisks.
This way we can manage the quorum at the object layer.
9 years ago
a97230dd56
XL/erasure: Reset dataBlocks to reduce the memory usage. ( #1749 )
...
Fixes #1748
9 years ago
1e393c6c5b
XL: Add new metadata for checksum. ( #1743 )
9 years ago
b38b9fea79
XL/erasure: fix for skipping 0 padding. ( #1737 )
...
Fixes #1736
9 years ago
293d246f95
XL/FS: Rewrite in new format.
9 years ago
3c1ef3fee2
Locking: move locking code from xl-erasure to xl-objects. ( #1673 )
...
Fixes #1639 #1649 #1650 #1624
9 years ago
9472299308
logging: Log only for unhandled errors, remove all the debug logging. ( #1652 )
...
This patch brings in the removal of debug logging altogether, instead
we bring in the functionality of being able to trace the errors properly
pointing back to the origination of the problem.
To enable tracing you need to enable "MINIO_TRACE" set to "1" or "true"
environment variable which would print back traces whenever there is an
error which is unhandled or at the handler layer.
By default this tracing is turned off and only user level logging is
provided.
9 years ago
025054fb36
XL: CreateFile/ReadFile should write and read from all disks in parallel. ( #1612 )
...
* XL: CreateFile should write to all disks in parallel.
* XL: ReadFile should read from all disks in parallel.
9 years ago
72748d2073
erasure: healVolume err should be different from shadowed version. ( #1590 )
...
Multiple go-routines updating the same shadowed variable can
cause a data race, avoid it by using its own err variable.
Fixes #1589
9 years ago
b044336329
XL/GetObject: If the offset does not fall in the first "dataBlock" it gives incorrect data. ( #1583 )
...
Fixes #1582
9 years ago
a8fdd04e62
erasure: ReadFile should honor proper offsets. ( #1542 )
...
Fixes #1535
9 years ago
0b74f5624e
xl: Fix how we deal with read offsets at erasure layer. ( #1511 )
...
Requires skipping necessary parts of dataBlocks during
decoding phase and requires us to properly skip the
entries as needed.
Thanks to Karthic for reproducing this important issue.
Fixes #1503
9 years ago
4e34e03dd4
xl/fs: Split object layer into interface. ( #1415 )
9 years ago
984903cce1
server: Add global namespace lock. ( #1398 )
...
Fixes #1393
9 years ago
a1a667ae5d
xl: Change fileMetadata to xlMetadata. ( #1404 )
...
Finalized backend format
```
{
"version": "1.0.0",
"stat": {
"size": 24256,
"modTime": "2016-04-28T00:11:37.843Z"
},
"erasure": {
"data": 5,
"parity": 5,
"blockSize": 4194304
],
"minio": {
"release": "RELEASE.2016-04-28T00-09-47Z"
}
}
```
9 years ago
5fffd558d0
xl/heal: Make healFile non-blocking for StatFile and ReadFile. ( #1399 )
...
Fixes #1355
9 years ago
ad1abc4486
xl-v1/Cleanup: use listOnlineDisks instead of getReadableDisks. ( #1389 )
...
Remove usage of getFileVersionQuorumMap, instead use listFileVersions
to get the version list and extract higherVersion.
Fixes #1379
Fixes #1378
Fixes #1377
9 years ago
8bce699dae
xl: Add logging. ( #1372 )
9 years ago
8c85815106
xl: refactor functions to xl-v1-common.go xl-v1-utils.go. ( #1357 )
9 years ago
becc814531
Xl layer selfheal quorum2
...
* xl/selfheal: selfheal based on read quorum on GET
* xl: getReadableDisks() also returns whether self-heal is needed so that this info can be used by ReadFile/SelfHeal/StatFile.
* xl: trigger selfheal from StatFile.
9 years ago
9bd9441107
xl: Simplify reading metadata and add a new fileMetadata type. ( #1346 )
9 years ago
5c33b68318
xl: code refactor, cleanup ReadFile and CreateFile.
9 years ago
45b3d3e21f
xl: add quorum support for create file
9 years ago
141a44bfbf
xl: Fix ReadFile to keep the order always for reading the data back. ( #1339 )
...
Also fixes a stackoverflow bug in namespace locking.
9 years ago
c7bf471c9e
list/xl: Fix the way marker is handled in leafDirectory verification.
9 years ago
b76f3f1d62
xl: Add more fixes and cleanup.
...
Simplify cleanup of temporary files during createFile operations.
9 years ago
Krishna Srinivas