d5ce2f6944
Make erasure matrix type not optional choose automatically
...
Remove option of providing Technique and handling errors based on that
choose a matrix type automatically based on number of data blocks.
INTEL recommends on using cauchy for consistent invertible matrices,
while vandermonde is faster we should default to cauchy for large
data blocks.
9 years ago
4ed50a8004
Fix encoding bug in donut during encoding phase
...
Stream reading needs to check for length parameter being non zero,
after Reading() a predefined set of buffer length an EOF might be returned
with length == 0.
Erasure taking this zeroed data in might wrongly encode it as part of existing
data blocks which leads to errors while decoding even when the other contents
are intact.
9 years ago
c8de5bad2f
Implement presigned policy
9 years ago
3b070dee16
Fix an important metadata getObject bug in donut
9 years ago
cdf93e534c
simplify probe APIs
9 years ago
45b59b8456
Probe revamped to provide for a new WrappedError struct to wrap probes as error interface
...
This convenience was necessary to be used for golang library functions like io.Copy and io.Pipe
where we shouldn't be writing proxies and alternatives returning *probe.Error
This change also brings more changes across code base for clear separation regarding where an error
interface should be passed encapsulating *probe.Error and where it should be used as is.
9 years ago
d09fd8b0a1
Migrate from iodine to probe
9 years ago
ee4432bc40
rename disk.OpenFile to Open which will do os.Open (which will be read-only). disk.OpenFile will do os.OpenFile (which can be rw, append)
9 years ago
fdd2c22fa5
Do go fmt on bucket.go
9 years ago
ae8089c9b6
when signature is not available there is no need to compute sha256
9 years ago
bdc00624fd
get erros from buffered channel. Return error during object read only if we have readers < k
9 years ago
e1280779ed
Read from the disks in parallel during object read
9 years ago
e082f26e10
Improving EncoderStream to return error only upon non io.EOF.
...
io.EOF is okay since io.ReadFull will not have read any bytes at all.
Also making error channel receive only for go routine.
9 years ago
bcfaa12a4d
Encoder now directly reads from the object stream. Using split.Stream() was causing lot of redundant memory operations.
9 years ago
63c9cf0c4b
Move from Minimalist Object Storage to Minio Cloud Storage
9 years ago
43c908d5b9
ListObjects now considers multipart objects, also move to upstream check.v1
9 years ago
c1da10a207
Donut multipart support no get, listobjects() support yet
9 years ago
e37c5315ec
Live multiple disk removal works properly
9 years ago
e1e4908515
Wire up sha512 matching inside donut along with md5sum
9 years ago
e885259584
Handle removal of disks - getObject() now reads if disks are missing underneath, add initial stub healing code
9 years ago
634f70f3b0
WriteObject() encoded data in go routine, fix another multipart issue
9 years ago
29838bb851
Cleanup temporary writers upon errors during putObject(), all metadata() write operations
9 years ago
15dd0df187
Support signature v4 at rest
9 years ago
d1deda3a96
Add API tests for both donut on disk and donut cache
9 years ago
3622fbc87d
Across donut, split, nimble some code cleanup
9 years ago
bbb89b5776
Add multi-thread protection and also allow atomic file creates, rename upon Close()
9 years ago
14844f48dd
Make donut do everything as an atomic operation, this avoids all the deadlocks and races
10 years ago
30fc14e703
Restructure codebase move crypto, checksum to top-level, move ``split`` into donut, move crypto/keys into api/auth
10 years ago
8a4e7bcdcf
Add full API tests, move storage/donut to donut, add disk tests as well
10 years ago
7c37e9d06a
Make donut fully integrated back into API handlers
10 years ago
38a6ce36e5
Remove slow AppendUniq code, rolling through over a slice is in-efficient
...
Remove it and use map instead
10 years ago
335c7827eb
More donut, cache, api cleanup
10 years ago
d07d0c670a
Return back proper errors in writeObjectData(), rename few functions
10 years ago
10c807f233
Put object on successful write returns full metadata, to avoid subsequent GetObjectMetadata() calls in driver
10 years ago
6921328b93
Avoid frivolous GetObjectMetadata() calls at driver level, return back all the information in donut ListObjects()
10 years ago
45a7eab804
An attempt to bring in memory layer into donut driver
10 years ago
f3c25bcfc4
Keeping the lexical order same add optimizations, provide a comprehensive response from ListObjects()
10 years ago
ae66ae42c4
Rename functions for their purpose
10 years ago
9a4680475f
Object metadata was wrongly misconstrued to be mutable, handle it
10 years ago
3aa6d90c5e
Avoid racy maps, read from disk on success return quickly. Many more optimizations
10 years ago
68974918ac
Donut moves to typed metadata, removing the necessity for strconv, parsing and string converstions
10 years ago
767d3743ee
Add new metadata definitions and use them
10 years ago
fb9adb5524
Donut cleanup another set
...
- Make sure to close all readers
- Fix errors in api_testsuite c.Assert(err, IsNil) should be done right after each function call
10 years ago
eec66f195a
Take all the ListObjects into bucket handlers
...
Earlier the listing would wait for all the objects to be processed
this is essentially very time consuming considering even for 100,000
files.
10 years ago
45e9d25931
Remove more bloated code - simplify
10 years ago
82dcbf262d
Add simple locking for donut API for now - fixes #671
10 years ago
03b4d3b308
Go vet fixes for donut
10 years ago
57a2b53178
Removing further bloated code simplifying
10 years ago
a2c205ff2e
Use external package disk for donut.
10 years ago
1682c748ac
Remove unnecessary interfaces from donut, cleanup
10 years ago
Harshavardhana