Harshavardhana
5cfb05465e
Add cache, donut tests separately - fix behavior differences
...
Remove priority queue, implement it using a simpler channels
9 years ago
Harshavardhana
ebe61d99d9
Use cache Append() for saving objects in memory, GetObject() caches un-cached entries while reading
9 years ago
Harshavardhana
bce93c1b3a
Integrate cache with donut, add tests
9 years ago
Harshavardhana
0533abf6a8
Make priority queue lambda function return error over a channel
9 years ago
Harshavardhana
38a6ce36e5
Remove slow AppendUniq code, rolling through over a slice is in-efficient
...
Remove it and use map instead
9 years ago
Harshavardhana
84810162f5
Add simple Version and GetSysInfo services
9 years ago
Harshavardhana
14ec42d646
Add initial implementation of priority queue, uses container/heap
9 years ago
Harshavardhana
eb5aa19dfa
Remove custom Config, will use quick Config instead for user access keys
9 years ago
Harshavardhana
701c3e5242
Add new RPC helpers wrapping over regular rpc packages, add middleware chaining ability
9 years ago
Harshavardhana
4addf7a996
Restructure API handlers, add JSON RPC simple HelloService right now.
9 years ago
Harshavardhana
335c7827eb
More donut, cache, api cleanup
9 years ago
Harshavardhana
dc0df3dc0e
Breakaway from driver model, move cache into donut
9 years ago
Harshavardhana
72572d6c71
Remove some api server code bringing in new cleanup
9 years ago
Harshavardhana
c2031ca066
Add server and control command
9 years ago
Frederick F. Kautz IV
cfbc169034
Fixing API definitions
9 years ago
Harshavardhana
8f61d6b6be
Move memory code out, add it as layer on top of existing donut code
...
Just like how http.Handlers can be overlayed on top of each other
with each implementing ServeHTTP().
drivers.Driver can be overlayed on top of each other in similar manner
which would implement the drivers.Driver interface.
API <----> cache <----> donut <----> donut(format)
9 years ago
Harshavardhana
ab6e16bb41
Add dummy driver for community to submit new drivers
9 years ago
Harshavardhana
12de98fb62
Rename memory driver as cache
9 years ago
Harshavardhana
2571342451
Filesystem goes the high road *again*
9 years ago
Harshavardhana
f74d6138da
IsValidBucket() is sufficient we don't need to verify for "."
9 years ago
Harshavardhana
3109909355
Handle couple of cases of OOM conditions, move caching to GetObject() rather than PutObject()
9 years ago
Harshavardhana
d07d0c670a
Return back proper errors in writeObjectData(), rename few functions
9 years ago
Harshavardhana
10c807f233
Put object on successful write returns full metadata, to avoid subsequent GetObjectMetadata() calls in driver
9 years ago
Harshavardhana
6921328b93
Avoid frivolous GetObjectMetadata() calls at driver level, return back all the information in donut ListObjects()
9 years ago
Harshavardhana
d8f7896a43
Expand http server struct to store more values
9 years ago
Harshavardhana
42c0287943
Use errorChannels only for services not for drivers, reduce them to use simple functions
9 years ago
Harshavardhana
91e5f648cb
Move to Set() not Append() due to large memory reference copy
9 years ago
Harshavardhana
ac4d8fe478
Add free() method for proxyReader to aggressively de-allocate Read data, to handle certain out of memory conditions
...
There are still some more out there
9 years ago
Harshavardhana
05f8654e3d
Add Append() method to trove cache for appending data to an existing key
...
This largely avoids a large buffer copy which would accumulate inside proxyReader{}
This patch also implements "initialize()" function to init and populate data
on all the existing buckets, avoiding the redundant ListBuckets() invoked by
every API call.
9 years ago
Harshavardhana
367772b988
Make sure to populate on disk data into memory upon first API requests
9 years ago
Harshavardhana
45a7eab804
An attempt to bring in memory layer into donut driver
9 years ago
Harshavardhana
f3c25bcfc4
Keeping the lexical order same add optimizations, provide a comprehensive response from ListObjects()
9 years ago
Harshavardhana
ae66ae42c4
Rename functions for their purpose
9 years ago
Harshavardhana
9a4680475f
Object metadata was wrongly misconstrued to be mutable, handle it
9 years ago
Harshavardhana
3aa6d90c5e
Avoid racy maps, read from disk on success return quickly. Many more optimizations
9 years ago
Harshavardhana
68974918ac
Donut moves to typed metadata, removing the necessity for strconv, parsing and string converstions
9 years ago
Harshavardhana
767d3743ee
Add new metadata definitions and use them
9 years ago
Harshavardhana
9958e34772
Import quick key value store from Minio Client for persistent state files, primarily for donut
9 years ago
Harshavardhana
fb9adb5524
Donut cleanup another set
...
- Make sure to close all readers
- Fix errors in api_testsuite c.Assert(err, IsNil) should be done right after each function call
9 years ago
Harshavardhana
eec66f195a
Take all the ListObjects into bucket handlers
...
Earlier the listing would wait for all the objects to be processed
this is essentially very time consuming considering even for 100,000
files.
9 years ago
Harshavardhana
45e9d25931
Remove more bloated code - simplify
9 years ago
Harshavardhana
82dcbf262d
Add simple locking for donut API for now - fixes #671
9 years ago
Harshavardhana
03b4d3b308
Go vet fixes for donut
9 years ago
Harshavardhana
57a2b53178
Removing further bloated code simplifying
9 years ago
Harshavardhana
a2c205ff2e
Use external package disk for donut.
9 years ago
Harshavardhana
841ff01412
Move disk into its own package, remove bloated code
9 years ago
Harshavardhana
1682c748ac
Remove unnecessary interfaces from donut, cleanup
9 years ago
Harshavardhana
3498872467
Add sha256 and sha512 windows compatibility layer
9 years ago
Harshavardhana
77d35b87d4
Fix a bug on windows regarding blockSSE3 calculation
9 years ago
Harshavardhana
f1410731db
Add windows code for sha1 and crc32c
9 years ago