minio

History

Harshavardhana 7e1661f4fa Performance improvements to SELECT API on certain query operations (#6752 ) This improves the performance of certain queries dramatically, such as 'count()' etc. Without this PR ``` ~ time mc select --query "select count() from S3Object" myminio/sjm-airlines/star2000.csv.gz 2173762 real 0m42.464s user 0m0.071s sys 0m0.010s ``` With this PR ``` ~ time mc select --query "select count(*) from S3Object" myminio/sjm-airlines/star2000.csv.gz 2173762 real 0m17.603s user 0m0.093s sys 0m0.008s ``` Almost a 250% improvement in performance. This PR avoids a lot of type conversions and instead relies on raw sequences of data and interprets them lazily. ``` benchcmp old new benchmark old ns/op new ns/op delta BenchmarkSQLAggregate_100K-4 551213 259782 -52.87% BenchmarkSQLAggregate_1M-4 6981901985 2432413729 -65.16% BenchmarkSQLAggregate_2M-4 13511978488 4536903552 -66.42% BenchmarkSQLAggregate_10M-4 68427084908 23266283336 -66.00% benchmark old allocs new allocs delta BenchmarkSQLAggregate_100K-4 2366 485 -79.50% BenchmarkSQLAggregate_1M-4 47455492 21462860 -54.77% BenchmarkSQLAggregate_2M-4 95163637 43110771 -54.70% BenchmarkSQLAggregate_10M-4 476959550 216906510 -54.52% benchmark old bytes new bytes delta BenchmarkSQLAggregate_100K-4 1233079 1086024 -11.93% BenchmarkSQLAggregate_1M-4 2607984120 557038536 -78.64% BenchmarkSQLAggregate_2M-4 5254103616 1128149168 -78.53% BenchmarkSQLAggregate_10M-4 26443524872 5722715992 -78.36% ```		6 years ago
..
auth	Check key length before adding a new user. (#6790 )	6 years ago
bpool	Add large bucket support for erasure coded backend (#5160 )	7 years ago
certs	Watch for symlinked certs in container envs (#6282 )	6 years ago
cgroup	sys/stats: return cgroup mem limit, fall back to sysinfo() (#4002 )	7 years ago
disk	Implement support for calculating disk usage per tenant (#5969 )	6 years ago
dns	Update federation target to etcd/clientv3 (#6119 )	6 years ago
ellipses	Add large bucket support for erasure coded backend (#5160 )	7 years ago
event	Enable default vet flags (#6810 )	6 years ago
handlers	Avoid logging the body of the http 206 response (#6258 )	6 years ago
hash	Add object compression support (#6292 )	6 years ago
iam	Re-populate public key if JWT fails to parse (#6786 )	6 years ago
ioutil	Add GetObjectNInfo to object layer (#6449 )	6 years ago
lock	Fix goroutine test fatalf (#6682 )	6 years ago
madmin	Check key length before adding a new user. (#6790 )	6 years ago
mimedb	Fix mimedb update files (#6744 )	6 years ago
mountinfo	disable disk-usage when export is root mount path (#6091 )	6 years ago
net	Remove panic() and handle it appropriately (#5807 )	6 years ago
policy	Sid value can be any unicode character support it (#6676 )	6 years ago
quick	Migrate config.json from config-dir to backend (#6195 )	6 years ago
s3select	Performance improvements to SELECT API on certain query operations (#6752 )	6 years ago
safe	Remove go1.9 specific code for windows (#5033 )	7 years ago
sync/errgroup	Add large bucket support for erasure coded backend (#5160 )	7 years ago
sys	Push max threads to little less than kernel limit (#5001 )	7 years ago
trie	trie: new package (#3729 )	8 years ago
wildcard	pkg/wildcard: Simplify the wildcard logic further. (#2555 )	8 years ago
words	words: new package Damerau Levenshtein distance function. (#3929 )	8 years ago