minio

Commit Graph

Author	SHA1	Message	Date
Harshavardhana	39face27cf	Simplify k8s replicated set deployment (#8666 ) Continuation from #8629 which basically broke zone deployments on k8s statefulset environment due to incorrect assumptions which made it work on replicated set. Fix this properly such that this container works for both replicated set and stateful set deployment	5 years ago
Andreas Auernhammer	e047ac52b8	remove github.com/minio/kes as a dependency (#8665 ) This commit removes github.com/minio/kes as a dependency and implements the necessary client-side functionality without relying on the KES project. This resolves the licensing issue since KES is licensed under AGPL while MinIO is licensed under Apache.	5 years ago
poornas	04de3ea4bd	Change cache purge routine granularity to hours (#8660 ) With this PR,cache eviction will continue until no LRU entries older than an hour can be cache evicted or sufficient percentage of disk space has been reclaimed.	5 years ago
Amol Umbarkar	e6ce9da087	fix BucketForward Handler for federated setup (#8646 ) fixes #8595	5 years ago
Harshavardhana	c9c0d5eec2	Allow CNAME records when specified as MINIO_PUBLIC_IPS (#8662 ) This is necessary for `m3` global bucket support	5 years ago
Harshavardhana	63c3114657	fix: doc notifications formatting issues (#8661 )	5 years ago
Harshavardhana	9bb0869b73	fix: populate buckets on etcd after config has loaded (#8658 )	5 years ago
Harshavardhana	5f2318567e	Allow metadata updates on meta bucket even in WORM mode (#8657 ) This ensures that we can update the - .minio.sys is updated for accounting/data usage purposes - .minio.sys is updated to indicate if backend is encrypted or not.	5 years ago
kannappanr	16ac4a3c64	PutBucketLifeCycleConfiguration: Return 200 instead of 204 (#8656 )	5 years ago
Harshavardhana	c8d82588c2	Fix crash in console logger and also handle bucket DNS updates (#8654 ) Also fix listenBucketNotification bugs seen by minio-js listen bucket notification API.	5 years ago
Harshavardhana	1dc5f2d0af	Remove safe mode for invalid entries in config (#8650 ) The approach is that now safe mode is only invoked when we cannot read the config or under some catastrophic situations, but not under situations when config entries are invalid or unreachable. This allows for maximum availability for MinIO and not fail on our users unlike most of our historical releases.	5 years ago
Harshavardhana	c10ecacf91	Always use SourceIP for host target filtering (#8649 )	5 years ago
poornas	1cf3e3b7b5	PutBucket: Case-insensitive validation of x-amz-bucket-object-lock-enabled (#8648 ) Fix: case insensitive validation of x-amz-bucket-object-lock-enabled header in PutBucket handler	5 years ago
Andreas Auernhammer	c3d4c1f584	add minio/keys KMS integration (#8631 ) This commit adds support for the minio/kes KMS. See: https://github.com/minio/kes In particular you can configure it as KMS by: - `export MINIO_KMS_KES_ENDPOINT=` // Server URL - `export MINIO_KMS_KES_KEY_FILE=` // TLS client private key - `export MINIO_KMS_KES_CERT_FILE=` // TLS client certificate - `export MINIO_KMS_KES_CA_PATH=` // Root CAs issuing server cert - `export MINIO_KMS_KES_KEY_NAME=` // The name of the (default) master key	5 years ago
Harshavardhana	471a3a650a	fix: Don't allow to set unconfigured notification ARNs (#8643 ) Fixes #8642	5 years ago
Harshavardhana	cc02bf0442	Remove old ListenBucketNotification API (#8645 )	5 years ago
Harshavardhana	39e8e4f4aa	Allow empty target KVS for notification targets (#8644 ) This is allowed with enable=off arg value	5 years ago
poornas	80558e839d	Clear cache if reverting to backend (#8637 ) Clear cached entry before reverting to backend for encrypted objects or those under retention to avoid stale objects remaining in cache.	5 years ago
Harshavardhana	ca62ac65d4	Reject mandatory KVS if not set for any sub-sys (#8641 )	5 years ago
Harshavardhana	f5abe4e1f1	Support ListenBucketNotificationV2 streaming (#8622 )	5 years ago
Klaus Post	3211cb5df6	Add encryption buffer (#8626 ) Quite hard to measure difference: ``` λ warp cmp put-before.csv.zst put-after2.csv.zst Operation: PUT Operations: 340 -> 353 * Average: +4.11% (+22.7 MB/s) throughput, +4.11% (+0.2) obj/s * 50% Median: +1.58% (+7.3 MB/s) throughput, +1.58% (+0.1) obj/s ``` Difference is likely bigger on Intel platforms due to higher syscall costs.	5 years ago
Ashish Kumar Sinha	abc266caa1	Add bucket and object count along with total object size (#8639 )	5 years ago
Harshavardhana	c364f0af6c	Start using custom HTTP transport for webhook endpoints (#8630 ) Use a more performant http transport for webhook endpoints with proper connection pooling, appropriate timeouts etc.	5 years ago
Anis Elleuch	555969ee42	Add data usage collect with its new admin API (#8553 ) Admin data usage info API returns the following (Only FS & XL, for now) - Number of buckets - Number of objects - The total size of objects - Objects histogram - Bucket sizes	5 years ago
Ashish Kumar Sinha	e2c5d29017	Bucket,Object count & Usage removed if set to default (#8638 )	5 years ago
Harshavardhana	fa00a84709	Avoid crashes on peers if IAMSys is not initialized (#8636 )	5 years ago
kannappanr	d266b3a066	Admin Info: Modify Uptime to return seconds (#8635 )	5 years ago
Ashish Kumar Sinha	24fb1bf258	New Admin Info (#8497 )	5 years ago
Harshavardhana	8b803491af	fix: CacheOpts parsing tests (#8632 )	5 years ago
Harshavardhana	10b2f15f6f	Add randomize sleep times for lock checkers (#8628 )	5 years ago
Harshavardhana	3e9ab5f4a9	Fix k8s replica set deployment (#8629 ) In replica sets, hosts resolve to localhost IP automatically until the deployment fully comes up. To avoid this issue we need to wait for such resolution.	5 years ago
Krishna Srinivas	3b67f629a4	Retry peer notification of events (#8621 )	5 years ago
poornas	3c30e4503d	Cache only the range requested for range GETs (#8599 )	5 years ago
poornas	8390bc26db	Fix cache hit metrics. (#8617 )	5 years ago
Nitish Tiwari	24ad59316d	Use atomic.Uint64 for gateway metrics count instead of mutex (#8615 )	5 years ago
poornas	be0c8b1ec0	Add support for missing Cache-Control directives (#8619 ) no-cache, only-if-cached and no-store directives are being enforced in this PR.	5 years ago
Harshavardhana	476111968a	Update help messages with new wording (#8616 ) Final update to all messages across sub-systems after final review, the only change here is that NATS now has TLS and TLSSkipVerify to be consistent for all other notification targets.	5 years ago
Harshavardhana	97deba2a7c	GetKVS should add new keys automatically, preserve order (#8612 )	5 years ago
Nitish Tiwari	3df7285c3c	Add Support for Cache and S3 related metrics in Prometheus endpoint (#8591 ) This PR adds support below metrics - Cache Hit Count - Cache Miss Count - Data served from Cache (in Bytes) - Bytes received from AWS S3 - Bytes sent to AWS S3 - Number of requests sent to AWS S3 Fixes #8549	5 years ago
Aleksandr Petruhin	d2dc964cb5	Support TLS auth for Kafka notification target (#8609 )	5 years ago
Harshavardhana	d8e3de0cae	Ensure comment is always a valid key (#8604 ) Also fix LDAP leaky connection	5 years ago
Harshavardhana	c9940d8c3f	Final changes to config sub-system (#8600 ) - Introduces changes such as certain types of errors that can be ignored or which need to go into safe mode. - Update help text as per the review	5 years ago
Harshavardhana	794eb54da8	Export command prints turned-off sub-sys as comments (#8594 ) This PR also tries to - Preserve the order of keys printed in export command - Fix cache to be enabled with _STATE env to keep backward compatibility	5 years ago
Harshavardhana	2ab8d5e47f	Enable build verification with race (#8583 )	5 years ago
Clemens Wolff	947bc8c7d3	Update Azure Gateway to azure-storage-blob SDK (#8537 ) The azure-sdk-for-go/storage package has been in maintenance- only mode since February 2018 (see [1]) and will be deprecated in the future.	5 years ago
Harshavardhana	5d3d57c12a	Start using error wrapping with fmt.Errorf (#8588 ) Use fatih/errwrap to fix all the code to use error wrapping with fmt.Errorf()	5 years ago
Harshavardhana	0bfd20a8e3	Add client_id support for OpenID (#8579 ) - One click OpenID authorization on Login page - Add client_id help, config keys etc Thanks to @egorkaru @ihostage for the original work and testing.	5 years ago
Klaus Post	db3dbcce3a	Print goroutines when shutdown hangs (#8574 )	5 years ago
Harshavardhana	b21835f195	Honor DurationSeconds properly for WebIdentity (#8581 ) Also cleanup code to add various constants for verbatim strings across the code base. Fixes #8482	5 years ago
Klaus Post	c7844fb1fb	posix: cache disk ID for a short while (#8564 ) `posix.getDiskID()` takes up to 30% of all CPU due to the `os.Stat` call on `GET` calls. Before: ``` Operation: GET - Concurrency: 12 Average: 1333.97 MB/s, 1365.99 obj/s, 1365.98 ops ended/s (4m59.975s) * First Byte: Average: 7.801487ms, Median: 7.9974ms, Best: 1.9822ms, Worst: 110.0021ms Aggregated, split into 299 x 1s time segments: * Fastest: 1453.50 MB/s, 1488.38 obj/s, 1492.00 ops ended/s (1s) * 50% Median: 1360.47 MB/s, 1393.12 obj/s, 1393.00 ops ended/s (1s) * Slowest: 978.68 MB/s, 1002.17 obj/s, 1004.00 ops ended/s (1s) ``` After: ``` Operation: GET - Concurrency: 12 * Average: 1706.07 MB/s, 1747.02 obj/s, 1747.01 ops ended/s (4m59.985s) * First Byte: Average: 5.797886ms, Median: 5.9959ms, Best: 996.3µs, Worst: 84.0007ms Aggregated, split into 299 x 1s time segments: * Fastest: 1830.03 MB/s, 1873.96 obj/s, 1872.00 ops ended/s (1s) * 50% Median: 1735.04 MB/s, 1776.68 obj/s, 1776.00 ops ended/s (1s) * Slowest: 994.94 MB/s, 1018.82 obj/s, 1018.00 ops ended/s (1s) ``` TLDR; `os.Stat` is not free.	5 years ago

... 3 4 5 6 7 ...

2409 Commits (9902c9baaafd2860e937d65469e32d9227d9278f)