minio

Commit Graph

Author	SHA1	Message	Date
Harshavardhana	1dc5f2d0af	Remove safe mode for invalid entries in config (#8650 ) The approach is that now safe mode is only invoked when we cannot read the config or under some catastrophic situations, but not under situations when config entries are invalid or unreachable. This allows for maximum availability for MinIO and not fail on our users unlike most of our historical releases.	5 years ago
Harshavardhana	c10ecacf91	Always use SourceIP for host target filtering (#8649 )	5 years ago
poornas	1cf3e3b7b5	PutBucket: Case-insensitive validation of x-amz-bucket-object-lock-enabled (#8648 ) Fix: case insensitive validation of x-amz-bucket-object-lock-enabled header in PutBucket handler	5 years ago
Andreas Auernhammer	c3d4c1f584	add minio/keys KMS integration (#8631 ) This commit adds support for the minio/kes KMS. See: https://github.com/minio/kes In particular you can configure it as KMS by: - `export MINIO_KMS_KES_ENDPOINT=` // Server URL - `export MINIO_KMS_KES_KEY_FILE=` // TLS client private key - `export MINIO_KMS_KES_CERT_FILE=` // TLS client certificate - `export MINIO_KMS_KES_CA_PATH=` // Root CAs issuing server cert - `export MINIO_KMS_KES_KEY_NAME=` // The name of the (default) master key	5 years ago
Harshavardhana	471a3a650a	fix: Don't allow to set unconfigured notification ARNs (#8643 ) Fixes #8642	5 years ago
Harshavardhana	cc02bf0442	Remove old ListenBucketNotification API (#8645 )	5 years ago
Harshavardhana	39e8e4f4aa	Allow empty target KVS for notification targets (#8644 ) This is allowed with enable=off arg value	5 years ago
poornas	80558e839d	Clear cache if reverting to backend (#8637 ) Clear cached entry before reverting to backend for encrypted objects or those under retention to avoid stale objects remaining in cache.	5 years ago
Harshavardhana	ca62ac65d4	Reject mandatory KVS if not set for any sub-sys (#8641 )	5 years ago
Harshavardhana	f5abe4e1f1	Support ListenBucketNotificationV2 streaming (#8622 )	5 years ago
Klaus Post	3211cb5df6	Add encryption buffer (#8626 ) Quite hard to measure difference: ``` λ warp cmp put-before.csv.zst put-after2.csv.zst Operation: PUT Operations: 340 -> 353 * Average: +4.11% (+22.7 MB/s) throughput, +4.11% (+0.2) obj/s * 50% Median: +1.58% (+7.3 MB/s) throughput, +1.58% (+0.1) obj/s ``` Difference is likely bigger on Intel platforms due to higher syscall costs.	5 years ago
Ashish Kumar Sinha	abc266caa1	Add bucket and object count along with total object size (#8639 )	5 years ago
Harshavardhana	c364f0af6c	Start using custom HTTP transport for webhook endpoints (#8630 ) Use a more performant http transport for webhook endpoints with proper connection pooling, appropriate timeouts etc.	5 years ago
Anis Elleuch	555969ee42	Add data usage collect with its new admin API (#8553 ) Admin data usage info API returns the following (Only FS & XL, for now) - Number of buckets - Number of objects - The total size of objects - Objects histogram - Bucket sizes	5 years ago
Ashish Kumar Sinha	e2c5d29017	Bucket,Object count & Usage removed if set to default (#8638 )	5 years ago
Harshavardhana	fa00a84709	Avoid crashes on peers if IAMSys is not initialized (#8636 )	5 years ago
kannappanr	d266b3a066	Admin Info: Modify Uptime to return seconds (#8635 )	5 years ago
Leletir	db3f41fcb4	Doc: change url for Total Population CSV (#8633 )	5 years ago
Ashish Kumar Sinha	24fb1bf258	New Admin Info (#8497 )	5 years ago
Harshavardhana	8b803491af	fix: CacheOpts parsing tests (#8632 )	5 years ago
Harshavardhana	10b2f15f6f	Add randomize sleep times for lock checkers (#8628 )	5 years ago
Harshavardhana	3e9ab5f4a9	Fix k8s replica set deployment (#8629 ) In replica sets, hosts resolve to localhost IP automatically until the deployment fully comes up. To avoid this issue we need to wait for such resolution.	5 years ago
brnbs	46b4dd8e20	Add command to list added lifecycle configurations (#8627 )	5 years ago
Klaus Post	bf3a97d3aa	S3 Select: Concurrent LINES delimited json parsing (#8610 ) The speedup is ~5x on a 6 core CPU	5 years ago
Krishna Srinivas	3b67f629a4	Retry peer notification of events (#8621 )	5 years ago
poornas	3c30e4503d	Cache only the range requested for range GETs (#8599 )	5 years ago
poornas	8390bc26db	Fix cache hit metrics. (#8617 )	5 years ago
Nitish Tiwari	24ad59316d	Use atomic.Uint64 for gateway metrics count instead of mutex (#8615 )	5 years ago
poornas	be0c8b1ec0	Add support for missing Cache-Control directives (#8619 ) no-cache, only-if-cached and no-store directives are being enforced in this PR.	5 years ago
Harshavardhana	476111968a	Update help messages with new wording (#8616 ) Final update to all messages across sub-systems after final review, the only change here is that NATS now has TLS and TLSSkipVerify to be consistent for all other notification targets.	5 years ago
Klaus Post	f1e2e1cc9e	S3 Select: Mismatched types don't match (#8608 ) When comparing for equality, if types cannot be matched, they don't match.	5 years ago
Harshavardhana	97deba2a7c	GetKVS should add new keys automatically, preserve order (#8612 )	5 years ago
Nitish Tiwari	3df7285c3c	Add Support for Cache and S3 related metrics in Prometheus endpoint (#8591 ) This PR adds support below metrics - Cache Hit Count - Cache Miss Count - Data served from Cache (in Bytes) - Bytes received from AWS S3 - Bytes sent to AWS S3 - Number of requests sent to AWS S3 Fixes #8549	5 years ago
Aleksandr Petruhin	d2dc964cb5	Support TLS auth for Kafka notification target (#8609 )	5 years ago
Harshavardhana	d8e3de0cae	Ensure comment is always a valid key (#8604 ) Also fix LDAP leaky connection	5 years ago
Harshavardhana	c9940d8c3f	Final changes to config sub-system (#8600 ) - Introduces changes such as certain types of errors that can be ignored or which need to go into safe mode. - Update help text as per the review	5 years ago
Harshavardhana	794eb54da8	Export command prints turned-off sub-sys as comments (#8594 ) This PR also tries to - Preserve the order of keys printed in export command - Fix cache to be enabled with _STATE env to keep backward compatibility	5 years ago
Harshavardhana	2ab8d5e47f	Enable build verification with race (#8583 )	5 years ago
Clemens Wolff	947bc8c7d3	Update Azure Gateway to azure-storage-blob SDK (#8537 ) The azure-sdk-for-go/storage package has been in maintenance- only mode since February 2018 (see [1]) and will be deprecated in the future.	5 years ago
Harshavardhana	5d3d57c12a	Start using error wrapping with fmt.Errorf (#8588 ) Use fatih/errwrap to fix all the code to use error wrapping with fmt.Errorf()	5 years ago
Harshavardhana	0bfd20a8e3	Add client_id support for OpenID (#8579 ) - One click OpenID authorization on Login page - Add client_id help, config keys etc Thanks to @egorkaru @ihostage for the original work and testing.	5 years ago
Klaus Post	db3dbcce3a	Print goroutines when shutdown hangs (#8574 )	5 years ago
Harshavardhana	b21835f195	Honor DurationSeconds properly for WebIdentity (#8581 ) Also cleanup code to add various constants for verbatim strings across the code base. Fixes #8482	5 years ago
Klaus Post	c7844fb1fb	posix: cache disk ID for a short while (#8564 ) `posix.getDiskID()` takes up to 30% of all CPU due to the `os.Stat` call on `GET` calls. Before: ``` Operation: GET - Concurrency: 12 Average: 1333.97 MB/s, 1365.99 obj/s, 1365.98 ops ended/s (4m59.975s) * First Byte: Average: 7.801487ms, Median: 7.9974ms, Best: 1.9822ms, Worst: 110.0021ms Aggregated, split into 299 x 1s time segments: * Fastest: 1453.50 MB/s, 1488.38 obj/s, 1492.00 ops ended/s (1s) * 50% Median: 1360.47 MB/s, 1393.12 obj/s, 1393.00 ops ended/s (1s) * Slowest: 978.68 MB/s, 1002.17 obj/s, 1004.00 ops ended/s (1s) ``` After: ``` Operation: GET - Concurrency: 12 * Average: 1706.07 MB/s, 1747.02 obj/s, 1747.01 ops ended/s (4m59.985s) * First Byte: Average: 5.797886ms, Median: 5.9959ms, Best: 996.3µs, Worst: 84.0007ms Aggregated, split into 299 x 1s time segments: * Fastest: 1830.03 MB/s, 1873.96 obj/s, 1872.00 ops ended/s (1s) * 50% Median: 1735.04 MB/s, 1776.68 obj/s, 1776.00 ops ended/s (1s) * Slowest: 994.94 MB/s, 1018.82 obj/s, 1018.00 ops ended/s (1s) ``` TLDR; `os.Stat` is not free.	5 years ago
Harshavardhana	2ff8132e2d	Fix the regression introduced in #8580	5 years ago
Harshavardhana	30e80d0a86	Add ReadFrom,WriteTo helpers for server config (#8580 )	5 years ago
Klaus Post	87443af49e	Cache Windows mount point info (#8572 ) It seems like every upload creates a new StorageInfo object. This makes it very sensitive to slow code paths. A particular slow function is `IsLikelyMountPoint` on Windows since `windows.GetVolumePathName` is slow. We cache the result of this operation. Before: ``` Operation: PUT - Concurrency: 12 * Average: 70.46 MB/s, 70.46 obj/s, 70.46 ops ended/s (59.58s) Aggregated, split into 59 x 1s time segments: * Fastest: 77.67 MB/s, 77.67 obj/s, 79.00 ops ended/s (1s) * 50% Median: 70.84 MB/s, 70.84 obj/s, 70.00 ops ended/s (1s) * Slowest: 61.51 MB/s, 61.51 obj/s, 61.00 ops ended/s (1s) ``` After: ``` Operation: PUT - Concurrency: 12 * Average: 166.80 MB/s, 166.80 obj/s, 166.74 ops ended/s (59.8s) Aggregated, split into 59 x 1s time segments: * Fastest: 183.83 MB/s, 183.83 obj/s, 186.00 ops ended/s (1s) * 50% Median: 167.47 MB/s, 167.47 obj/s, 166.00 ops ended/s (1s) * Slowest: 139.30 MB/s, 139.30 obj/s, 141.00 ops ended/s (1s) ```	5 years ago
Harshavardhana	5d65428b29	Handle localhost distributed setups properly (#8577 ) Fixes an issue reported by @klauspost and @vadmeste This PR also allows users to expand their clusters from single node XL deployment to distributed mode.	5 years ago
Harshavardhana	78eb3b78bb	Repurpose Get/SetConfig as import/export support (#8578 )	5 years ago
Harshavardhana	720442b1a2	Add lock expiry handler to expire state locks (#8562 )	5 years ago

1 2 3 4 5 ...

6229 Commits (1dc5f2d0afc61be72faa43f2cad60dad01c024b8) All Branches Search

6229 Commits (1dc5f2d0afc61be72faa43f2cad60dad01c024b8)

All Branches