Harshavardhana
5f2318567e
Allow metadata updates on meta bucket even in WORM mode ( #8657 )
...
This ensures that we can update the
- .minio.sys is updated for accounting/data usage purposes
- .minio.sys is updated to indicate if backend is encrypted
or not.
5 years ago
kannappanr
16ac4a3c64
PutBucketLifeCycleConfiguration: Return 200 instead of 204 ( #8656 )
5 years ago
Harshavardhana
c8d82588c2
Fix crash in console logger and also handle bucket DNS updates ( #8654 )
...
Also fix listenBucketNotification bugs seen by minio-js
listen bucket notification API.
5 years ago
Praveen raj Mani
842d0241ed
Fix aws-sdk-php functional test cases for GCS gateway ( #8613 )
...
Fixes #8570
5 years ago
Harshavardhana
1dc5f2d0af
Remove safe mode for invalid entries in config ( #8650 )
...
The approach is that now safe mode is only invoked when
we cannot read the config or under some catastrophic
situations, but not under situations when config entries
are invalid or unreachable. This allows for maximum
availability for MinIO and not fail on our users unlike
most of our historical releases.
5 years ago
Harshavardhana
c10ecacf91
Always use SourceIP for host target filtering ( #8649 )
5 years ago
poornas
1cf3e3b7b5
PutBucket: Case-insensitive validation of x-amz-bucket-object-lock-enabled ( #8648 )
...
Fix: case insensitive validation of x-amz-bucket-object-lock-enabled header in PutBucket handler
5 years ago
Andreas Auernhammer
c3d4c1f584
add minio/keys KMS integration ( #8631 )
...
This commit adds support for the minio/kes KMS.
See: https://github.com/minio/kes
In particular you can configure it as KMS by:
- `export MINIO_KMS_KES_ENDPOINT=` // Server URL
- `export MINIO_KMS_KES_KEY_FILE=` // TLS client private key
- `export MINIO_KMS_KES_CERT_FILE=` // TLS client certificate
- `export MINIO_KMS_KES_CA_PATH=` // Root CAs issuing server cert
- `export MINIO_KMS_KES_KEY_NAME=` // The name of the (default)
master key
5 years ago
Harshavardhana
471a3a650a
fix: Don't allow to set unconfigured notification ARNs ( #8643 )
...
Fixes #8642
5 years ago
Harshavardhana
cc02bf0442
Remove old ListenBucketNotification API ( #8645 )
5 years ago
Harshavardhana
39e8e4f4aa
Allow empty target KVS for notification targets ( #8644 )
...
This is allowed with enable=off arg value
5 years ago
poornas
80558e839d
Clear cache if reverting to backend ( #8637 )
...
Clear cached entry before reverting to backend for
encrypted objects or those under retention to avoid
stale objects remaining in cache.
5 years ago
Harshavardhana
ca62ac65d4
Reject mandatory KVS if not set for any sub-sys ( #8641 )
5 years ago
Harshavardhana
f5abe4e1f1
Support ListenBucketNotificationV2 streaming ( #8622 )
5 years ago
Klaus Post
3211cb5df6
Add encryption buffer ( #8626 )
...
Quite hard to measure difference:
```
λ warp cmp put-before.csv.zst put-after2.csv.zst
Operation: PUT
Operations: 340 -> 353
* Average: +4.11% (+22.7 MB/s) throughput, +4.11% (+0.2) obj/s
* 50% Median: +1.58% (+7.3 MB/s) throughput, +1.58% (+0.1) obj/s
```
Difference is likely bigger on Intel platforms due to higher syscall costs.
5 years ago
Ashish Kumar Sinha
abc266caa1
Add bucket and object count along with total object size ( #8639 )
5 years ago
Harshavardhana
c364f0af6c
Start using custom HTTP transport for webhook endpoints ( #8630 )
...
Use a more performant http transport for webhook
endpoints with proper connection pooling, appropriate
timeouts etc.
5 years ago
Anis Elleuch
555969ee42
Add data usage collect with its new admin API ( #8553 )
...
Admin data usage info API returns the following
(Only FS & XL, for now)
- Number of buckets
- Number of objects
- The total size of objects
- Objects histogram
- Bucket sizes
5 years ago
Ashish Kumar Sinha
e2c5d29017
Bucket,Object count & Usage removed if set to default ( #8638 )
5 years ago
Harshavardhana
fa00a84709
Avoid crashes on peers if IAMSys is not initialized ( #8636 )
5 years ago
kannappanr
d266b3a066
Admin Info: Modify Uptime to return seconds ( #8635 )
5 years ago
Leletir
db3f41fcb4
Doc: change url for Total Population CSV ( #8633 )
5 years ago
Ashish Kumar Sinha
24fb1bf258
New Admin Info ( #8497 )
5 years ago
Harshavardhana
8b803491af
fix: CacheOpts parsing tests ( #8632 )
5 years ago
Harshavardhana
10b2f15f6f
Add randomize sleep times for lock checkers ( #8628 )
5 years ago
Harshavardhana
3e9ab5f4a9
Fix k8s replica set deployment ( #8629 )
...
In replica sets, hosts resolve to localhost
IP automatically until the deployment fully
comes up. To avoid this issue we need to
wait for such resolution.
5 years ago
brnbs
46b4dd8e20
Add command to list added lifecycle configurations ( #8627 )
5 years ago
Klaus Post
bf3a97d3aa
S3 Select: Concurrent LINES delimited json parsing ( #8610 )
...
The speedup is ~5x on a 6 core CPU
5 years ago
Krishna Srinivas
3b67f629a4
Retry peer notification of events ( #8621 )
5 years ago
poornas
3c30e4503d
Cache only the range requested for range GETs ( #8599 )
5 years ago
poornas
8390bc26db
Fix cache hit metrics. ( #8617 )
5 years ago
Nitish Tiwari
24ad59316d
Use atomic.Uint64 for gateway metrics count instead of mutex ( #8615 )
5 years ago
poornas
be0c8b1ec0
Add support for missing Cache-Control directives ( #8619 )
...
no-cache, only-if-cached and no-store directives are
being enforced in this PR.
5 years ago
Harshavardhana
476111968a
Update help messages with new wording ( #8616 )
...
Final update to all messages across sub-systems
after final review, the only change here is that
NATS now has TLS and TLSSkipVerify to be consistent
for all other notification targets.
5 years ago
Klaus Post
f1e2e1cc9e
S3 Select: Mismatched types don't match ( #8608 )
...
When comparing for equality, if types cannot be matched, they don't match.
5 years ago
Harshavardhana
97deba2a7c
GetKVS should add new keys automatically, preserve order ( #8612 )
5 years ago
Nitish Tiwari
3df7285c3c
Add Support for Cache and S3 related metrics in Prometheus endpoint ( #8591 )
...
This PR adds support below metrics
- Cache Hit Count
- Cache Miss Count
- Data served from Cache (in Bytes)
- Bytes received from AWS S3
- Bytes sent to AWS S3
- Number of requests sent to AWS S3
Fixes #8549
5 years ago
Aleksandr Petruhin
d2dc964cb5
Support TLS auth for Kafka notification target ( #8609 )
5 years ago
Harshavardhana
d8e3de0cae
Ensure comment is always a valid key ( #8604 )
...
Also fix LDAP leaky connection
5 years ago
Harshavardhana
c9940d8c3f
Final changes to config sub-system ( #8600 )
...
- Introduces changes such as certain types of
errors that can be ignored or which need to
go into safe mode.
- Update help text as per the review
5 years ago
Harshavardhana
794eb54da8
Export command prints turned-off sub-sys as comments ( #8594 )
...
This PR also tries to
- Preserve the order of keys printed in export command
- Fix cache to be enabled with _STATE env to keep
backward compatibility
5 years ago
Harshavardhana
2ab8d5e47f
Enable build verification with race ( #8583 )
5 years ago
Clemens Wolff
947bc8c7d3
Update Azure Gateway to azure-storage-blob SDK ( #8537 )
...
The azure-sdk-for-go/storage package has been in maintenance-
only mode since February 2018 (see [1]) and will be deprecated in the future.
5 years ago
Harshavardhana
5d3d57c12a
Start using error wrapping with fmt.Errorf ( #8588 )
...
Use fatih/errwrap to fix all the code to use
error wrapping with fmt.Errorf()
5 years ago
Harshavardhana
0bfd20a8e3
Add client_id support for OpenID ( #8579 )
...
- One click OpenID authorization on Login page
- Add client_id help, config keys etc
Thanks to @egorkaru @ihostage for the
original work and testing.
5 years ago
Klaus Post
db3dbcce3a
Print goroutines when shutdown hangs ( #8574 )
5 years ago
Harshavardhana
b21835f195
Honor DurationSeconds properly for WebIdentity ( #8581 )
...
Also cleanup code to add various constants for
verbatim strings across the code base.
Fixes #8482
5 years ago
Klaus Post
c7844fb1fb
posix: cache disk ID for a short while ( #8564 )
...
`*posix.getDiskID()` takes up to 30% of all CPU due to the `os.Stat` call on `GET` calls.
Before:
```
Operation: GET - Concurrency: 12
* Average: 1333.97 MB/s, 1365.99 obj/s, 1365.98 ops ended/s (4m59.975s)
* First Byte: Average: 7.801487ms, Median: 7.9974ms, Best: 1.9822ms, Worst: 110.0021ms
Aggregated, split into 299 x 1s time segments:
* Fastest: 1453.50 MB/s, 1488.38 obj/s, 1492.00 ops ended/s (1s)
* 50% Median: 1360.47 MB/s, 1393.12 obj/s, 1393.00 ops ended/s (1s)
* Slowest: 978.68 MB/s, 1002.17 obj/s, 1004.00 ops ended/s (1s)
```
After:
```
Operation: GET - Concurrency: 12
* Average: 1706.07 MB/s, 1747.02 obj/s, 1747.01 ops ended/s (4m59.985s)
* First Byte: Average: 5.797886ms, Median: 5.9959ms, Best: 996.3µs, Worst: 84.0007ms
Aggregated, split into 299 x 1s time segments:
* Fastest: 1830.03 MB/s, 1873.96 obj/s, 1872.00 ops ended/s (1s)
* 50% Median: 1735.04 MB/s, 1776.68 obj/s, 1776.00 ops ended/s (1s)
* Slowest: 994.94 MB/s, 1018.82 obj/s, 1018.00 ops ended/s (1s)
```
TLDR; `os.Stat` is not free.
5 years ago
Harshavardhana
2ff8132e2d
Fix the regression introduced in #8580
5 years ago
Harshavardhana
30e80d0a86
Add ReadFrom,WriteTo helpers for server config ( #8580 )
5 years ago