minio

Commit Graph

Author	SHA1	Message	Date
Harshavardhana	5afb1b6747	Add support for {jwt:sub} substitutions for policies (#8393 ) Fixes #8345	5 years ago
Harshavardhana	f2cc97a44c	Improve MQTT token registration retry (#8397 )	5 years ago
Harshavardhana	d48fd6fde9	Remove unusued params and functions (#8399 )	5 years ago
Andreas Auernhammer	b7ee0bbbc9	madmin: Migrate to provable secure channel construction (#8395 ) This commit replaces the currently used channel construction for en/decrypting config data with a provable secure scheme.	5 years ago
Harshavardhana	c33bae057f	Fix build issues with event target logger	5 years ago
poornas	d7060c4c32	Allow logging targets to be configured to receive `minio` (#8347 ) specific errors, `application` errors or `all` by default. console logging on server by default lists all logs - enhance admin console API to accept `type` as query parameter to subscribe to application/minio logs.	5 years ago
Harshavardhana	bd10640846	Fix logger and audit http endpoint config lookup	5 years ago
Harshavardhana	175b07d6e4	Fix queueStore stops working with concurrent PUT/DELETE requests (#8381 ) - This PR fixes situation to avoid underflow, this is possible because of disconnected operations in replay/sendEvents - Hold right locks if Del() operation is performed in Get() - Remove panic in the code and use loggerOnce - Remove Timer and instead use Ticker instead for proper ticks	5 years ago
Harshavardhana	f16df2a4e7	Avoid crash in PopulatePublicKey() by re-initializing the args (#8388 ) This is to avoid nil pointer dereference when method by pointer reference and method by value reference are implemented. Fixes #8387	5 years ago
Harshavardhana	290ad0996f	Move etcd, logger, crypto into their own packages (#8366 ) - Deprecates _MINIO_PROFILER, `mc admin profile` does the job - Move ENVs to common location in cmd/config/	5 years ago
Harshavardhana	3b8adf7528	Move storageclass config handling into cmd/config/storageclass (#8360 ) Continuation of the changes done in PR #8351 to refactor, add tests and move global handling into a more idiomatic style for Go as packages.	5 years ago
Klaus Post	002ac82631	S3 Select: Add parser support for lists. (#8329 )	5 years ago
Harshavardhana	589e32a4ed	Refactor config and split them in packages (#8351 ) This change is related to larger config migration PR change, this is a first stage change to move our configs to `cmd/config/` - divided into its subsystems	5 years ago
Ashish Kumar Sinha	74008446fe	CPU hardware info (#8187 )	5 years ago
Harshavardhana	fb1374f2f7	Rename iam/validator -> iam/openid and add tests (#8340 ) Refactor as part of config migration	5 years ago
Klaus Post	c1a17c2561	S3 Select: Aggregate AVG/SUM as float (#8326 ) Force sum/average to be calculated as a float. As noted in #8221 > run SELECT AVG(CAST (Score as int)) FROM S3Object on ``` Name,Score alice,80 bob,81 ``` > AWS S3 gives 80.5 and MinIO gives 80. This also makes overflows much more unlikely.	5 years ago
Klaus Post	1c5b05c130	S3 select: Fix output conversion on select * (#8303 ) Fixes #8268	5 years ago
Bala FA	2a2ff96ee1	change `ReadPerf` into `ReadThroughput` in NetPerfInfo. (#8316 ) Previously `ReadPerf` was in time.Duration is changed to `ReadThroughput` in uint64.	5 years ago
Harshavardhana	fd53057654	Add InfoCannedPolicy API to fetch only necessary policy (#8307 ) This PR adds - InfoCannedPolicy() API for efficiency in fetching policies - Send group memberships for LDAPUser if available	5 years ago
Klaus Post	be313f1758	S3 Select: Workaround java buffer size (#8312 ) Updates #7475 The Java implementation has a 128KB buffer and a message must be emitted before that is used. #7475 therefore limits the message size to 128KB. But up to 256 bytes are written to the buffer in each call. This means we must emit a message before shorter than 128KB. Therefore we change the limit to 128KB minus 256 bytes.	5 years ago
poornas	2e02e1889b	Cleanup ResponseWriter function for audit and trace (#8283 )	5 years ago
poornas	4925bc3e80	log server startup messages to admin console api (#8264 )	5 years ago
Klaus Post	520552ffa9	S3 select: flush when reaching limit (#8279 ) Add missing flush when reaching select limit.	5 years ago
Praveen raj Mani	dc0dce9beb	Check for connection errors during event transit in elasticsearch and webhook (#8273 )	5 years ago
Praveen raj Mani	456ce4cc92	Add rootCAs support to Kafka & MQTT (#8236 ) Fixes #8211	5 years ago
Klaus Post	dac1cf5a9a	S3 Select: Parsing tweaks (#8261 ) * Don't output empty lines. * Trim whitespace from byte to int/float/bool conversions.	5 years ago
Klaus Post	c9b8bd8de2	S3 Select: optimize output (#8238 ) Queue output items and reuse them. Remove the unneeded type system in sql and just use the Go type system. In best case this is more than an order of magnitude speedup: ``` BenchmarkSelectAll_1M-12 1 1841049400 ns/op 274299728 B/op 4198522 allocs/op BenchmarkSelectAll_1M-12 14 84833400 ns/op 169228346 B/op 3146541 allocs/op ```	5 years ago
Klaus Post	017456df63	Wait clearing the close channel (#8250 ) Close channel should not be nilled before goroutines have exited. Fixes potential hang on closing.	5 years ago
Harshavardhana	9ac12cf898	Remove unusued Set/GetConfigKeys API (#8235 )	5 years ago
Klaus Post	ddea0bdf11	Concurrent CSV parsing and reduce S3 select allocations (#8200 ) ``` CSV parsing, BEFORE: BenchmarkReaderBasic-12 2842 407533 ns/op 397860 B/op 957 allocs/op BenchmarkReaderReplace-12 2718 429914 ns/op 397844 B/op 957 allocs/op BenchmarkReaderReplaceTwo-12 2718 435556 ns/op 397855 B/op 957 allocs/op BenchmarkAggregateCount_100K-12 171 6798974 ns/op 16667102 B/op 308077 allocs/op BenchmarkAggregateCount_1M-12 19 65657411 ns/op 168057743 B/op 3146610 allocs/op BenchmarkSelectAll_10M-12 1 20882119900 ns/op 2758799896 B/op 41978762 allocs/op CSV parsing, AFTER: BenchmarkReaderBasic-12 3721 312549 ns/op 101920 B/op 338 allocs/op BenchmarkReaderReplace-12 3776 318810 ns/op 101993 B/op 340 allocs/op BenchmarkReaderReplaceTwo-12 3610 330967 ns/op 102012 B/op 341 allocs/op BenchmarkAggregateCount_100K-12 295 4149588 ns/op 3553623 B/op 103261 allocs/op BenchmarkAggregateCount_1M-12 30 37746503 ns/op 33827931 B/op 1049435 allocs/op BenchmarkSelectAll_10M-12 1 17608495800 ns/op 1416504040 B/op 21007082 allocs/op ~ benchcmp old.txt new.txt benchmark old ns/op new ns/op delta BenchmarkReaderBasic-12 407533 312549 -23.31% BenchmarkReaderReplace-12 429914 318810 -25.84% BenchmarkReaderReplaceTwo-12 435556 330967 -24.01% BenchmarkAggregateCount_100K-12 6798974 4149588 -38.97% BenchmarkAggregateCount_1M-12 65657411 37746503 -42.51% BenchmarkSelectAll_10M-12 20882119900 17608495800 -15.68% benchmark old allocs new allocs delta BenchmarkReaderBasic-12 957 338 -64.68% BenchmarkReaderReplace-12 957 340 -64.47% BenchmarkReaderReplaceTwo-12 957 341 -64.37% BenchmarkAggregateCount_100K-12 308077 103261 -66.48% BenchmarkAggregateCount_1M-12 3146610 1049435 -66.65% BenchmarkSelectAll_10M-12 41978762 21007082 -49.96% benchmark old bytes new bytes delta BenchmarkReaderBasic-12 397860 101920 -74.38% BenchmarkReaderReplace-12 397844 101993 -74.36% BenchmarkReaderReplaceTwo-12 397855 102012 -74.36% BenchmarkAggregateCount_100K-12 16667102 3553623 -78.68% BenchmarkAggregateCount_1M-12 168057743 33827931 -79.87% BenchmarkSelectAll_10M-12 2758799896 1416504040 -48.66% ``` ``` BenchmarkReaderHuge/97K-12 2200 540840 ns/op 184.32 MB/s 1604450 B/op 687 allocs/op BenchmarkReaderHuge/194K-12 1522 752257 ns/op 265.04 MB/s 2143135 B/op 1335 allocs/op BenchmarkReaderHuge/389K-12 1190 947858 ns/op 420.69 MB/s 3221831 B/op 2630 allocs/op BenchmarkReaderHuge/778K-12 806 1472486 ns/op 541.61 MB/s 5201856 B/op 5187 allocs/op BenchmarkReaderHuge/1557K-12 426 2575269 ns/op 619.36 MB/s 9101330 B/op 10233 allocs/op BenchmarkReaderHuge/3115K-12 286 4034656 ns/op 790.66 MB/s 12397968 B/op 16099 allocs/op BenchmarkReaderHuge/6230K-12 172 6830563 ns/op 934.05 MB/s 16008416 B/op 26844 allocs/op BenchmarkReaderHuge/12461K-12 100 11409467 ns/op 1118.39 MB/s 22655163 B/op 48107 allocs/op BenchmarkReaderHuge/24922K-12 66 19780395 ns/op 1290.19 MB/s 35158559 B/op 90216 allocs/op BenchmarkReaderHuge/49844K-12 34 37282559 ns/op 1369.03 MB/s 60528624 B/op 174497 allocs/op ```	5 years ago
Praveen raj Mani	8700945cdf	Handle connection failures on webhook/url pings (#8204 ) Properly handle connection failures while replaying events Fixes #8194	5 years ago
Krishnan Parthasarathi	6ba323b009	Add ability to test drive speeds on a MinIO setup (#7664 ) - Extends existing Admin API to measure disk performance	5 years ago
Harshavardhana	53e4887e02	Simplify and cleanup metadata r/w functions (#8146 )	5 years ago
Harshavardhana	a7be313230	Start using new errors package (#8207 )	5 years ago
Yao Zongyou	18fedc67d5	friendly prompt for s3select MalformedXML error (#8171 ) partly fix #7911	5 years ago
Harshavardhana	b52a3e523c	Avoid using fastjson parser pool, move back to jsoniter (#8190 ) It looks like from implementation point of view fastjson parser pool doesn't behave the same way as expected when dealing many `xl.json` from multiple disks. The fastjson parser pool usage ends up returning incorrect xl.json entries for checksums, with references pointing to older entries. This led to the subtle bug where checksum info is duplicated from a previous xl.json read of a different file from different disk.	5 years ago
Praveen raj Mani	428836d4e1	Ignore "connection reset" errors while initializing the targets with queue store enabled (#8185 ) Fixes #8178	5 years ago
Andreas Auernhammer	810a44e951	KMS Admin-API: add route and handler for KMS key info (#7955 ) This commit adds an admin API route and handler for requesting status information about a KMS key. Therefore, the client specifies the KMS key ID (when empty / not set the server takes the currently configured default key-ID) and the server tries to perform a dummy encryption, re-wrap and decryption operation. If all three succeed we know that the server can access the KMS and has permissions to generate, re-wrap and decrypt data keys (policy is set correctly).	5 years ago
poornas	8a71b0ec5a	Add admin API to send console log messages (#7784 ) Utilized by mc admin console command.	5 years ago
Yao Zongyou	ec9bfd3aef	speed up the performance of s3select on csv (#7945 )	5 years ago
Bala FA	fa3546bb03	Add NetPerfInfo() API in madmin (#8112 )	5 years ago
Harshavardhana	83d4c5763c	Decouple ServiceUpdate to ServerUpdate to be more native (#8138 ) The change now is to ensure that we take custom URL as well for updating the deployment, this is required for hotfix deliveries for certain deployments - other than the community release. This commit changes the previous work `d65a2c6725` with newer set of requirements. Also deprecates PeerUptime()	5 years ago
Ashish Kumar Sinha	c6f86d35d3	Rectify Examples and README.MD (#8141 )	5 years ago
Harshavardhana	d65a2c6725	Implement cluster-wide in-place updates (#8070 ) This PR is a breaking change and also deprecates `minio update` command, from this release onwards all users are advised to just use `mc admin update`	5 years ago
Krishna Srinivas	c38ada1a26	write() to disk in 4MB blocks for better performance (#7888 )	5 years ago
poornas	7bf1caa0fe	Fix broken link to go install docs (#8090 )	5 years ago
Aditya Manthramurthy	bf9b619d86	Set the policy mapping for a user or group (#8036 ) Add API to set policy mapping for a user or group Contains a breaking Admin APIs change. - Also enforce all applicable policies - Removes the previous /set-user-policy API Bump up peerRESTVersion Add get user info API to show groups of a user	5 years ago
maihde	0ed6daab59	fix: #8051 so that stale DNS entries are cleaned-up (#8053 )	5 years ago
Praveen raj Mani	e96f19c867	Use default ports for the schemes when dialing (#7960 ) Fixes #7946	5 years ago
Harshavardhana	aaef18b1a3	Fail for disks which overflow upon usage calculation (#8056 ) Fixes #8035	5 years ago

... 2 3 4 5 6 ...

1473 Commits (83ccae6c8b11434d4cc5126f4967905fa2c89845)