oyd
/
minio
6
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
6504 Commits
1 Branch
0 Tags
89 MiB
Tag: Branch: Tree: 38cf263409
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '38cf263409'
${ noResults }
minio/cmd/api-router.go

324 lines
18 KiB
Raw Normal View History Unescape

routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
/*
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
* MinIO Cloud Storage, (C) 2016-2020 MinIO, Inc.
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
server: Move all the top level files into cmd folder. (#2490) This change brings a change which was done for the 'mc' package to allow for clean repo and have a cleaner github drop in experience.
8 years ago
package cmd
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
7 years ago
import (
"net/http"
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
"strconv"
"time"
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
7 years ago
use package name correctly (#5827)
7 years ago
"github.com/gorilla/mux"
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
"github.com/minio/minio/cmd/config"
Ensure that we use constants everywhere (#7845) This allows for canonicalization of the strings throughout our code and provides a common space for all these constants to reside. This list is rather non-exhaustive but captures all the headers used in AWS S3 API operations
5 years ago
xhttp "github.com/minio/minio/cmd/http"
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
"github.com/minio/minio/cmd/logger"
"github.com/minio/minio/pkg/env"
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
7 years ago
)
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
Enable build verification with race (#8583)
5 years ago
func newHTTPServerFn() *xhttp.Server {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
return globalHTTPServer
}
Bring in safe mode support (#8478) This PR refactors object layer handling such that upon failure in sub-system initialization server reaches a stage of safe-mode operation wherein only certain API operations are enabled and available. This allows for fixing many scenarios such as - incorrect configuration in vault, etcd, notification targets - missing files, incomplete config migrations unable to read encrypted content etc - any other issues related to notification, policies, lifecycle etc
5 years ago
func newObjectLayerWithoutSafeModeFn() ObjectLayer {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
return globalObjectAPI
}
func newObjectLayerFn() ObjectLayer {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
if globalSafeMode {
return nil
}
return globalObjectAPI
}
func newCachedObjectLayerFn() CacheObjectLayer {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
if globalSafeMode {
return nil
}
return globalCacheObjectAPI
}
storage/server/client: Enable storage server, enable client storage.
9 years ago
// objectAPIHandler implements and provides http handlers for S3 API.
type objectAPIHandlers struct {
distribute: Make server work with multiple remote disks This change initializes rpc servers associated with disks that are local. It makes object layer initialization on demand, namely on the first request to the object layer. Also adds lock RPC service vendorized minio/dsync
8 years ago
ObjectAPI func() ObjectLayer
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
7 years ago
CacheAPI func() CacheObjectLayer
Add double encryption at S3 gateway. (#6423) This PR adds pass-through, single encryption at gateway and double encryption support (gateway encryption with pass through of SSE headers to backend). If KMS is set up (either with Vault as KMS or using MINIO_SSE_MASTER_KEY),gateway will automatically perform single encryption. If MINIO_GATEWAY_SSE is set up in addition to Vault KMS, double encryption is performed.When neither KMS nor MINIO_GATEWAY_SSE is set, do a pass through to backend. When double encryption is specified, MINIO_GATEWAY_SSE can be set to "C" for SSE-C encryption at gateway and backend, "S3" for SSE-S3 encryption at gateway/backend or both to support more than one option. Fixes #6323, #6696
6 years ago
// Returns true of handlers should interpret encryption.
EncryptionEnabled func() bool
enable SSE-KMS pass-through on S3 gateway (#7788) This commit relaxes the restriction that the MinIO gateway does not accept SSE-KMS headers. Now, the S3 gateway allows SSE-KMS headers for PUT and MULTIPART PUT requests and forwards them to the S3 gateway backend (AWS). This is considered SSE pass-through mode. Fixes #7753
6 years ago
// Returns true if handlers allow SSE-KMS encryption headers.
AllowSSEKMS func() bool
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
}
// registerAPIRouter - registers S3 compatible APIs.
enable SSE-KMS pass-through on S3 gateway (#7788) This commit relaxes the restriction that the MinIO gateway does not accept SSE-KMS headers. Now, the S3 gateway allows SSE-KMS headers for PUT and MULTIPART PUT requests and forwards them to the S3 gateway backend (AWS). This is considered SSE pass-through mode. Fixes #7753
6 years ago
func registerAPIRouter(router *mux.Router, encryptionEnabled, allowSSEKMS bool) {
server: Add more elaborate startup messages. (#2731) These messages based on our prep stage during XL and prints more informative message regarding drive information. This change also does a much needed refactoring.
8 years ago
// Initialize API.
api := objectAPIHandlers{
Bring in safe mode support (#8478) This PR refactors object layer handling such that upon failure in sub-system initialization server reaches a stage of safe-mode operation wherein only certain API operations are enabled and available. This allows for fixing many scenarios such as - incorrect configuration in vault, etcd, notification targets - missing files, incomplete config migrations unable to read encrypted content etc - any other issues related to notification, policies, lifecycle etc
5 years ago
ObjectAPI: newObjectLayerFn,
CacheAPI: newCachedObjectLayerFn,
Add double encryption at S3 gateway. (#6423) This PR adds pass-through, single encryption at gateway and double encryption support (gateway encryption with pass through of SSE headers to backend). If KMS is set up (either with Vault as KMS or using MINIO_SSE_MASTER_KEY),gateway will automatically perform single encryption. If MINIO_GATEWAY_SSE is set up in addition to Vault KMS, double encryption is performed.When neither KMS nor MINIO_GATEWAY_SSE is set, do a pass through to backend. When double encryption is specified, MINIO_GATEWAY_SSE can be set to "C" for SSE-C encryption at gateway and backend, "S3" for SSE-S3 encryption at gateway/backend or both to support more than one option. Fixes #6323, #6696
6 years ago
EncryptionEnabled: func() bool {
return encryptionEnabled
},
enable SSE-KMS pass-through on S3 gateway (#7788) This commit relaxes the restriction that the MinIO gateway does not accept SSE-KMS headers. Now, the S3 gateway allows SSE-KMS headers for PUT and MULTIPART PUT requests and forwards them to the S3 gateway backend (AWS). This is considered SSE pass-through mode. Fixes #7753
6 years ago
AllowSSEKMS: func() bool {
return allowSSEKMS
},
server: Add more elaborate startup messages. (#2731) These messages based on our prep stage during XL and prints more informative message regarding drive information. This change also does a much needed refactoring.
8 years ago
}
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
// API Router
Use const slashSeparator instead of "/" everywhere (#8028)
5 years ago
apiRouter := router.PathPrefix(SlashSeparator).Subrouter()
use package name correctly (#5827)
7 years ago
var routers []*mux.Router
Support multiple-domains in MINIO_DOMAIN (#7274) Fixes #7173
6 years ago
for _, domainName := range globalDomainNames {
Fix: Handle regression caused by gorilla mux v1.7.0 (#7625) PR #7595 fixed part of the regression, but did not handle the scenario, where in docker, the internal port is different from the port on the host. This PR modifies the regular expression such that all the scenarios are handled. Fixes #7619
6 years ago
routers = append(routers, apiRouter.Host("{bucket:.+}."+domainName).Subrouter())
routers = append(routers, apiRouter.Host("{bucket:.+}."+domainName+":{port:.*}").Subrouter())
Virtual host style S3 requests (#5095)
7 years ago
}
routers = append(routers, apiRouter.PathPrefix("/{bucket}").Subrouter())
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
requestsMax, _ := strconv.Atoi(env.Get(config.EnvAPIRequestsMax, "0"))
if requestsMax < 0 {
requestsMax = 0 // 0 means unlimited
}
var (
enabled bool
requestsMaxPerServer int
requestsDeadline time.Duration
)
if len(globalEndpoints.Hosts()) == 0 {
requestsMaxPerServer = requestsMax
} else {
requestsMaxPerServer = requestsMax / len(globalEndpoints.Hosts())
}
enabled = requestsMaxPerServer > 0
if enabled {
var err error
requestsDeadline, err = time.ParseDuration(env.Get(config.EnvAPIRequestsDeadline, "10s"))
if err != nil {
logger.Info("(%s) parsing environment value MINIO_API_REQUESTS_DEADLINE, defaulting to 10 seconds", err)
requestsDeadline = 10 * time.Second
}
}
requestsMaxCh := make(chan struct{}, requestsMaxPerServer)
Virtual host style S3 requests (#5095)
7 years ago
for _, bucket := range routers {
// Object operations
// HeadObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodHead).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("headobject", httpTraceAll(api.HeadObjectHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
7 years ago
// CopyObjectPart
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HeadersRegexp(xhttp.AmzCopySource, ".*?(\\/|%2F).*?").HandlerFunc(maxClients(collectAPIStats("copyobjectpart", httpTraceAll(api.CopyObjectPartHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("partNumber", "{partNumber:[0-9]+}", "uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
7 years ago
// PutObjectPart
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobjectpart", httpTraceHdrs(api.PutObjectPartHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("partNumber", "{partNumber:[0-9]+}", "uploadId", "{uploadId:.*}")
Add listBucketObjectsVersions implementation (#8093) This API implementation simply behaves like listObjects() but returns back single version for each object, this implementation should be considered dummy it is only meant for some applications which rely on this.
5 years ago
// ListObjectParts
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("listobjectparts", httpTraceAll(api.ListObjectPartsHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
7 years ago
// CompleteMultipartUpload
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("completemutipartupload", httpTraceAll(api.CompleteMultipartUploadHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
7 years ago
// NewMultipartUpload
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("newmultipartupload", httpTraceAll(api.NewMultipartUploadHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("uploads", "")
Virtual host style S3 requests (#5095)
7 years ago
// AbortMultipartUpload
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("abortmultipartupload", httpTraceAll(api.AbortMultipartUploadHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("uploadId", "{uploadId:.*}")
Fix s3cmd issue with ACL handling (#5999) With the implementation of dummy GET ACL handlers, tools like s3cmd perform few operations which causes the ACL call to be invoked. Make sure that in our router configuration GET?acl comes before actual GET call to facilitate this dummy call.
7 years ago
// GetObjectACL - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("getobjectacl", httpTraceHdrs(api.GetObjectACLHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("acl", "")
acl: Support PUT calls with success for 'private' ACL's (#9000) Add dummy calls which respond success when ACL's are set to be private and fails, if user tries to change them from their default 'private' Some applications such as nuxeo may have an unnecessary requirement for this operation, we support this anyways such that don't have to fully implement the functionality just that we can respond with success for default ACLs
5 years ago
// PutObjectACL - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobjectacl", httpTraceHdrs(api.PutObjectACLHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("acl", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
5 years ago
// GetObjectTagging
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("getobjecttagging", httpTraceHdrs(api.GetObjectTaggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("tagging", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
5 years ago
// PutObjectTagging
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobjecttagging", httpTraceHdrs(api.PutObjectTaggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("tagging", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
5 years ago
// DeleteObjectTagging
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("deleteobjecttagging", httpTraceHdrs(api.DeleteObjectTaggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("tagging", "")
S3 Select API Support for CSV (#6127) Add support for trivial where clause cases
6 years ago
// SelectObjectContent
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("selectobjectcontent", httpTraceHdrs(api.SelectObjectContentHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("select", "").Queries("select-type", "2")
Add object retention at the per object (#8528) level - this PR builds on #8120 which added PutBucketObjectLockConfiguration and GetBucketObjectLockConfiguration APIS This PR implements PutObjectRetention, GetObjectRetention API and enhances PUT and GET API operations to display governance metadata if permissions allow.
5 years ago
// GetObjectRetention
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("getobjectretention", httpTraceAll(api.GetObjectRetentionHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("retention", "")
Add support for object locking with legal hold. (#8634)
5 years ago
// GetObjectLegalHold
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("getobjectlegalhold", httpTraceAll(api.GetObjectLegalHoldHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("legal-hold", "")
Virtual host style S3 requests (#5095)
7 years ago
// GetObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("getobject", httpTraceHdrs(api.GetObjectHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
7 years ago
// CopyObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HeadersRegexp(xhttp.AmzCopySource, ".*?(\\/|%2F).*?").HandlerFunc(maxClients(collectAPIStats("copyobject", httpTraceAll(api.CopyObjectHandler)), enabled, requestsMaxCh, requestsDeadline))
Add object retention at the per object (#8528) level - this PR builds on #8120 which added PutBucketObjectLockConfiguration and GetBucketObjectLockConfiguration APIS This PR implements PutObjectRetention, GetObjectRetention API and enhances PUT and GET API operations to display governance metadata if permissions allow.
5 years ago
// PutObjectRetention
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobjectretention", httpTraceAll(api.PutObjectRetentionHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("retention", "")
Add support for object locking with legal hold. (#8634)
5 years ago
// PutObjectLegalHold
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobjectlegalhold", httpTraceAll(api.PutObjectLegalHoldHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("legal-hold", "")
Add support for object locking with legal hold. (#8634)
5 years ago
Virtual host style S3 requests (#5095)
7 years ago
// PutObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobject", httpTraceHdrs(api.PutObjectHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
7 years ago
// DeleteObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("deleteobject", httpTraceAll(api.DeleteObjectHandler)), enabled, requestsMaxCh, requestsDeadline))
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
Virtual host style S3 requests (#5095)
7 years ago
/// Bucket operations
// GetBucketLocation
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketlocation", httpTraceAll(api.GetBucketLocationHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("location", "")
Virtual host style S3 requests (#5095)
7 years ago
// GetBucketPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketpolicy", httpTraceAll(api.GetBucketPolicyHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("policy", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
5 years ago
// GetBucketLifecycle
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketlifecycle", httpTraceAll(api.GetBucketLifecycleHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("lifecycle", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
5 years ago
// GetBucketEncryption
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketencryption", httpTraceAll(api.GetBucketEncryptionHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("encryption", "")
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
7 years ago
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// Dummy Bucket Calls
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
7 years ago
// GetBucketACL -- this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketacl", httpTraceAll(api.GetBucketACLHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("acl", "")
acl: Support PUT calls with success for 'private' ACL's (#9000) Add dummy calls which respond success when ACL's are set to be private and fails, if user tries to change them from their default 'private' Some applications such as nuxeo may have an unnecessary requirement for this operation, we support this anyways such that don't have to fully implement the functionality just that we can respond with success for default ACLs
5 years ago
// PutBucketACL -- this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketacl", httpTraceAll(api.PutBucketACLHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("acl", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketCors - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketcors", httpTraceAll(api.GetBucketCorsHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("cors", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketWebsiteHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketwebsite", httpTraceAll(api.GetBucketWebsiteHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("website", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketAccelerateHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketaccelerate", httpTraceAll(api.GetBucketAccelerateHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("accelerate", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketRequestPaymentHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketrequestpayment", httpTraceAll(api.GetBucketRequestPaymentHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("requestPayment", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketLoggingHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketlogging", httpTraceAll(api.GetBucketLoggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("logging", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketLifecycleHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketlifecycle", httpTraceAll(api.GetBucketLifecycleHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("lifecycle", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketReplicationHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketreplication", httpTraceAll(api.GetBucketReplicationHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("replication", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketTaggingHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbuckettagging", httpTraceAll(api.GetBucketTaggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("tagging", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
//DeleteBucketWebsiteHandler
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucketwebsite", httpTraceAll(api.DeleteBucketWebsiteHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("website", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// DeleteBucketTaggingHandler
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebuckettagging", httpTraceAll(api.DeleteBucketTaggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("tagging", "")
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
7 years ago
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
5 years ago
// GetBucketObjectLockConfig
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketobjectlockconfiguration", httpTraceAll(api.GetBucketObjectLockConfigHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("object-lock", "")
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
5 years ago
// GetBucketVersioning
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketversioning", httpTraceAll(api.GetBucketVersioningHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("versioning", "")
Virtual host style S3 requests (#5095)
7 years ago
// GetBucketNotification
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketnotification", httpTraceAll(api.GetBucketNotificationHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("notification", "")
Virtual host style S3 requests (#5095)
7 years ago
// ListenBucketNotification
The prometheus metrics refractoring (#8003) The measures are consolidated to the following metrics - `disk_storage_used` : Disk space used by the disk. - `disk_storage_available`: Available disk space left on the disk. - `disk_storage_total`: Total disk space on the disk. - `disks_offline`: Total number of offline disks in current MinIO instance. - `disks_total`: Total number of disks in current MinIO instance. - `s3_requests_total`: Total number of s3 requests in current MinIO instance. - `s3_errors_total`: Total number of errors in s3 requests in current MinIO instance. - `s3_requests_current`: Total number of active s3 requests in current MinIO instance. - `internode_rx_bytes_total`: Total number of internode bytes received by current MinIO server instance. - `internode_tx_bytes_total`: Total number of bytes sent to the other nodes by current MinIO server instance. - `s3_rx_bytes_total`: Total number of s3 bytes received by current MinIO server instance. - `s3_tx_bytes_total`: Total number of s3 bytes sent by current MinIO server instance. - `minio_version_info`: Current MinIO version with commit-id. - `s3_ttfb_seconds_bucket`: Histogram that holds the latency information of the requests. And this PR also modifies the current StorageInfo queries - Decouples StorageInfo from ServerInfo . - StorageInfo is enhanced to give endpoint information. NOTE: ADMIN API VERSION IS BUMPED UP IN THIS PR Fixes #7873
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(collectAPIStats("listenbucketnotification", httpTraceAll(api.ListenBucketNotificationHandler))).Queries("events", "{events:.*}")
Virtual host style S3 requests (#5095)
7 years ago
// ListMultipartUploads
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("listmultipartuploads", httpTraceAll(api.ListMultipartUploadsHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("uploads", "")
Enhance ListObjectsV2 API to return UserDefined metadata (#8539)
5 years ago
// ListObjectsV2M
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("listobjectsv2M", httpTraceAll(api.ListObjectsV2MHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("list-type", "2", "metadata", "true")
Virtual host style S3 requests (#5095)
7 years ago
// ListObjectsV2
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("listobjectsv2", httpTraceAll(api.ListObjectsV2Handler)), enabled, requestsMaxCh, requestsDeadline)).Queries("list-type", "2")
Add listBucketObjectsVersions implementation (#8093) This API implementation simply behaves like listObjects() but returns back single version for each object, this implementation should be considered dummy it is only meant for some applications which rely on this.
5 years ago
// ListBucketVersions
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("listbucketversions", httpTraceAll(api.ListBucketObjectVersionsHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("versions", "")
Virtual host style S3 requests (#5095)
7 years ago
// ListObjectsV1 (Legacy)
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("listobjectsv1", httpTraceAll(api.ListObjectsV1Handler)), enabled, requestsMaxCh, requestsDeadline))
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
5 years ago
// PutBucketLifecycle
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketlifecycle", httpTraceAll(api.PutBucketLifecycleHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("lifecycle", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
5 years ago
// PutBucketEncryption
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketencryption", httpTraceAll(api.PutBucketEncryptionHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("encryption", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
5 years ago
Virtual host style S3 requests (#5095)
7 years ago
// PutBucketPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketpolicy", httpTraceAll(api.PutBucketPolicyHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("policy", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
5 years ago
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
5 years ago
// PutBucketObjectLockConfig
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketobjectlockconfig", httpTraceAll(api.PutBucketObjectLockConfigHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("object-lock", "")
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
5 years ago
// PutBucketVersioning
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketversioning", httpTraceAll(api.PutBucketVersioningHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("versioning", "")
Virtual host style S3 requests (#5095)
7 years ago
// PutBucketNotification
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketnotification", httpTraceAll(api.PutBucketNotificationHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("notification", "")
Virtual host style S3 requests (#5095)
7 years ago
// PutBucket
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucket", httpTraceAll(api.PutBucketHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
7 years ago
// HeadBucket
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodHead).HandlerFunc(
maxClients(collectAPIStats("headbucket", httpTraceAll(api.HeadBucketHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
7 years ago
// PostPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPost).HeadersRegexp(xhttp.ContentType, "multipart/form-data*").HandlerFunc(
maxClients(collectAPIStats("postpolicybucket", httpTraceHdrs(api.PostPolicyBucketHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
7 years ago
// DeleteMultipleObjects
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPost).HandlerFunc(
maxClients(collectAPIStats("deletemultipleobjects", httpTraceAll(api.DeleteMultipleObjectsHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("delete", "")
Virtual host style S3 requests (#5095)
7 years ago
// DeleteBucketPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucketpolicy", httpTraceAll(api.DeleteBucketPolicyHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("policy", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
5 years ago
// DeleteBucketLifecycle
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucketlifecycle", httpTraceAll(api.DeleteBucketLifecycleHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("lifecycle", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
5 years ago
// DeleteBucketEncryption
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucketencryption", httpTraceAll(api.DeleteBucketEncryptionHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("encryption", "")
Virtual host style S3 requests (#5095)
7 years ago
// DeleteBucket
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucket", httpTraceAll(api.DeleteBucketHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
7 years ago
}
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
/// Root operation
// ListBuckets
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
apiRouter.Methods(http.MethodGet).Path(SlashSeparator).HandlerFunc(
maxClients(collectAPIStats("listbuckets", httpTraceAll(api.ListBucketsHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
7 years ago
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
5 years ago
// If none of the routes match add default error handler routes
apiRouter.NotFoundHandler = http.HandlerFunc(collectAPIStats("notfound", httpTraceAll(errorResponseHandler)))
apiRouter.MethodNotAllowedHandler = http.HandlerFunc(collectAPIStats("methodnotallowed", httpTraceAll(errorResponseHandler)))
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
}