oyd
/
minio
6
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
7573 Commits
1 Branch
0 Tags
89 MiB
Tag: Branch: Tree: 3a0082f0f1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '3a0082f0f1'
${ noResults }
minio/cmd/api-router.go

383 lines
18 KiB
Raw Normal View History Unescape

routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
/*
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
* MinIO Cloud Storage, (C) 2016-2020 MinIO, Inc.
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
server: Move all the top level files into cmd folder. (#2490) This change brings a change which was done for the 'mc' package to allow for clean repo and have a cleaner github drop in experience.
8 years ago
package cmd
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
7 years ago
import (
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
4 years ago
"net"
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
7 years ago
"net/http"
use package name correctly (#5827)
7 years ago
"github.com/gorilla/mux"
Ensure that we use constants everywhere (#7845) This allows for canonicalization of the strings throughout our code and provides a common space for all these constants to reside. This list is rather non-exhaustive but captures all the headers used in AWS S3 API operations
5 years ago
xhttp "github.com/minio/minio/cmd/http"
fix: cors handling after gorilla mux update (#9980) fixes #9979
4 years ago
"github.com/minio/minio/pkg/wildcard"
"github.com/rs/cors"
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
7 years ago
)
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
Enable build verification with race (#8583)
5 years ago
func newHTTPServerFn() *xhttp.Server {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
return globalHTTPServer
}
Bring in safe mode support (#8478) This PR refactors object layer handling such that upon failure in sub-system initialization server reaches a stage of safe-mode operation wherein only certain API operations are enabled and available. This allows for fixing many scenarios such as - incorrect configuration in vault, etcd, notification targets - missing files, incomplete config migrations unable to read encrypted content etc - any other issues related to notification, policies, lifecycle etc
5 years ago
func newObjectLayerFn() ObjectLayer {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
return globalObjectAPI
}
func newCachedObjectLayerFn() CacheObjectLayer {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
return globalCacheObjectAPI
}
ListObjects Metadata Caching (#10648) Design: https://gist.github.com/klauspost/025c09b48ed4a1293c917cecfabdf21c Gist of improvements: * Cross-server caching and listing will use the same data across servers and requests. * Lists can be arbitrarily resumed at a constant speed. * Metadata for all files scanned is stored for streaming retrieval. * The existing bloom filters controlled by the crawler is used for validating caches. * Concurrent requests for the same data (or parts of it) will not spawn additional walkers. * Listing a subdirectory of an existing recursive cache will use the cache. * All listing operations are fully streamable so the number of objects in a bucket no longer dictates the amount of memory. * Listings can be handled by any server within the cluster. * Caches are cleaned up when out of date or superseded by a more recent one.
4 years ago
func setObjectLayer(o ObjectLayer) {
globalObjLayerMutex.Lock()
globalObjectAPI = o
globalObjLayerMutex.Unlock()
}
storage/server/client: Enable storage server, enable client storage.
9 years ago
// objectAPIHandler implements and provides http handlers for S3 API.
type objectAPIHandlers struct {
distribute: Make server work with multiple remote disks This change initializes rpc servers associated with disks that are local. It makes object layer initialization on demand, namely on the first request to the object layer. Also adds lock RPC service vendorized minio/dsync
8 years ago
ObjectAPI func() ObjectLayer
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
7 years ago
CacheAPI func() CacheObjectLayer
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
}
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
4 years ago
// getHost tries its best to return the request host.
// According to section 14.23 of RFC 2616 the Host header
// can include the port number if the default value of 80 is not used.
func getHost(r *http.Request) string {
if r.URL.IsAbs() {
return r.URL.Host
}
return r.Host
}
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
// registerAPIRouter - registers S3 compatible APIs.
fix: S3 gateway doesn't support full passthrough for encryption (#10484) The entire encryption layer is dependent on the fact that KMS should be configured for S3 encryption to work properly and we only support passing the headers as is to the backend for encryption only if KMS is configured. Make sure that this predictability is maintained, currently the code was allowing encryption to go through and fail at later to indicate that KMS was not configured. We should simply reply "NotImplemented" if KMS is not configured, this allows clients to simply proceed with their tests.
4 years ago
func registerAPIRouter(router *mux.Router) {
server: Add more elaborate startup messages. (#2731) These messages based on our prep stage during XL and prints more informative message regarding drive information. This change also does a much needed refactoring.
8 years ago
// Initialize API.
api := objectAPIHandlers{
Bring in safe mode support (#8478) This PR refactors object layer handling such that upon failure in sub-system initialization server reaches a stage of safe-mode operation wherein only certain API operations are enabled and available. This allows for fixing many scenarios such as - incorrect configuration in vault, etcd, notification targets - missing files, incomplete config migrations unable to read encrypted content etc - any other issues related to notification, policies, lifecycle etc
5 years ago
ObjectAPI: newObjectLayerFn,
CacheAPI: newCachedObjectLayerFn,
server: Add more elaborate startup messages. (#2731) These messages based on our prep stage during XL and prints more informative message regarding drive information. This change also does a much needed refactoring.
8 years ago
}
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
// API Router
Use const slashSeparator instead of "/" everywhere (#8028)
5 years ago
apiRouter := router.PathPrefix(SlashSeparator).Subrouter()
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
4 years ago
use package name correctly (#5827)
7 years ago
var routers []*mux.Router
Support multiple-domains in MINIO_DOMAIN (#7274) Fixes #7173
6 years ago
for _, domainName := range globalDomainNames {
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
4 years ago
if IsKubernetes() {
routers = append(routers, apiRouter.MatcherFunc(func(r *http.Request, match *mux.RouteMatch) bool {
fix: host extraction without port
4 years ago
host, _, err := net.SplitHostPort(getHost(r))
if err != nil {
host = r.Host
}
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
4 years ago
// Make sure to skip matching minio.<domain>` this is
// specifically meant for operator/k8s deployment
// The reason we need to skip this is for a special
// usecase where we need to make sure that
// minio.<namespace>.svc.<cluster_domain> is ignored
// by the bucketDNS style to ensure that path style
// is available and honored at this domain.
//
// All other `<bucket>.<namespace>.svc.<cluster_domain>`
// makes sure that buckets are routed through this matcher
// to match for `<bucket>`
return host != minioReservedBucket+"."+domainName
}).Host("{bucket:.+}."+domainName).Subrouter())
} else {
routers = append(routers, apiRouter.Host("{bucket:.+}."+domainName).Subrouter())
}
Virtual host style S3 requests (#5095)
7 years ago
}
routers = append(routers, apiRouter.PathPrefix("/{bucket}").Subrouter())
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
Virtual host style S3 requests (#5095)
7 years ago
for _, bucket := range routers {
// Object operations
// HeadObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodHead).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("headobject", httpTraceAll(api.HeadObjectHandler))))
Virtual host style S3 requests (#5095)
7 years ago
// CopyObjectPart
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
4 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").
HeadersRegexp(xhttp.AmzCopySource, ".*?(\\/|%2F).*?").
HandlerFunc(maxClients(collectAPIStats("copyobjectpart", httpTraceAll(api.CopyObjectPartHandler)))).
Queries("partNumber", "{partNumber:[0-9]+}", "uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
7 years ago
// PutObjectPart
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putobjectpart", httpTraceHdrs(api.PutObjectPartHandler)))).Queries("partNumber", "{partNumber:[0-9]+}", "uploadId", "{uploadId:.*}")
Add listBucketObjectsVersions implementation (#8093) This API implementation simply behaves like listObjects() but returns back single version for each object, this implementation should be considered dummy it is only meant for some applications which rely on this.
5 years ago
// ListObjectParts
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("listobjectparts", httpTraceAll(api.ListObjectPartsHandler)))).Queries("uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
7 years ago
// CompleteMultipartUpload
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("completemutipartupload", httpTraceAll(api.CompleteMultipartUploadHandler)))).Queries("uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
7 years ago
// NewMultipartUpload
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("newmultipartupload", httpTraceAll(api.NewMultipartUploadHandler)))).Queries("uploads", "")
Virtual host style S3 requests (#5095)
7 years ago
// AbortMultipartUpload
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("abortmultipartupload", httpTraceAll(api.AbortMultipartUploadHandler)))).Queries("uploadId", "{uploadId:.*}")
Fix s3cmd issue with ACL handling (#5999) With the implementation of dummy GET ACL handlers, tools like s3cmd perform few operations which causes the ACL call to be invoked. Make sure that in our router configuration GET?acl comes before actual GET call to facilitate this dummy call.
7 years ago
// GetObjectACL - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getobjectacl", httpTraceHdrs(api.GetObjectACLHandler)))).Queries("acl", "")
acl: Support PUT calls with success for 'private' ACL's (#9000) Add dummy calls which respond success when ACL's are set to be private and fails, if user tries to change them from their default 'private' Some applications such as nuxeo may have an unnecessary requirement for this operation, we support this anyways such that don't have to fully implement the functionality just that we can respond with success for default ACLs
5 years ago
// PutObjectACL - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putobjectacl", httpTraceHdrs(api.PutObjectACLHandler)))).Queries("acl", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
5 years ago
// GetObjectTagging
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getobjecttagging", httpTraceHdrs(api.GetObjectTaggingHandler)))).Queries("tagging", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
5 years ago
// PutObjectTagging
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putobjecttagging", httpTraceHdrs(api.PutObjectTaggingHandler)))).Queries("tagging", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
5 years ago
// DeleteObjectTagging
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("deleteobjecttagging", httpTraceHdrs(api.DeleteObjectTaggingHandler)))).Queries("tagging", "")
S3 Select API Support for CSV (#6127) Add support for trivial where clause cases
6 years ago
// SelectObjectContent
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("selectobjectcontent", httpTraceHdrs(api.SelectObjectContentHandler)))).Queries("select", "").Queries("select-type", "2")
Add object retention at the per object (#8528) level - this PR builds on #8120 which added PutBucketObjectLockConfiguration and GetBucketObjectLockConfiguration APIS This PR implements PutObjectRetention, GetObjectRetention API and enhances PUT and GET API operations to display governance metadata if permissions allow.
5 years ago
// GetObjectRetention
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getobjectretention", httpTraceAll(api.GetObjectRetentionHandler)))).Queries("retention", "")
Add support for object locking with legal hold. (#8634)
5 years ago
// GetObjectLegalHold
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getobjectlegalhold", httpTraceAll(api.GetObjectLegalHoldHandler)))).Queries("legal-hold", "")
Virtual host style S3 requests (#5095)
7 years ago
// GetObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getobject", httpTraceHdrs(api.GetObjectHandler))))
Virtual host style S3 requests (#5095)
7 years ago
// CopyObject
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
4 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HeadersRegexp(xhttp.AmzCopySource, ".*?(\\/|%2F).*?").
HandlerFunc(maxClients(collectAPIStats("copyobject", httpTraceAll(api.CopyObjectHandler))))
Add object retention at the per object (#8528) level - this PR builds on #8120 which added PutBucketObjectLockConfiguration and GetBucketObjectLockConfiguration APIS This PR implements PutObjectRetention, GetObjectRetention API and enhances PUT and GET API operations to display governance metadata if permissions allow.
5 years ago
// PutObjectRetention
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putobjectretention", httpTraceAll(api.PutObjectRetentionHandler)))).Queries("retention", "")
Add support for object locking with legal hold. (#8634)
5 years ago
// PutObjectLegalHold
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putobjectlegalhold", httpTraceAll(api.PutObjectLegalHoldHandler)))).Queries("legal-hold", "")
Add support for object locking with legal hold. (#8634)
5 years ago
Virtual host style S3 requests (#5095)
7 years ago
// PutObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putobject", httpTraceHdrs(api.PutObjectHandler))))
Virtual host style S3 requests (#5095)
7 years ago
// DeleteObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("deleteobject", httpTraceAll(api.DeleteObjectHandler))))
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
Virtual host style S3 requests (#5095)
7 years ago
/// Bucket operations
// GetBucketLocation
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketlocation", httpTraceAll(api.GetBucketLocationHandler)))).Queries("location", "")
Virtual host style S3 requests (#5095)
7 years ago
// GetBucketPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketpolicy", httpTraceAll(api.GetBucketPolicyHandler)))).Queries("policy", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
5 years ago
// GetBucketLifecycle
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketlifecycle", httpTraceAll(api.GetBucketLifecycleHandler)))).Queries("lifecycle", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
5 years ago
// GetBucketEncryption
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketencryption", httpTraceAll(api.GetBucketEncryptionHandler)))).Queries("encryption", "")
Add support for server side bucket replication (#9882)
4 years ago
// GetBucketObjectLockConfig
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketobjectlockconfiguration", httpTraceAll(api.GetBucketObjectLockConfigHandler)))).Queries("object-lock", "")
// GetBucketReplicationConfig
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketreplicationconfiguration", httpTraceAll(api.GetBucketReplicationConfigHandler)))).Queries("replication", "")
// GetBucketVersioning
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketversioning", httpTraceAll(api.GetBucketVersioningHandler)))).Queries("versioning", "")
// GetBucketNotification
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketnotification", httpTraceAll(api.GetBucketNotificationHandler)))).Queries("notification", "")
// ListenNotification
bucket.Methods(http.MethodGet).HandlerFunc(collectAPIStats("listennotification", httpTraceAll(api.ListenNotificationHandler))).Queries("events", "{events:.*}")
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
7 years ago
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// Dummy Bucket Calls
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
7 years ago
// GetBucketACL -- this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketacl", httpTraceAll(api.GetBucketACLHandler)))).Queries("acl", "")
acl: Support PUT calls with success for 'private' ACL's (#9000) Add dummy calls which respond success when ACL's are set to be private and fails, if user tries to change them from their default 'private' Some applications such as nuxeo may have an unnecessary requirement for this operation, we support this anyways such that don't have to fully implement the functionality just that we can respond with success for default ACLs
5 years ago
// PutBucketACL -- this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putbucketacl", httpTraceAll(api.PutBucketACLHandler)))).Queries("acl", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketCors - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketcors", httpTraceAll(api.GetBucketCorsHandler)))).Queries("cors", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketWebsiteHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketwebsite", httpTraceAll(api.GetBucketWebsiteHandler)))).Queries("website", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketAccelerateHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketaccelerate", httpTraceAll(api.GetBucketAccelerateHandler)))).Queries("accelerate", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketRequestPaymentHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketrequestpayment", httpTraceAll(api.GetBucketRequestPaymentHandler)))).Queries("requestPayment", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketLoggingHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketlogging", httpTraceAll(api.GetBucketLoggingHandler)))).Queries("logging", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// GetBucketLifecycleHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbucketlifecycle", httpTraceAll(api.GetBucketLifecycleHandler)))).Queries("lifecycle", "")
add bucket tagging support (#9389) This patch also simplifies object tagging support
5 years ago
// GetBucketTaggingHandler
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("getbuckettagging", httpTraceAll(api.GetBucketTaggingHandler)))).Queries("tagging", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
//DeleteBucketWebsiteHandler
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("deletebucketwebsite", httpTraceAll(api.DeleteBucketWebsiteHandler)))).Queries("website", "")
Implements dummy tagging handlers for Terraform (#7040)
6 years ago
// DeleteBucketTaggingHandler
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("deletebuckettagging", httpTraceAll(api.DeleteBucketTaggingHandler)))).Queries("tagging", "")
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
7 years ago
Virtual host style S3 requests (#5095)
7 years ago
// ListMultipartUploads
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("listmultipartuploads", httpTraceAll(api.ListMultipartUploadsHandler)))).Queries("uploads", "")
Enhance ListObjectsV2 API to return UserDefined metadata (#8539)
5 years ago
// ListObjectsV2M
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("listobjectsv2M", httpTraceAll(api.ListObjectsV2MHandler)))).Queries("list-type", "2", "metadata", "true")
Virtual host style S3 requests (#5095)
7 years ago
// ListObjectsV2
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("listobjectsv2", httpTraceAll(api.ListObjectsV2Handler)))).Queries("list-type", "2")
Support bucket versioning (#9377) - Implement a new xl.json 2.0.0 format to support, this moves the entire marshaling logic to POSIX layer, top layer always consumes a common FileInfo construct which simplifies the metadata reads. - Implement list object versions - Migrate to siphash from crchash for new deployments for object placements. Fixes #2111
5 years ago
// ListObjectVersions
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
Support bucket versioning (#9377) - Implement a new xl.json 2.0.0 format to support, this moves the entire marshaling logic to POSIX layer, top layer always consumes a common FileInfo construct which simplifies the metadata reads. - Implement list object versions - Migrate to siphash from crchash for new deployments for object placements. Fixes #2111
5 years ago
maxClients(collectAPIStats("listobjectversions", httpTraceAll(api.ListObjectVersionsHandler)))).Queries("versions", "")
Virtual host style S3 requests (#5095)
7 years ago
// ListObjectsV1 (Legacy)
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodGet).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("listobjectsv1", httpTraceAll(api.ListObjectsV1Handler))))
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
5 years ago
// PutBucketLifecycle
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putbucketlifecycle", httpTraceAll(api.PutBucketLifecycleHandler)))).Queries("lifecycle", "")
Add support for server side bucket replication (#9882)
4 years ago
// PutBucketReplicationConfig
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketreplicationconfiguration", httpTraceAll(api.PutBucketReplicationConfigHandler)))).Queries("replication", "")
// GetObjectRetention
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
5 years ago
// PutBucketEncryption
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putbucketencryption", httpTraceAll(api.PutBucketEncryptionHandler)))).Queries("encryption", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
5 years ago
Virtual host style S3 requests (#5095)
7 years ago
// PutBucketPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putbucketpolicy", httpTraceAll(api.PutBucketPolicyHandler)))).Queries("policy", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
5 years ago
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
5 years ago
// PutBucketObjectLockConfig
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putbucketobjectlockconfig", httpTraceAll(api.PutBucketObjectLockConfigHandler)))).Queries("object-lock", "")
add bucket tagging support (#9389) This patch also simplifies object tagging support
5 years ago
// PutBucketTaggingHandler
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbuckettagging", httpTraceAll(api.PutBucketTaggingHandler)))).Queries("tagging", "")
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
5 years ago
// PutBucketVersioning
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putbucketversioning", httpTraceAll(api.PutBucketVersioningHandler)))).Queries("versioning", "")
Virtual host style S3 requests (#5095)
7 years ago
// PutBucketNotification
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putbucketnotification", httpTraceAll(api.PutBucketNotificationHandler)))).Queries("notification", "")
Virtual host style S3 requests (#5095)
7 years ago
// PutBucket
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPut).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("putbucket", httpTraceAll(api.PutBucketHandler))))
Virtual host style S3 requests (#5095)
7 years ago
// HeadBucket
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodHead).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("headbucket", httpTraceAll(api.HeadBucketHandler))))
Virtual host style S3 requests (#5095)
7 years ago
// PostPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPost).HeadersRegexp(xhttp.ContentType, "multipart/form-data*").HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("postpolicybucket", httpTraceHdrs(api.PostPolicyBucketHandler))))
Virtual host style S3 requests (#5095)
7 years ago
// DeleteMultipleObjects
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodPost).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("deletemultipleobjects", httpTraceAll(api.DeleteMultipleObjectsHandler)))).Queries("delete", "")
Virtual host style S3 requests (#5095)
7 years ago
// DeleteBucketPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("deletebucketpolicy", httpTraceAll(api.DeleteBucketPolicyHandler)))).Queries("policy", "")
Add support for server side bucket replication (#9882)
4 years ago
// DeleteBucketReplication
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucketreplicationconfiguration", httpTraceAll(api.DeleteBucketReplicationConfigHandler)))).Queries("replication", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
5 years ago
// DeleteBucketLifecycle
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("deletebucketlifecycle", httpTraceAll(api.DeleteBucketLifecycleHandler)))).Queries("lifecycle", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
5 years ago
// DeleteBucketEncryption
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("deletebucketencryption", httpTraceAll(api.DeleteBucketEncryptionHandler)))).Queries("encryption", "")
Virtual host style S3 requests (#5095)
7 years ago
// DeleteBucket
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
bucket.Methods(http.MethodDelete).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("deletebucket", httpTraceAll(api.DeleteBucketHandler))))
Add support for ILM transition (#10565) This PR adds transition support for ILM to transition data to another MinIO target represented by a storage class ARN. Subsequent GET or HEAD for that object will be streamed from the transition tier. If PostRestoreObject API is invoked, the transitioned object can be restored for duration specified to the source cluster.
4 years ago
// PostRestoreObject
bucket.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("restoreobject", httpTraceAll(api.PostRestoreObjectHandler)))).Queries("restore", "")
Virtual host style S3 requests (#5095)
7 years ago
}
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
/// Root operation
feat: Add notification support for bucketCreates and removal (#10075)
4 years ago
// ListenNotification
apiRouter.Methods(http.MethodGet).Path(SlashSeparator).HandlerFunc(
collectAPIStats("listennotification", httpTraceAll(api.ListenNotificationHandler))).Queries("events", "{events:.*}")
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
// ListBuckets
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
5 years ago
apiRouter.Methods(http.MethodGet).Path(SlashSeparator).HandlerFunc(
config: Add api requests max & deadline configs (#9273) Add two new configuration entries, api.requests-max and api.requests-deadline which have the same role of MINIO_API_REQUESTS_MAX and MINIO_API_REQUESTS_DEADLINE.
5 years ago
maxClients(collectAPIStats("listbuckets", httpTraceAll(api.ListBucketsHandler))))
Virtual host style S3 requests (#5095)
7 years ago
add missing signature v2 query params (#9670)
5 years ago
// S3 browser with signature v4 adds '//' for ListBuckets request, so rather
// than failing with UnknownAPIRequest we simply handle it for now.
apiRouter.Methods(http.MethodGet).Path(SlashSeparator + SlashSeparator).HandlerFunc(
maxClients(collectAPIStats("listbuckets", httpTraceAll(api.ListBucketsHandler))))
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
5 years ago
// If none of the routes match add default error handler routes
Remove redundant http.HandlerFunc type conversion. (#10576)
4 years ago
apiRouter.NotFoundHandler = collectAPIStats("notfound", httpTraceAll(errorResponseHandler))
ListObjects Metadata Caching (#10648) Design: https://gist.github.com/klauspost/025c09b48ed4a1293c917cecfabdf21c Gist of improvements: * Cross-server caching and listing will use the same data across servers and requests. * Lists can be arbitrarily resumed at a constant speed. * Metadata for all files scanned is stored for streaming retrieval. * The existing bloom filters controlled by the crawler is used for validating caches. * Concurrent requests for the same data (or parts of it) will not spawn additional walkers. * Listing a subdirectory of an existing recursive cache will use the cache. * All listing operations are fully streamable so the number of objects in a bucket no longer dictates the amount of memory. * Listings can be handled by any server within the cluster. * Caches are cleaned up when out of date or superseded by a more recent one.
4 years ago
apiRouter.MethodNotAllowedHandler = collectAPIStats("methodnotallowed", httpTraceAll(methodNotAllowedHandler("S3")))
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
5 years ago
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
9 years ago
}
fix: cors handling after gorilla mux update (#9980) fixes #9979
4 years ago
fix: cors handling again for not just OPTIONS request (#10025) CORS is notorious requires specific headers to be handled appropriately in request and response, using cors package as part of handlerFunc() for options method lacks the necessary control this package needs to add headers.
4 years ago
// corsHandler handler for CORS (Cross Origin Resource Sharing)
func corsHandler(handler http.Handler) http.Handler {
fix: cors handling after gorilla mux update (#9980) fixes #9979
4 years ago
commonS3Headers := []string{
xhttp.Date,
xhttp.ETag,
xhttp.ServerInfo,
xhttp.Connection,
xhttp.AcceptRanges,
xhttp.ContentRange,
xhttp.ContentEncoding,
xhttp.ContentLength,
xhttp.ContentType,
fix: add missing content-disposition from CORS handler (#10137)
4 years ago
xhttp.ContentDisposition,
fix: add missing response headers to CORS handler (#10124)
4 years ago
xhttp.LastModified,
xhttp.ContentLanguage,
xhttp.CacheControl,
xhttp.RetryAfter,
xhttp.AmzBucketRegion,
xhttp.Expires,
fix: cors handling after gorilla mux update (#9980) fixes #9979
4 years ago
"X-Amz*",
"x-amz*",
"*",
}
fix: cors handling again for not just OPTIONS request (#10025) CORS is notorious requires specific headers to be handled appropriately in request and response, using cors package as part of handlerFunc() for options method lacks the necessary control this package needs to add headers.
4 years ago
return cors.New(cors.Options{
fix: cors handling after gorilla mux update (#9980) fixes #9979
4 years ago
AllowOriginFunc: func(origin string) bool {
for _, allowedOrigin := range globalAPIConfig.getCorsAllowOrigins() {
if wildcard.MatchSimple(allowedOrigin, origin) {
return true
}
}
return false
},
AllowedMethods: []string{
http.MethodGet,
http.MethodPut,
http.MethodHead,
http.MethodPost,
http.MethodDelete,
http.MethodOptions,
http.MethodPatch,
},
AllowedHeaders: commonS3Headers,
ExposedHeaders: commonS3Headers,
AllowCredentials: true,
fix: cors handling again for not just OPTIONS request (#10025) CORS is notorious requires specific headers to be handled appropriately in request and response, using cors package as part of handlerFunc() for options method lacks the necessary control this package needs to add headers.
4 years ago
}).Handler(handler)
fix: cors handling after gorilla mux update (#9980) fixes #9979
4 years ago
}