You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Harshavardhana 0104af6bcc delayed locks until we have started reading the body (#10474) ... This is to ensure that Go contexts work properly, after some interesting experiments I found that Go net/http doesn't cancel the context when Body is non-zero and hasn't been read till EOF. The following gist explains this, this can lead to pile up of go-routines on the server which will never be canceled and will die at a really later point in time, which can simply overwhelm the server. https://gist.github.com/harshavardhana/c51dcfd055780eaeb71db54f9c589150 To avoid this refactor the locking such that we take locks after we have started reading from the body and only take locks when needed. Also, remove contextReader as it's not useful, doesn't work as expected context is not canceled until the body reaches EOF so there is no point in wrapping it with context and putting a `select {` on it which can unnecessarily increase the CPU overhead. We will still use the context to cancel the lockers etc. Additional simplification in the locker code to avoid timers as re-using them is a complicated ordeal avoid them in the hot path, since locking is very common this may avoid lots of allocations.		4 years ago
..
config	Continous healing: add optional bitrot check (#10417)	4 years ago
crypto	Copy metadata before spawning goroutine + prealloc maps (#10458)	4 years ago
gateway	delayed locks until we have started reading the body (#10474)	4 years ago
http	Fix common Windows failure (#10397)	4 years ago
jwt	…
logger	Copy metadata before spawning goroutine + prealloc maps (#10458)	4 years ago
rest	Remove MaxConnsPerHost settings to avoid potential hangs (#10438)	4 years ago
acl-handlers.go	…
admin-bucket-handlers.go	fix: Change ListBucketTargets handler (#10217)	4 years ago
admin-handlers-config-kv.go	fix: make sure to use uniform drive count calculation (#10208)	4 years ago
admin-handlers-users.go	…
admin-handlers.go	Remove MaxConnsPerHost settings to avoid potential hangs (#10438)	4 years ago
admin-handlers_test.go	…
admin-heal-ops.go	allow ctrl+c to be consistent at early startup (#10435)	4 years ago
admin-router.go	fix: Change ListBucketTargets handler (#10217)	4 years ago
admin-server-info.go	fix: admin info output and improve overall performance (#10015)	4 years ago
api-datatypes.go	azure: Return success when deleting non-existent object (#9981)	4 years ago
api-errors.go	fix: return correct error code for MetadataTooLarge (#10470)	4 years ago
api-errors_test.go	…
api-headers.go	security: Remove insecure custom headers (#10244)	4 years ago
api-headers_test.go	…
api-resources.go	…
api-resources_test.go	…
api-response.go	fix: bring back delayed leaf detection in listing (#10346)	4 years ago
api-response_test.go	fix: obtain correct location string with DNS style buckets (#10060)	4 years ago
api-router.go	skip subdomain from bucket DNS which start with `minio.domain` (#10390)	4 years ago
api-utils.go	…
api-utils_test.go	…
auth-handler.go	add missing ListBucketVersions from policy actions (#10414)	4 years ago
auth-handler_test.go	feat: Add notification support for bucketCreates and removal (#10075)	4 years ago
background-heal-ops.go	update continous heal metrics appropriately for scanned items (#10352)	4 years ago
background-newdisks-heal-ops.go	re-attach offline drive after new drive replacement (#10416)	4 years ago
benchmark-utils_test.go	…
bitrot-streaming.go	Add storage layer contexts (#10321)	4 years ago
bitrot-whole.go	Add storage layer contexts (#10321)	4 years ago
bitrot.go	…
bitrot_test.go	Add storage layer contexts (#10321)	4 years ago
bootstrap-peer-server.go	allow ctrl+c to be consistent at early startup (#10435)	4 years ago
bucket-encryption-handlers.go	…
bucket-encryption.go	…
bucket-encryption_test.go	…
bucket-handlers.go	delayed locks until we have started reading the body (#10474)	4 years ago
bucket-handlers_test.go	feat: Add notification support for bucketCreates and removal (#10075)	4 years ago
bucket-lifecycle-handlers.go	…
bucket-lifecycle-handlers_test.go	feat: Add notification support for bucketCreates and removal (#10075)	4 years ago
bucket-lifecycle.go	…
bucket-listobjects-handlers.go	add missing ListBucketVersions from policy actions (#10414)	4 years ago
bucket-metadata-sys.go	Use constants for gateway names to avoid bugs caused by spelling. (#10355)	4 years ago
bucket-metadata.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
bucket-metadata_gen.go	Rename replication target handler (#10142)	4 years ago
bucket-metadata_gen_test.go	…
bucket-notification-handlers.go	feat: Add notification support for bucketCreates and removal (#10075)	4 years ago
bucket-object-lock.go	…
bucket-policy-handlers.go	…
bucket-policy-handlers_test.go	feat: Add notification support for bucketCreates and removal (#10075)	4 years ago
bucket-policy.go	support 'ldap:user' variable replacement properly (#10391)	4 years ago
bucket-quota.go	Apply quota usage cache invalidation per second (#10127)	4 years ago
bucket-replication.go	fix: web handlers to enforce replication (#10249)	4 years ago
bucket-targets.go	improve error message when remote target missing (#10412)	4 years ago
bucket-versioning-handler.go	fix: disallow versioning to be suspended with object lock (#9930)	4 years ago
bucket-versioning.go	…
build-constants.go	…
common-main.go	simplify webhook DNS further generalize for gateway (#10448)	4 years ago
config-common.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
config-current.go	update KMS docs indicating deprecation of AUTO_ENCRYPTION env	4 years ago
config-current_test.go	…
config-dir.go	…
config-encrypted.go	fix: migrate to go.etcd.io import path (#9987)	4 years ago
config-encrypted_test.go	…
config-migrate.go	fix: support healing older content (#10076)	4 years ago
config-migrate_test.go	…
config-versions.go	…
config.go	fix: support healing older content (#10076)	4 years ago
consolelogger.go	add validation logs for configured Logger/Audit HTTP targets (#10274)	4 years ago
copy-part-range.go	…
copy-part-range_test.go	…
crossdomain-xml-handler.go	…
crossdomain-xml-handler_test.go	…
data-crawler.go	Continous healing: add optional bitrot check (#10417)	4 years ago
data-update-tracker.go	continous healing based on crawler (#10103)	4 years ago
data-update-tracker_test.go	add validation logs for configured Logger/Audit HTTP targets (#10274)	4 years ago
data-usage-cache.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
data-usage-cache_gen.go	…
data-usage-cache_gen_test.go	…
data-usage.go	…
data-usage_test.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
disk-cache-backend.go	Copy metadata before spawning goroutine + prealloc maps (#10458)	4 years ago
disk-cache-check-support_other.go	…
disk-cache-check-support_windows.go	…
disk-cache-stats.go	…
disk-cache-utils.go	cache: Only start at high watermark (#10403)	4 years ago
disk-cache-utils_test.go	cache: Only start at high watermark (#10403)	4 years ago
disk-cache.go	Copy metadata before spawning goroutine + prealloc maps (#10458)	4 years ago
disk-cache_test.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
dummy-data-generator_test.go	…
dummy-handlers.go	Add support for server side bucket replication (#9882)	4 years ago
dynamic-timeouts.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
dynamic-timeouts_test.go	…
encryption-v1.go	Copy metadata before spawning goroutine + prealloc maps (#10458)	4 years ago
encryption-v1_test.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
endpoint-ellipses.go	fix: make sure to use uniform drive count calculation (#10208)	4 years ago
endpoint-ellipses_test.go	fix: make sure to use uniform drive count calculation (#10208)	4 years ago
endpoint.go	Remove MaxConnsPerHost settings to avoid potential hangs (#10438)	4 years ago
endpoint_test.go	Move dependency from minio-go v6 to v7 (#10042)	4 years ago
erasure-bucket.go	fix: for defer'ed deleteObject use internal context (#10463)	4 years ago
erasure-coding.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
erasure-common.go	erasure: Call Walk() from all disks (#10445)	4 years ago
erasure-common_test.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
erasure-decode.go	fix: pop entries from each drives in parallel (#9918)	4 years ago
erasure-decode_test.go	Add storage layer contexts (#10321)	4 years ago
erasure-encode.go	…
erasure-encode_test.go	Add storage layer contexts (#10321)	4 years ago
erasure-errors.go	…
erasure-heal_test.go	…
erasure-healing-common.go	Add storage layer contexts (#10321)	4 years ago
erasure-healing-common_test.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
erasure-healing.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
erasure-healing_test.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
erasure-lowlevel-heal.go	…
erasure-metadata-utils.go	Add storage layer contexts (#10321)	4 years ago
erasure-metadata-utils_test.go	…
erasure-metadata.go	Add storage layer contexts (#10321)	4 years ago
erasure-metadata_test.go	…
erasure-multipart.go	delayed locks until we have started reading the body (#10474)	4 years ago
erasure-object.go	delayed locks until we have started reading the body (#10474)	4 years ago
erasure-object_test.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
erasure-sets.go	delayed locks until we have started reading the body (#10474)	4 years ago
erasure-sets_test.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
erasure-utils.go	…
erasure-zones.go	delayed locks until we have started reading the body (#10474)	4 years ago
erasure.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
erasure_test.go	Add storage layer contexts (#10321)	4 years ago
etcd.go	fix: migrate to go.etcd.io import path (#9987)	4 years ago
fallocate.go	…
fallocate_linux.go	…
fallocate_linux_test.go	…
fastwalk.go	…
format-disk-cache.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
format-disk-cache_test.go	…
format-erasure.go	Add storage layer contexts (#10321)	4 years ago
format-erasure_test.go	tolerate listing with only readQuorum disks (#10357)	4 years ago
format-fs.go	…
format-fs_test.go	…
format-meta.go	…
fs-v1-helpers.go	simplify removing temporary files (#10389)	4 years ago
fs-v1-helpers_test.go	…
fs-v1-metadata.go	…
fs-v1-metadata_test.go	…
fs-v1-multipart.go	delayed locks until we have started reading the body (#10474)	4 years ago
fs-v1-multipart_test.go	delayed locks until we have started reading the body (#10474)	4 years ago
fs-v1-rwpool.go	…
fs-v1-rwpool_test.go	…
fs-v1.go	Copy metadata before spawning goroutine + prealloc maps (#10458)	4 years ago
fs-v1_test.go	…
gateway-common.go	Copy metadata before spawning goroutine + prealloc maps (#10458)	4 years ago
gateway-common_test.go	…
gateway-env.go	…
gateway-interface.go	Use constants for gateway names to avoid bugs caused by spelling. (#10355)	4 years ago
gateway-main.go	certs: refactor cert manager to support multiple certificates (#10207)	4 years ago
gateway-main_test.go	…
gateway-metrics.go	…
gateway-startup-msg.go	…
gateway-startup-msg_test.go	…
gateway-unsupported.go	delayed locks until we have started reading the body (#10474)	4 years ago
generic-handlers.go	simplify webhook DNS further generalize for gateway (#10448)	4 years ago
generic-handlers_test.go	update rulesguard with new rules (#10392)	4 years ago
global-heal.go	fix: server panic in FS mode (#10455)	4 years ago
globals.go	simplify webhook DNS further generalize for gateway (#10448)	4 years ago
handler-api.go	add support for configurable remote transport deadline (#10447)	4 years ago
handler-utils.go	skip subdomain from bucket DNS which start with `minio.domain` (#10390)	4 years ago
handler-utils_test.go	…
hasher.go	…
healthcheck-handler.go	re-attach offline drive after new drive replacement (#10416)	4 years ago
healthcheck-router.go	fix: readiness needs to be like liveness (#9941)	4 years ago
http-stats.go	…
http-tracer.go	update rulesguard with new rules (#10392)	4 years ago
httprange.go	…
httprange_test.go	…
iam-etcd-store.go	feat: allow service accounts to be generated with OpenID STS (#10184)	4 years ago
iam-object-store.go	use GlobalContext instead of context.Background when possible (#10254)	4 years ago
iam.go	use single dynamic timeout for most locked API/heal ops (#10275)	4 years ago
jwt.go	…
jwt_test.go	…
leak-detect_test.go	…
listen-notification-handlers.go	Add nancy vulnerability scanner (#10289)	4 years ago
local-locker.go	fix: Pass context all the way down to the network call in lockers (#10161)	4 years ago
lock-rest-client.go	Add storage layer contexts (#10321)	4 years ago
lock-rest-client_test.go	fix: Pass context all the way down to the network call in lockers (#10161)	4 years ago
lock-rest-server-common.go	…
lock-rest-server-common_test.go	…
lock-rest-server.go	fix: health handler for lockers (#10280)	4 years ago
main.go	…
merge-walk-pool.go	fix: fix lockup in merge-walk pool (#10098)	4 years ago
merge-walk-pool_test.go	fix: fix lockup in merge-walk pool (#10098)	4 years ago
metrics-router.go	…
metrics.go	Use constants for gateway names to avoid bugs caused by spelling. (#10355)	4 years ago
namespace-lock.go	allow lock tolerance to match storage-class drive tolerance (#10270)	4 years ago
namespace-lock_test.go	…
naughty-disk_test.go	Add storage layer contexts (#10321)	4 years ago
net.go	Move dependency from minio-go v6 to v7 (#10042)	4 years ago
net_test.go	update for smartos build (solaris too) (#10378)	4 years ago
notification.go	load all blocking metadata in background (#10298)	4 years ago
obdinfo.go	add additional fdatasync before close() on writes (#9947)	4 years ago
obdinfo_other.go	add additional fdatasync before close() on writes (#9947)	4 years ago
obdinfo_unix.go	add additional fdatasync before close() on writes (#9947)	4 years ago
object-api-common.go	Add storage layer contexts (#10321)	4 years ago
object-api-datatypes.go	add CopyObject optimization when source and destination are same (#10170)	4 years ago
object-api-deleteobject_test.go	…
object-api-errors.go	improve error message when remote target missing (#10412)	4 years ago
object-api-getobject_test.go	…
object-api-getobjectinfo_test.go	…
object-api-input-checks.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
object-api-interface.go	delayed locks until we have started reading the body (#10474)	4 years ago
object-api-listobjects_test.go	fix: commonPrefixes behavior in ListObjectVersions (#10286)	4 years ago
object-api-multipart_test.go	delayed locks until we have started reading the body (#10474)	4 years ago
object-api-options.go	Copy metadata before spawning goroutine + prealloc maps (#10458)	4 years ago
object-api-putobject_test.go	Remove MaxConnsPerHost settings to avoid potential hangs (#10438)	4 years ago
object-api-utils.go	delayed locks until we have started reading the body (#10474)	4 years ago
object-api-utils_test.go	…
object-handlers-common.go	delayed locks until we have started reading the body (#10474)	4 years ago
object-handlers-common_test.go	…
object-handlers.go	delayed locks until we have started reading the body (#10474)	4 years ago
object-handlers_test.go	update rulesguard with new rules (#10392)	4 years ago
object_api_suite_test.go	delayed locks until we have started reading the body (#10474)	4 years ago
os-dirent_fileino.go	…
os-dirent_ino.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
os-dirent_namelen_bsd.go	…
os-dirent_namelen_linux.go	…
os-readdir_other.go	update for smartos build (solaris too) (#10378)	4 years ago
os-readdir_test.go	…
os-readdir_unix.go	fix: bring back delayed leaf detection in listing (#10346)	4 years ago
os-readdir_windows.go	Fix crash on Windows when crawling (#10385)	4 years ago
os-reliable.go	…
os-reliable_test.go	…
peer-rest-client.go	Add storage layer contexts (#10321)	4 years ago
peer-rest-common.go	refactor server update behavior (#10107)	4 years ago
peer-rest-server.go	use GlobalContext instead of context.Background when possible (#10254)	4 years ago
policy_test.go	Move dependency from minio-go v6 to v7 (#10042)	4 years ago
post-policy_test.go	Allow region errors to be dynamic (#10323)	4 years ago
postpolicyform.go	…
postpolicyform_test.go	Move dependency from minio-go v6 to v7 (#10042)	4 years ago
prepare-storage.go	Remove MaxConnsPerHost settings to avoid potential hangs (#10438)	4 years ago
routers.go	fix: cors handling after gorilla mux update (#9980)	4 years ago
server-main.go	delayed locks until we have started reading the body (#10474)	4 years ago
server-main_test.go	…
server-rlimit.go	…
server-startup-msg.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
server-startup-msg_test.go	…
server_test.go	update rulesguard with new rules (#10392)	4 years ago
service.go	…
setup-type.go	…
signals.go	fix: allow LDAP identity to support form body POST (#10468)	4 years ago
signature-v2.go	Copy metadata before spawning goroutine + prealloc maps (#10458)	4 years ago
signature-v2_test.go	…
signature-v4-parser.go	Allow region errors to be dynamic (#10323)	4 years ago
signature-v4-parser_test.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
signature-v4-utils.go	update rulesguard with new rules (#10392)	4 years ago
signature-v4-utils_test.go	feat: Add notification support for bucketCreates and removal (#10075)	4 years ago
signature-v4.go	Allow region errors to be dynamic (#10323)	4 years ago
signature-v4_test.go	Allow region errors to be dynamic (#10323)	4 years ago
storage-datatypes.go	fix: support healing older content (#10076)	4 years ago
storage-errors.go	fix: reduce an extra system call for writes instead fail later (#10187)	4 years ago
storage-interface.go	Add storage layer contexts (#10321)	4 years ago
storage-rest-client.go	Add storage layer contexts (#10321)	4 years ago
storage-rest-common.go	fix: crash in storage rest client due to spurious query params (#9924)	4 years ago
storage-rest-server.go	Add storage layer contexts (#10321)	4 years ago
storage-rest_test.go	Add storage layer contexts (#10321)	4 years ago
streaming-signature-v4.go	…
streaming-signature-v4_test.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
sts-datatypes.go	…
sts-errors.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
sts-handlers.go	fix: allow LDAP identity to support form body POST (#10468)	4 years ago
test-utils_test.go	Remove erasureSets and erasureObjects from ObjectLayer (#10442)	4 years ago
tree-walk-pool.go	fix: fix lockup in merge-walk pool (#10098)	4 years ago
tree-walk-pool_test.go	fix: threadwalk lockup under high load (#9992)	4 years ago
tree-walk.go	fix: bring back delayed leaf detection in listing (#10346)	4 years ago
tree-walk_test.go	Add storage layer contexts (#10321)	4 years ago
typed-errors.go	…
update-notifier.go	…
update-notifier_test.go	…
update.go	add support for new UA values for update an check (#10451)	4 years ago
update_test.go	add ruleguard support, fix all the reported issues (#10335)	4 years ago
utils.go	Copy metadata before spawning goroutine + prealloc maps (#10458)	4 years ago
utils_test.go	feat: Add notification support for bucketCreates and removal (#10075)	4 years ago
version_test.go	…
web-handler-context.go	…
web-handler-context_test.go	…
web-handlers.go	add support for configurable remote transport deadline (#10447)	4 years ago
web-handlers_test.go	feat: Add notification support for bucketCreates and removal (#10075)	4 years ago
web-router.go	feat: Add notification support for bucketCreates and removal (#10075)	4 years ago
xl-storage-disk-id-check.go	Add storage layer contexts (#10321)	4 years ago
xl-storage-errors.go	update for smartos build (solaris too) (#10378)	4 years ago
xl-storage-errors_test.go	…
xl-storage-format-utils.go	fix: support healing older content (#10076)	4 years ago
xl-storage-format-v1.go	…
xl-storage-format-v1_gen.go	…
xl-storage-format-v1_gen_test.go	…
xl-storage-format-v2.go	Revert "ilm: Remove null version if not latest with proper config (#10467)"	4 years ago
xl-storage-format-v2_gen.go	fix: simplify data structure before release (#9968)	4 years ago
xl-storage-format-v2_gen_test.go	…
xl-storage-format_test.go	…
xl-storage.go	Continous healing: add optional bitrot check (#10417)	4 years ago
xl-storage_test.go	Add storage layer contexts (#10321)	4 years ago
xl-storage_unix_test.go	Add storage layer contexts (#10321)	4 years ago
xl-storage_windows_test.go	Add storage layer contexts (#10321)	4 years ago