In a distributed setup that the server should not perform any operation
on the storage layer after it is exported via RPC. e.g, cleaning up of
temporary directories under .minio.sys/tmp may interfere with ongoing
PUT objects being served by the distributed setup.
These messages based on our prep stage during XL
and prints more informative message regarding
drive information.
This change also does a much needed refactoring.
Initialization when disk was down the network disk
reported an incorrect error rather than errDiskNotFound.
This resulted in incorrect error handling during
prepInitStorage() stage.
Fixes#2577
This change initializes rpc servers associated with disks that are
local. It makes object layer initialization on demand, namely on the
first request to the object layer.
Also adds lock RPC service vendorized minio/dsync
By default server heals/creates missing directories and re-populates
`format.json`, in some scenarios when disk is down for maintainenance
it would be beneficial for users to ignore such disks rather than
mistakenly using `root` partition.
Fixes#2128
Each metadata ops have a list of errors which can be
ignored, this is essentially needed when
- disks are not found
- disks are found but cannot be accessed (permission denied)
- disks are there but fresh disks were added
This is needed since we don't have healing code in place where
it would have healed the fresh disks added.
Fixes#2072
Fixes#1636
New format now generates a UUID and includes it along with
the order of disks. So that UUID is the real order of disks
and on command line user is able to specify disks in any order.
This pre-dominantly solves our dilemma.
```
{
"format" : "xl",
"xl" : {
"version" : "1",
"disk": "00e4cf06-5bf5-4bb5-b885-4b2fff4a7959",
"jbod" : [
"00e4cf06-5bf5-4bb5-b885-4b2fff4a7959",
....
"c47d2608-5067-4ed7-b1e4-fb81bdbb549f",
"a543293e-99f1-4310-b540-1e450878e844",
"18f97cbe-529a-456a-b6d4-0feacf64534d"
]
},
"version" : "1"
}
```
* posix: Avoid using getAllVolumeInfo() in getVolumeDir()
This is necessary compromise to avoid significant slowness this
causes under load. The compromise is also substantial in a way
so that to avoid penalizing common cases v/s special cases.
For buckets with Caps on Unixes, we filter buckets based on the
latest anyways, so this is completely acceptable.
* XL/fs: Change the usage of verification of existance of buckets.
Optimize calling isBucketExists, it is not needed for all call
paths. isBucketExist should be called only for calls which use
temporary volume location for operations, for the rest rely on
the errors returned on their original call path.
Remove usage of filtering as well across all volume names.
This patch brings in the removal of debug logging altogether, instead
we bring in the functionality of being able to trace the errors properly
pointing back to the origination of the problem.
To enable tracing you need to enable "MINIO_TRACE" set to "1" or "true"
environment variable which would print back traces whenever there is an
error which is unhandled or at the handler layer.
By default this tracing is turned off and only user level logging is
provided.
All transactions happen through this directory inside ".minio/temp".
Adding this allows us to remove any temporary files which were not
committed before.
Fixes#1462Fixes#1444
- PutObject()
- PutObjectPart()
- NewMultipartUpload()
- AbortMultipartUpload()
Implementations across both FS and XL object layer
share common implementation.
Krishna Srinivas