listObjects was returning inconsistent results, i.e missing
entries during recursive and non-recursive listing. This led
to 'mc mirror' copying contents repeatedly consisdering
these files to be missing on the destination.
This patch addresses this problem - fixes#1056
HMAC is a much simpler implementation, providing the same
benefits as RSA, avoids additional steps and keeps the code
simpler.
This patch also additionally
- Implements PutObjectURL API.
- GetObjectURL, PutObjectURL take TargetHost as another
argument for generating URL's for proper target destination.
- Adds experimental TLS support for JSON RPC calls.
Without this change listObjects() goes into an infinite loop for
files which have special characters i.e "++" encoded with "%2B%2B".
We have to unescape and convert them to their native representation
before being used internally.
Fixes#1052
Existing code
```
{
if os.IsNotExist(e) {
e = os.MkdirAll(objectDir, 0700)
if e != nil {
return "", probe.NewError(e)
}
}
return "", probe.NewError(e) ---> Error was here.
}
```
For a successful 'MkdirAll' it would still return an empty uploadID,
but the 'error' would be nil. This would succeed the request but
client would fail.
Fix is to re-arrange the logic. Thanks to Alexander Neumann @fd0, for
reporting this problem.
- limit list buckets to limit only 100 buckets, all uppercase buckets
are now lowercase and work transparently with all calls.
- Change disk.Stat to disk.GetInfo and return back disk.Info{} struct.
- Introduce new ioutils package which implements ReadDirN(path, n),
ReadDirNamesN(path, n)
Golang http server strips off 'Expect' header, if the
client sent this as part of signed headers we need to
handle otherwise we would see a signature mismatch.
`aws-cli` sets this as part of signed headers which is
a bad idea since servers trying to implement AWS
Signature version '4' will all encounter this issue.
According to
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.20
Expect header is always of form:
Expect = "Expect" ":" 1#expectation
expectation = "100-continue" | expectation-extension
So it safe to assume that '100-continue' is what would
be sent, for the time being keep this work around.
Harshavardhana