Golang HTTP client automatically detects content-type but
for S3 clients this content-type might be incorrect or
might misbehave.
For example:
```
Content-Type: text/xml; charset=utf-8
```
Should be
```
Content-Type: application/xml
```
Allow this to be set properly.
This is implemented so that the issues like in the
following flow don't affect the behavior of operation.
```
GetObjectInfo()
.... --> Time window for mutation (no lock held)
.... --> Time window for mutation (no lock held)
GetObject()
```
This happens when two simultaneous uploads are made
to the same object the object has returned wrong
info to the client.
Another classic example is "CopyObject" API itself
which reads from a source object and copies to
destination object.
Fixes#3370Fixes#2912
- abstract out instrumentation information.
- use separate lockInstance type that encapsulates the nsMutex, volume,
path and opsID as the frontend or top-level lock object.
- When modifying notification configuration
- When modifying listener configuration
- When modifying policy configuration
With this change we also stop early checking if the bucket exists, since
that uses a Read-lock and causes a deadlock due to the outer Write-lock.
Current master has a regression 'mc policy <policy-type> alias/bucket/prefix'
does not work anymore, due to the way new minio-go changes do json marshalling.
This led to a regression on server side when a ``prefix`` is provided
policy is rejected as malformed from th server which is not the case with
AWS S3.
This patch uses the new ``minio-go/pkg/set`` package to address the
unmarshalling problems.
Fixes#2503
Policies are read once during server startup and subsequently
managed through in memory map. In-memory map is updated as
and when there are new changes coming in.
- Support for '?' wildcard for resource matching.
- Wildcard package is added with Match functions.
- Wildcard.Match supports '*' and wild.MatchExtended supports both '*'
and '?' wildcards in the pattern string.
- Tests for the same for the wide range of cases.
PR contains,
- New setup utilities for running object handler tests. Here is why they are essential,
- Unit tests have to be run in isolation without being have to be associated with other functionalities which are not under test.
- The integration tests follows the philosophy of running a Test Server and registers all handlers and fires HTTP requests over the socket to simulate the system functionality under usual work load scenarios and test for correctness. But this philosophy cannot be adopted for running unit tests for HTTP handlers.
- Running Unit tests for API handlers,
- Shouldn't run a test server. Should purely call the handlers `ServeHTTP` under isolation.
- Shouldn't register all handlers, should only register handlers under test and so that the system is close to be in an isolated setup.
- As an example PutBucketPolicy test is illustrated using the new setup. Exhaustive cases has to be added and has been listen in TODO for now.
This patch brings in the removal of debug logging altogether, instead
we bring in the functionality of being able to trace the errors properly
pointing back to the origination of the problem.
To enable tracing you need to enable "MINIO_TRACE" set to "1" or "true"
environment variable which would print back traces whenever there is an
error which is unhandled or at the handler layer.
By default this tracing is turned off and only user level logging is
provided.
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.
In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
Krishnan Parthasarathi