Doc: Modified the contents for Doctor. (#2262)

master
koolhead17 8 years ago committed by Harshavardhana
parent 5730d40478
commit a7b5b8e63f
  1. 32
      README.md
  2. 76
      docs/AWS-SDK-GO.md
  3. 27
      docs/Caddy.md
  4. 15
      docs/Docker.md
  5. 61
      docs/FreeBSD.md
  6. 84
      docs/INSTALLGO.md
  7. 16
      docs/Minio-erasure-code-quickStart-guide.md
  8. 40
      docs/TBD.md

@ -20,17 +20,22 @@ Minio server is light enough to be bundled with the application stack, similar t
Source installation is only intended for developers and advanced users. If you do not have a working Golang environment, please follow [How to install Golang](https://docs.minio.io/docs/how-to-install-golang).
```sh
$ go get -d github.com/minio/minio
$ cd $GOPATH/src/github.com/minio/minio
$ make
```
## 2. Run Minio Server
### 1. GNU/Linux
### GNU/Linux
```sh
$ chmod +x minio
$ ./minio --help
$ ./minio server ~/Photos
@ -51,11 +56,14 @@ Object API (Amazon S3 compatible):
Java: https://docs.minio.io/docs/java-client-quickstart-guide
Python: https://docs.minio.io/docs/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
```
### 2. OS X
### OS X
```sh
$ chmod 755 minio
$ ./minio --help
$ ./minio server ~/Photos
@ -76,11 +84,13 @@ Object API (Amazon S3 compatible):
Java: https://docs.minio.io/docs/java-client-quickstart-guide
Python: https://docs.minio.io/docs/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
```
### 3.Microsoft Windows
### Microsoft Windows
```sh
C:\Users\Username\Downloads> minio.exe --help
C:\Users\Username\Downloads> minio.exe server D:\Photos
@ -101,18 +111,22 @@ Object API (Amazon S3 compatible):
Python: https://docs.minio.io/docs/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
```
### 4. Docker Container
### Docker Container
```sh
$ docker pull minio/minio
$ docker run -p 9000:9000 minio/minio
```
### 5. FreeBSD
### FreeBSD
```sh
$ chmod 755 minio
$ ./minio --help
$ ./minio server ~/Photos
@ -134,24 +148,30 @@ Object API (Amazon S3 compatible):
Python: https://docs.minio.io/docs/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
```
## 3. Test Minio Server using Minio Browser
Open a web browser and navigate to http://127.0.0.1:9000 to view your buckets on minio server.
![Screenshot](https://github.com/minio/minio/blob/master/docs/screenshots/Minio_Browser.jpg?raw=true)
## 4. Test Minio Server using `mc`
Install mc from [here](https://docs.minio.io/docs/minio-client-quick-start-guide). Use `mc ls` command to list all the buckets on your minio server.
```sh
$ mc ls myminio/
[2015-08-05 08:13:22 IST] 0B andoria/
[2015-08-05 06:14:26 IST] 0B deflector/
[2015-08-05 08:13:11 IST] 0B ferenginar/
[2016-03-08 14:56:35 IST] 0B jarjarbing/
[2016-01-20 16:07:41 IST] 0B my.minio.io/
```
For more examples please navigate to [Minio Client Complete Guide](https://docs.minio.io/docs/minio-client-complete-guide).
@ -168,4 +188,4 @@ For more examples please navigate to [Minio Client Complete Guide](https://docs.
## 6. Contribute to Minio Project
Please follow Minio [Contributor's Guide](./CONTRIBUTING.md)
Please follow Minio [Contributor's Guide](https://github.com/minio/minio/blob/master/CONTRIBUTING.md)

@ -1,76 +0,0 @@
## Using aws-sdk-go with Minio
aws-sdk-go is the official AWS SDK for the Go programming language. This document covers
how to use aws-sdk-go with Minio server.
### Install AWS SDK S3 service
```sh
$ go get github.com/aws/aws-sdk-go/service/s3
```
### List all buckets on Minio
```go
package main
import (
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
)
func main() {
newSession := session.New()
s3Config := &aws.Config{
Credentials: credentials.NewStaticCredentials("<YOUR-ACCESS-KEY-ID>", "<YOUR-SECRET-ACCESS-KEY", ""),
Endpoint: aws.String("http://localhost:9000"),
Region: aws.String("us-east-1"),
DisableSSL: aws.Bool(true),
S3ForcePathStyle: aws.Bool(true),
}
// Create an S3 service object in the default region.
s3Client := s3.New(newSession, s3Config)
cparams := &s3.CreateBucketInput{
Bucket: aws.String("newbucket"), // Required
}
_, err := s3Client.CreateBucket(cparams)
if err != nil {
// Message from an error.
fmt.Println(err.Error())
return
}
var lparams *s3.ListBucketsInput
// Call the ListBuckets() Operation
resp, err := s3Client.ListBuckets(lparams)
if err != nil {
// Message from an error.
fmt.Println(err.Error())
return
}
// Pretty-print the response data.
fmt.Println(resp)
}
```
Populate your AccessKeyId and SecretAccessKey credentials and run the program as shown below.
```sh
$ go run aws-sdk-minio.go
{
Buckets: [{
CreationDate: 2015-10-22 01:46:04 +0000 UTC,
Name: "newbucket"
}],
Owner: {
DisplayName: "minio",
ID: "minio"
}
}
```

@ -1,27 +0,0 @@
## Setting up Proxy using Caddy.
Please download [Caddy Server](https://caddyserver.com/download)
Create a caddy configuration file as below, change the ip addresses according to your local
minio and DNS configuration.
```bash
$ ./minio --address localhost:9000 server <your_export_dir>
```
```bash
your.public.com {
proxy / localhost:9000 {
proxy_header Host {host}
proxy_header X-Real-IP {remote}
proxy_header X-Forwarded-Proto {scheme}
}
}
```
```bash
$ ./caddy
Activating privacy features... done.
your.public.com:443
your.public.com:80
```

@ -4,27 +4,36 @@
## 1. Test Minio Docker Container
Minio generates new access and secret keys each time you run this command. Container state is lost after you end this session. This mode is only intended for testing purpose.
```bash
```sh
docker run -p 9000:9000 minio/minio /export
```
## 2. Run Minio Docker Container
Minio container requires a persistent volume to store configuration and application data. Following command maps local persistent directories from the host OS to virtual config `~/.minio` and export `/export` directories.
```bash
```sh
docker run -p 9000:9000 --name minio1 \
-v /mnt/export/minio1:/export \
-v /mnt/config/minio1:/root/.minio \
minio/minio /export
```
## 3. Custom Access and Secret Keys
To override Minio's auto-generated keys, you may pass secret and access keys explicitly as environment variables. Minio server also allows regular strings as access and secret keys.
```bash
```sh
docker run -p 9000:9000 --name minio1 \
-e "MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE" \
-e "MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" \
-v /mnt/export/minio1:/export \
-v /mnt/config/minio1:/root/.minio \
minio/minio /export
```

@ -4,48 +4,62 @@
This example assumes that you have a FreeBSD 10.x running
#### Step *1*
#### Step 1.
As root on the FreeBSD edit `/etc/rc.conf`
```
```sh
zfs_enable="YES"
```
Start ZFS service
```
```sh
# service zfs start
```
```
```sh
# dd if=/dev/zero of=/zfs bs=1M count=4000
```
Configure a loopback device on the `/zfs` file.
```
```sh
# mdconfig -a -t vnode -f /zfs
```
Create zfs pool
```
```sh
# zpool create minio-example /dev/md0
```
```
```sh
# df /minio-example
Filesystem 512-blocks Used Avail Capacity Mounted on
minio-example 7872440 38 7872402 0% /minio-example
```
Verify if it is writable
```
```sh
# touch /minio-example/testfile
# ls -l /minio-example/testfile
-rw-r--r-- 1 root wheel 0 Apr 26 00:51 /minio-example/testfile
```
Now you have successfully created a ZFS pool for futher reading please refer to [ZFS Quickstart Guide](https://www.freebsd.org/doc/handbook/zfs-quickstart.html)
@ -53,14 +67,17 @@ Now you have successfully created a ZFS pool for futher reading please refer to
However, this pool is not taking advantage of any ZFS features, so let's create a ZFS filesytem on this pool with compression enabled. ZFS supports many compression algorithms: lzjb, gzip, zle, lz4. LZ4 is often the most performant algorithm in terms of compression of data versus system overhead.
```
```sh
# zfs create minio-example/compressed-objects
# zfs set compression=lz4 minio-example/compressed-objects
```
To keep monitoring your pool use
```
```sh
# zpool status
pool: minio-example
state: ONLINE
@ -72,19 +89,23 @@ config:
md0 ONLINE 0 0 0
errors: No known data errors
```
#### Step *2*
#### Step 2.
Now start minio server on the ``/minio-example/compressed-objects``, change the permissions such that this directory is accessibly by a normal user
```
```sh
# chown -R minio-user:minio-user /minio-example/compressed-objects
```
Now login as ``minio-user`` and start minio server.
```
```sh
$ curl https://dl.minio.io/server/minio/release/freebsd-amd64/minio > minio
$ chmod 755 minio
$ ./minio server /minio-example/compressed-objects
@ -105,6 +126,7 @@ To configure Minio Client:
$ wget https://dl.minio.io/client/mc/release/freebsd-amd64/mc
$ chmod 755 mc
$ ./mc config host add myminio http://localhost:9000 X3RU3Q6B4T20TYB281W5 bvp0DF8c+6MzL60UxFqOZGlA5Z8UPfIAATUQuzLS
```
Point your browser to http://localhost:9000 and login with the credentials displayed on the command line.
@ -113,13 +135,15 @@ Now you have a S3 compatible server running on top of your ZFS backend which tra
Thanks for using Minio, awaiting feedback :-)
#### Building Minio Server From Source
It is possible to build the minio server from source on FreeBSD. To do this we will used the golang distribution provided by the FreeBSD pkg infrastructure.
First we will need to install golang as well as GNU make:
```
```sh
$ sudo pkg install go gmake
Updating FreeBSD repository catalogue..
FreeBSD repository is up-to-date.
@ -145,19 +169,24 @@ New packages to be INSTALLED:
Next we need to configure our environment for golang. Insert the following lines into your ~/.profile file, and be sure to source the file before proceeding to the next step:
```
```sh
GOPATH=$HOME/golang; export GOPATH
GOROOT=/usr/local/go/; export GOROOT
```
Now we can proceed with the normal build process of minio server as found [here](https://github.com/nomadlogic/minio/blob/master/CONTRIBUTING.md). The only caveat is we need to specify gmake (GNU make) when building minio server as the current Makefile is not BSD make compatible:
```
```sh
$ mkdir -p $GOPATH/src/github.com/minio
$ cd $GOPATH/src/github.com/minio
$ git clone <paste saved URL for personal forked minio repo>
$ cd minio
$ gmake
```
From here you can start the server as you would with a precompiled minio server build.

@ -1,84 +0,0 @@
## Ubuntu (Kylin) 14.04
### Build Dependencies
This installation document assumes Ubuntu 14.04+ on x86-64 platform.
##### Install Git, GCC
```sh
$ sudo apt-get install git build-essential
```
##### Install Go 1.6+
Download Go 1.6+ from [https://golang.org/dl/](https://golang.org/dl/).
```sh
$ wget https://storage.googleapis.com/golang/go1.6.linux-amd64.tar.gz
$ mkdir -p ${HOME}/bin/
$ mkdir -p ${HOME}/go/
$ tar -C ${HOME}/bin/ -xzf go1.6.linux-amd64.tar.gz
```
##### Setup GOROOT and GOPATH
Add the following exports to your ``~/.bashrc``. Environment variable GOROOT specifies the location of your golang binaries
and GOPATH specifies the location of your project workspace.
```sh
export GOROOT=${HOME}/bin/go
export GOPATH=${HOME}/go
export PATH=${HOME}/bin/go/bin:${GOPATH}/bin:$PATH
```
##### Source the new enviornment
```sh
$ source ~/.bashrc
```
##### Testing it all
```sh
$ go env
```
## OS X (Yosemite) 10.10
### Build Dependencies
This installation document assumes OS X Yosemite 10.10+ on x86-64 platform.
##### Install brew
```sh
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
```
##### Install Git, Python
```sh
$ brew install git python
```
##### Install Go 1.6+
Install golang binaries using `brew`
```sh
$ brew install go
$ mkdir -p $HOME/go
```
##### Setup GOROOT and GOPATH
Add the following exports to your ``~/.bash_profile``. Environment variable GOROOT specifies the location of your golang binaries
and GOPATH specifies the location of your project workspace.
```sh
export GOPATH=${HOME}/go
export GOVERSION=$(brew list go | head -n 1 | cut -d '/' -f 6)
export GOROOT=$(brew --prefix)/Cellar/go/${GOVERSION}/libexec
export PATH=${GOPATH}/bin:$PATH
```
##### Source the new enviornment
```sh
$ source ~/.bash_profile
```
##### Testing it all
```sh
$ go env
```

@ -3,33 +3,40 @@
Minio protects data against hardware failures and silent data corruption using erasure code and checksums. You may lose half the number (N/2) of drives and still be able to recover the data.
## 1. Prerequisites:
Install Minio - [Minio Quickstart Guide](https://docs.minio.io/docs/minio)
## What is Erasure Code?
Erasure code is a mathematical algorithm to reconstruct missing or corrupted data. Minio uses Reed-Solomon code to shard objects into N/2 data and N/2 parity blocks. This means that in a 12 drive setup, an object is sharded across as 6 data and 6 parity blocks. You can lose as many as 6 drives (be it parity or data) and still reconstruct the data reliably from the remaining drives.
## Why is Erasure Code useful?
Erasure code protects data from multiple drives failure unlike RAID or replication. Minio encodes each object individually with a high parity count. Storage servers once deployed should not require drive replacement or healing for the lifetime of the server. Minio's erasure coded backend is designed for operational efficiency and takes full advantage of hardware acceleration whenever available.
[![Screenshot](https://www.filepicker.io/api/file/hbaiKmmR9Fg3NZ1yEb8A)](https://docs.minio.io/)
[![Screenshot](https://raw.githubusercontent.com/minio/minio/master/docs/screenshots/Erasure_Code.jpg)](https://docs.minio.io/)
## What is Bit Rot protection?
Bit Rot also known as Data Rot or Silent Data Corruption is a serious data loss issue faced by disk drives today. Data on the drive may silently get corrupted without signalling an error has occurred. This makes Bit Rot more dangerous than permanent hard drive failure.
Minio's erasure coded backend uses high speed [BLAKE2](https://blog.minio.io/accelerating-blake2b-by-4x-using-simd-in-go-assembly-33ef16c8a56b#.jrp1fdwer) hash based checksums to protect against Bit Rot.
## Deployment Scenarios
Minio server runs on a variety of hardware, operating systems and virtual/container environments.
Minio erasure code backend is limited by design to a minimum of 6 drives and a maximum of 16 drives. The hard limit of 16 drives comes from operational experience. Failure domain becomes too large beyond 16 drives. If you need to scale beyond 16 drives, you may run multiple instances of Minio server on different ports.
#### Reference Physical Hardware:
* [SMC 5018A-AR12L (Intel Atom)](http://www.supermicro.com/products/system/1U/5018/SSG-5018A-AR12L.cfm?parts=SHOW) - SMC 1U SoC Atom C2750 platform with 12x 3.5” drive bays
* [Quanta Grid D51B-2U (OCP Compliant) ](http://www.qct.io/Product/Servers/Rackmount-Servers/2U/QuantaGrid-D51B-2U-p256c77c70c83c118)- Quanta 2U DP E5-2600v3 platform with 12x 3.5” drive bays
* [Cisco UCS C240 M4 Rack Server](http://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-c240-m4-rack-server/index.html) - Cisco 2U DP E5-2600v3 platform with 12x 3.5” drive bays
* [Intel® Server System R2312WTTYSR](http://ark.intel.com/products/88286) - Intel 2U DP E5-2600v3 platform with 12x 3.5” drive bays
#### Reference Cloud Hosting Providers:
* [Packet](https://www.packet.net): Packet is the baremetal cloud provider.
* [Digital Ocean](https://www.digitalocean.com): Deploy an SSD cloud server in 55 seconds.
* [OVH](https://www.ovh.com/us): Build your own infrastructure with OVH public cloud.
@ -37,10 +44,15 @@ Minio erasure code backend is limited by design to a minimum of 6 drives and a m
* [SSD Nodes](https://www.ssdnodes.com): Simple, high performance cloud provider with truly personalized support.
## 2. Run Minio Server with Erasure Code.
Example: Start Minio server in a 12 drives setup.
```
```sh
$ minio server /mnt/export1/backend /mnt/export2/backend /mnt/export3/backend /mnt/export4/backend /mnt/export5/backend /mnt/export6/backend /mnt/export7/backend /mnt/export8/backend /mnt/export9/backend /mnt/export10/backend /mnt/export11/backend /mnt/export12/backend
```
## 3. Test your setup
You may unplug drives randomly and continue to perform I/O on the system.

@ -1,40 +0,0 @@
# Server command line SPEC - Fri Apr 1 19:49:56 PDT 2016
## Single disk
Regular single server, single disk mode.
```
$ minio server <disk>
```
## Multi disk
```
$ minio controller start - Start minio controller.
```
Prints a secret token to be used by all parties.
Start all servers without disk with `MINIO_CONTROLLER` env set, start in cluster mode.
```
$ MINIO_CONTROLLER=<host>:<token> minio server
```
## Minio Controller cli.
Set controller host and token.
```
$ export MINIO_CONTROLLER=<host>:<token>
```
Create a cluster from the pool of nodes.
```
$ minioctl new <clustername> <ip1>:/mnt/disk1 .. <ip16>:/mnt/disk1
```
Start the cluster.
```
$ minioctl start <clustername>
```
Stop the cluster
```
$ minioctl stop <clustername>
```
Loading…
Cancel
Save