From a7b5b8e63f4ec985646eac84abb4afb6df27c953 Mon Sep 17 00:00:00 2001 From: koolhead17 Date: Thu, 21 Jul 2016 14:58:16 -0700 Subject: [PATCH] Doc: Modified the contents for Doctor. (#2262) --- README.md | 32 ++++++-- docs/AWS-SDK-GO.md | 76 ------------------- docs/Caddy.md | 27 ------- docs/Docker.md | 15 +++- docs/FreeBSD.md | 61 +++++++++++---- docs/INSTALLGO.md | 84 --------------------- docs/Minio-erasure-code-quickStart-guide.md | 16 +++- docs/TBD.md | 40 ---------- 8 files changed, 97 insertions(+), 254 deletions(-) delete mode 100644 docs/AWS-SDK-GO.md delete mode 100644 docs/Caddy.md delete mode 100644 docs/INSTALLGO.md delete mode 100644 docs/TBD.md diff --git a/README.md b/README.md index e917d88a1..4cb0fdb17 100644 --- a/README.md +++ b/README.md @@ -20,17 +20,22 @@ Minio server is light enough to be bundled with the application stack, similar t Source installation is only intended for developers and advanced users. If you do not have a working Golang environment, please follow [How to install Golang](https://docs.minio.io/docs/how-to-install-golang). + ```sh + $ go get -d github.com/minio/minio $ cd $GOPATH/src/github.com/minio/minio $ make + ``` ## 2. Run Minio Server -### 1. GNU/Linux + +### GNU/Linux ```sh + $ chmod +x minio $ ./minio --help $ ./minio server ~/Photos @@ -51,11 +56,14 @@ Object API (Amazon S3 compatible): Java: https://docs.minio.io/docs/java-client-quickstart-guide Python: https://docs.minio.io/docs/python-client-quickstart-guide JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide + ``` -### 2. OS X +### OS X + ```sh + $ chmod 755 minio $ ./minio --help $ ./minio server ~/Photos @@ -76,11 +84,13 @@ Object API (Amazon S3 compatible): Java: https://docs.minio.io/docs/java-client-quickstart-guide Python: https://docs.minio.io/docs/python-client-quickstart-guide JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide + ``` -### 3.Microsoft Windows +### Microsoft Windows ```sh + C:\Users\Username\Downloads> minio.exe --help C:\Users\Username\Downloads> minio.exe server D:\Photos @@ -101,18 +111,22 @@ Object API (Amazon S3 compatible): Python: https://docs.minio.io/docs/python-client-quickstart-guide JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide + ``` -### 4. Docker Container +### Docker Container ```sh + $ docker pull minio/minio $ docker run -p 9000:9000 minio/minio + ``` -### 5. FreeBSD +### FreeBSD ```sh + $ chmod 755 minio $ ./minio --help $ ./minio server ~/Photos @@ -134,24 +148,30 @@ Object API (Amazon S3 compatible): Python: https://docs.minio.io/docs/python-client-quickstart-guide JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide + ``` ## 3. Test Minio Server using Minio Browser + Open a web browser and navigate to http://127.0.0.1:9000 to view your buckets on minio server. ![Screenshot](https://github.com/minio/minio/blob/master/docs/screenshots/Minio_Browser.jpg?raw=true) + ## 4. Test Minio Server using `mc` + Install mc from [here](https://docs.minio.io/docs/minio-client-quick-start-guide). Use `mc ls` command to list all the buckets on your minio server. ```sh + $ mc ls myminio/ [2015-08-05 08:13:22 IST] 0B andoria/ [2015-08-05 06:14:26 IST] 0B deflector/ [2015-08-05 08:13:11 IST] 0B ferenginar/ [2016-03-08 14:56:35 IST] 0B jarjarbing/ [2016-01-20 16:07:41 IST] 0B my.minio.io/ + ``` For more examples please navigate to [Minio Client Complete Guide](https://docs.minio.io/docs/minio-client-complete-guide). @@ -168,4 +188,4 @@ For more examples please navigate to [Minio Client Complete Guide](https://docs. ## 6. Contribute to Minio Project -Please follow Minio [Contributor's Guide](./CONTRIBUTING.md) +Please follow Minio [Contributor's Guide](https://github.com/minio/minio/blob/master/CONTRIBUTING.md) diff --git a/docs/AWS-SDK-GO.md b/docs/AWS-SDK-GO.md deleted file mode 100644 index 899c6232a..000000000 --- a/docs/AWS-SDK-GO.md +++ /dev/null @@ -1,76 +0,0 @@ -## Using aws-sdk-go with Minio - -aws-sdk-go is the official AWS SDK for the Go programming language. This document covers -how to use aws-sdk-go with Minio server. - -### Install AWS SDK S3 service - -```sh -$ go get github.com/aws/aws-sdk-go/service/s3 -``` - -### List all buckets on Minio - -```go -package main - -import ( - "fmt" - - "github.com/aws/aws-sdk-go/aws" - "github.com/aws/aws-sdk-go/aws/credentials" - "github.com/aws/aws-sdk-go/aws/session" - "github.com/aws/aws-sdk-go/service/s3" -) - -func main() { - newSession := session.New() - s3Config := &aws.Config{ - Credentials: credentials.NewStaticCredentials("", " -``` - -```bash -your.public.com { - proxy / localhost:9000 { - proxy_header Host {host} - proxy_header X-Real-IP {remote} - proxy_header X-Forwarded-Proto {scheme} - } -} -``` - -```bash -$ ./caddy -Activating privacy features... done. -your.public.com:443 -your.public.com:80 -``` diff --git a/docs/Docker.md b/docs/Docker.md index 5842c3ed3..fd8b880a3 100644 --- a/docs/Docker.md +++ b/docs/Docker.md @@ -4,27 +4,36 @@ ## 1. Test Minio Docker Container Minio generates new access and secret keys each time you run this command. Container state is lost after you end this session. This mode is only intended for testing purpose. -```bash +```sh + docker run -p 9000:9000 minio/minio /export + ``` ## 2. Run Minio Docker Container + Minio container requires a persistent volume to store configuration and application data. Following command maps local persistent directories from the host OS to virtual config `~/.minio` and export `/export` directories. -```bash +```sh + docker run -p 9000:9000 --name minio1 \ -v /mnt/export/minio1:/export \ -v /mnt/config/minio1:/root/.minio \ minio/minio /export + ``` ## 3. Custom Access and Secret Keys + To override Minio's auto-generated keys, you may pass secret and access keys explicitly as environment variables. Minio server also allows regular strings as access and secret keys. -```bash + +```sh + docker run -p 9000:9000 --name minio1 \ -e "MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE" \ -e "MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" \ -v /mnt/export/minio1:/export \ -v /mnt/config/minio1:/root/.minio \ minio/minio /export + ``` diff --git a/docs/FreeBSD.md b/docs/FreeBSD.md index 1a34e567c..dee2da5b7 100644 --- a/docs/FreeBSD.md +++ b/docs/FreeBSD.md @@ -4,48 +4,62 @@ This example assumes that you have a FreeBSD 10.x running -#### Step *1* +#### Step 1. As root on the FreeBSD edit `/etc/rc.conf` -``` +```sh + zfs_enable="YES" + ``` Start ZFS service -``` +```sh + # service zfs start -``` ``` + +```sh + # dd if=/dev/zero of=/zfs bs=1M count=4000 + ``` Configure a loopback device on the `/zfs` file. -``` +```sh + # mdconfig -a -t vnode -f /zfs + ``` Create zfs pool -``` +```sh + # zpool create minio-example /dev/md0 -``` ``` + +```sh + # df /minio-example Filesystem 512-blocks Used Avail Capacity Mounted on minio-example 7872440 38 7872402 0% /minio-example + ``` Verify if it is writable -``` +```sh + # touch /minio-example/testfile # ls -l /minio-example/testfile -rw-r--r-- 1 root wheel 0 Apr 26 00:51 /minio-example/testfile + ``` Now you have successfully created a ZFS pool for futher reading please refer to [ZFS Quickstart Guide](https://www.freebsd.org/doc/handbook/zfs-quickstart.html) @@ -53,14 +67,17 @@ Now you have successfully created a ZFS pool for futher reading please refer to However, this pool is not taking advantage of any ZFS features, so let's create a ZFS filesytem on this pool with compression enabled. ZFS supports many compression algorithms: lzjb, gzip, zle, lz4. LZ4 is often the most performant algorithm in terms of compression of data versus system overhead. -``` +```sh + # zfs create minio-example/compressed-objects # zfs set compression=lz4 minio-example/compressed-objects + ``` To keep monitoring your pool use -``` +```sh + # zpool status pool: minio-example state: ONLINE @@ -72,19 +89,23 @@ config: md0 ONLINE 0 0 0 errors: No known data errors + ``` -#### Step *2* +#### Step 2. Now start minio server on the ``/minio-example/compressed-objects``, change the permissions such that this directory is accessibly by a normal user -``` +```sh + # chown -R minio-user:minio-user /minio-example/compressed-objects + ``` Now login as ``minio-user`` and start minio server. -``` +```sh + $ curl https://dl.minio.io/server/minio/release/freebsd-amd64/minio > minio $ chmod 755 minio $ ./minio server /minio-example/compressed-objects @@ -105,6 +126,7 @@ To configure Minio Client: $ wget https://dl.minio.io/client/mc/release/freebsd-amd64/mc $ chmod 755 mc $ ./mc config host add myminio http://localhost:9000 X3RU3Q6B4T20TYB281W5 bvp0DF8c+6MzL60UxFqOZGlA5Z8UPfIAATUQuzLS + ``` Point your browser to http://localhost:9000 and login with the credentials displayed on the command line. @@ -113,13 +135,15 @@ Now you have a S3 compatible server running on top of your ZFS backend which tra Thanks for using Minio, awaiting feedback :-) + #### Building Minio Server From Source It is possible to build the minio server from source on FreeBSD. To do this we will used the golang distribution provided by the FreeBSD pkg infrastructure. First we will need to install golang as well as GNU make: -``` +```sh + $ sudo pkg install go gmake Updating FreeBSD repository catalogue.. FreeBSD repository is up-to-date. @@ -145,19 +169,24 @@ New packages to be INSTALLED: Next we need to configure our environment for golang. Insert the following lines into your ~/.profile file, and be sure to source the file before proceeding to the next step: -``` + +```sh + GOPATH=$HOME/golang; export GOPATH GOROOT=/usr/local/go/; export GOROOT + ``` Now we can proceed with the normal build process of minio server as found [here](https://github.com/nomadlogic/minio/blob/master/CONTRIBUTING.md). The only caveat is we need to specify gmake (GNU make) when building minio server as the current Makefile is not BSD make compatible: -``` +```sh + $ mkdir -p $GOPATH/src/github.com/minio $ cd $GOPATH/src/github.com/minio $ git clone $ cd minio $ gmake + ``` From here you can start the server as you would with a precompiled minio server build. diff --git a/docs/INSTALLGO.md b/docs/INSTALLGO.md deleted file mode 100644 index 6ec1ffde2..000000000 --- a/docs/INSTALLGO.md +++ /dev/null @@ -1,84 +0,0 @@ -## Ubuntu (Kylin) 14.04 -### Build Dependencies -This installation document assumes Ubuntu 14.04+ on x86-64 platform. - -##### Install Git, GCC -```sh -$ sudo apt-get install git build-essential -``` - -##### Install Go 1.6+ - -Download Go 1.6+ from [https://golang.org/dl/](https://golang.org/dl/). - -```sh -$ wget https://storage.googleapis.com/golang/go1.6.linux-amd64.tar.gz -$ mkdir -p ${HOME}/bin/ -$ mkdir -p ${HOME}/go/ -$ tar -C ${HOME}/bin/ -xzf go1.6.linux-amd64.tar.gz -``` -##### Setup GOROOT and GOPATH - -Add the following exports to your ``~/.bashrc``. Environment variable GOROOT specifies the location of your golang binaries -and GOPATH specifies the location of your project workspace. - -```sh -export GOROOT=${HOME}/bin/go -export GOPATH=${HOME}/go -export PATH=${HOME}/bin/go/bin:${GOPATH}/bin:$PATH -``` -##### Source the new enviornment - -```sh -$ source ~/.bashrc -``` - -##### Testing it all - -```sh -$ go env -``` - -## OS X (Yosemite) 10.10 -### Build Dependencies -This installation document assumes OS X Yosemite 10.10+ on x86-64 platform. - -##### Install brew -```sh -$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" -``` - -##### Install Git, Python -```sh -$ brew install git python -``` - -##### Install Go 1.6+ - -Install golang binaries using `brew` - -```sh -$ brew install go -$ mkdir -p $HOME/go -``` - -##### Setup GOROOT and GOPATH - -Add the following exports to your ``~/.bash_profile``. Environment variable GOROOT specifies the location of your golang binaries -and GOPATH specifies the location of your project workspace. - -```sh -export GOPATH=${HOME}/go -export GOVERSION=$(brew list go | head -n 1 | cut -d '/' -f 6) -export GOROOT=$(brew --prefix)/Cellar/go/${GOVERSION}/libexec -export PATH=${GOPATH}/bin:$PATH -``` -##### Source the new enviornment -```sh -$ source ~/.bash_profile -``` -##### Testing it all - -```sh -$ go env -``` diff --git a/docs/Minio-erasure-code-quickStart-guide.md b/docs/Minio-erasure-code-quickStart-guide.md index 1738cd768..d508df361 100644 --- a/docs/Minio-erasure-code-quickStart-guide.md +++ b/docs/Minio-erasure-code-quickStart-guide.md @@ -3,33 +3,40 @@ Minio protects data against hardware failures and silent data corruption using erasure code and checksums. You may lose half the number (N/2) of drives and still be able to recover the data. ## 1. Prerequisites: + Install Minio - [Minio Quickstart Guide](https://docs.minio.io/docs/minio) ## What is Erasure Code? + Erasure code is a mathematical algorithm to reconstruct missing or corrupted data. Minio uses Reed-Solomon code to shard objects into N/2 data and N/2 parity blocks. This means that in a 12 drive setup, an object is sharded across as 6 data and 6 parity blocks. You can lose as many as 6 drives (be it parity or data) and still reconstruct the data reliably from the remaining drives. ## Why is Erasure Code useful? + Erasure code protects data from multiple drives failure unlike RAID or replication. Minio encodes each object individually with a high parity count. Storage servers once deployed should not require drive replacement or healing for the lifetime of the server. Minio's erasure coded backend is designed for operational efficiency and takes full advantage of hardware acceleration whenever available. -[![Screenshot](https://www.filepicker.io/api/file/hbaiKmmR9Fg3NZ1yEb8A)](https://docs.minio.io/) +[![Screenshot](https://raw.githubusercontent.com/minio/minio/master/docs/screenshots/Erasure_Code.jpg)](https://docs.minio.io/) ## What is Bit Rot protection? + Bit Rot also known as Data Rot or Silent Data Corruption is a serious data loss issue faced by disk drives today. Data on the drive may silently get corrupted without signalling an error has occurred. This makes Bit Rot more dangerous than permanent hard drive failure. Minio's erasure coded backend uses high speed [BLAKE2](https://blog.minio.io/accelerating-blake2b-by-4x-using-simd-in-go-assembly-33ef16c8a56b#.jrp1fdwer) hash based checksums to protect against Bit Rot. ## Deployment Scenarios + Minio server runs on a variety of hardware, operating systems and virtual/container environments. Minio erasure code backend is limited by design to a minimum of 6 drives and a maximum of 16 drives. The hard limit of 16 drives comes from operational experience. Failure domain becomes too large beyond 16 drives. If you need to scale beyond 16 drives, you may run multiple instances of Minio server on different ports. #### Reference Physical Hardware: + * [SMC 5018A-AR12L (Intel Atom)](http://www.supermicro.com/products/system/1U/5018/SSG-5018A-AR12L.cfm?parts=SHOW) - SMC 1U SoC Atom C2750 platform with 12x 3.5” drive bays * [Quanta Grid D51B-2U (OCP Compliant) ](http://www.qct.io/Product/Servers/Rackmount-Servers/2U/QuantaGrid-D51B-2U-p256c77c70c83c118)- Quanta 2U DP E5-2600v3 platform with 12x 3.5” drive bays * [Cisco UCS C240 M4 Rack Server](http://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-c240-m4-rack-server/index.html) - Cisco 2U DP E5-2600v3 platform with 12x 3.5” drive bays * [Intel® Server System R2312WTTYSR](http://ark.intel.com/products/88286) - Intel 2U DP E5-2600v3 platform with 12x 3.5” drive bays #### Reference Cloud Hosting Providers: + * [Packet](https://www.packet.net): Packet is the baremetal cloud provider. * [Digital Ocean](https://www.digitalocean.com): Deploy an SSD cloud server in 55 seconds. * [OVH](https://www.ovh.com/us): Build your own infrastructure with OVH public cloud. @@ -37,10 +44,15 @@ Minio erasure code backend is limited by design to a minimum of 6 drives and a m * [SSD Nodes](https://www.ssdnodes.com): Simple, high performance cloud provider with truly personalized support. ## 2. Run Minio Server with Erasure Code. + Example: Start Minio server in a 12 drives setup. -``` + +```sh + $ minio server /mnt/export1/backend /mnt/export2/backend /mnt/export3/backend /mnt/export4/backend /mnt/export5/backend /mnt/export6/backend /mnt/export7/backend /mnt/export8/backend /mnt/export9/backend /mnt/export10/backend /mnt/export11/backend /mnt/export12/backend + ``` ## 3. Test your setup + You may unplug drives randomly and continue to perform I/O on the system. diff --git a/docs/TBD.md b/docs/TBD.md deleted file mode 100644 index 9dad6be8e..000000000 --- a/docs/TBD.md +++ /dev/null @@ -1,40 +0,0 @@ -# Server command line SPEC - Fri Apr 1 19:49:56 PDT 2016 - -## Single disk -Regular single server, single disk mode. -``` -$ minio server -``` - -## Multi disk -``` -$ minio controller start - Start minio controller. -``` -Prints a secret token to be used by all parties. - -Start all servers without disk with `MINIO_CONTROLLER` env set, start in cluster mode. -``` -$ MINIO_CONTROLLER=: minio server -``` - -## Minio Controller cli. - -Set controller host and token. -``` -$ export MINIO_CONTROLLER=: -``` - -Create a cluster from the pool of nodes. -``` -$ minioctl new :/mnt/disk1 .. :/mnt/disk1 -``` - -Start the cluster. -``` -$ minioctl start -``` - -Stop the cluster -``` -$ minioctl stop -```