Added kernel tuning docs (#3921)

master
Nitish Tiwari 8 years ago committed by Harshavardhana
parent a099319e66
commit a9c0f1e0a4
  1. 71
      docs/deployment/kernel-tuning/README.md
  2. 48
      docs/deployment/kernel-tuning/disk-tuning.sh

@ -0,0 +1,71 @@
# Kernel Tuning for Minio Production Deployment on Linux Servers [![Slack](https://slack.minio.io/slack?type=svg)](https://slack.minio.io) [![Go Report Card](https://goreportcard.com/badge/minio/minio)](https://goreportcard.com/report/minio/minio) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/) [![codecov](https://codecov.io/gh/minio/minio/branch/master/graph/badge.svg)](https://codecov.io/gh/minio/minio)
## Tuning Network Parameters
Following network parameter settings can help ensure optimal Minio server performance on production workloads.
- *`tcp_fin_timeout`* : A socket left in memory takes approximately 1.5Kb of memory. It makes sense to close the unused sockets preemptively to ensure no memory leakage. This way, even if a peer doesn't close the socket due to some reason, the system itself closes it after a timeout. `tcp_fin_timeout` variable defines this timeout and tells kernel how long to keep sockets in the state FIN-WAIT-2. We recommend setting it to 30. You can set it as shown below
```sh
`sysctl -w net.ipv4.tcp_fin_timeout=30`
```
- *`tcp_keepalive_probes`* : This variable defines the number of unacknowledged probes to be sent before considering a connection dead. You can set it as shown below
```sh
sysctl -w net.ipv4.tcp_keepalive_probes=5
```
- *`wmem_max`*: This parameter sets the max OS send buffer size for all types of connections.
```sh
sysctl -w net.core.wmem_max=540000
```
- *`rmem_max`*: This parameter sets the max OS receive buffer size for all types of connections.
```sh
sysctl -w net.core.rmem_max=540000
```
## Tuning Virtual Memory
Recommended virtual memory settings are as follows.
- *`swappiness`* : This parameter controls the relative weight given to swapping out runtime memory, as opposed to dropping pages from the system page cache. It takes values from 0 to 100, both inclusive. We recommend setting it to 10.
```sh
sysctl -w vm.swappiness=10
```
- *`dirty_background_ratio`*: This is the percentage of system memory that can be filled with `dirty` pages, i.e. memory pages that still need to be written to disk. We recommend writing the data to the disk as soon as possible. To do this, set the `dirty_background_ratio` to 1.
```sh
sysctl -w vm.dirty_background_ratio=1
```
- *`dirty_ratio`*: This defines is the absolute maximum amount of system memory that can be filled with dirty pages before everything must get committed to disk.
```sh
sysctl -w vm.dirty_ratio=5
```
## Tuning Scheduler
Proper scheduler configuration makes sure Minio process gets adequate CPU time. Here are the recommended scheduler settings
- *`sched_min_granularity_ns`*: This parameter decides the minimum time a task will be be allowed to run on CPU before being pre-empted out. We recommend setting it to 10ms.
```sh
sysctl -w kernel.sched_min_granularity_ns=10
```
- *`sched_wakeup_granularity_ns`*: Lowering this parameter improves wake-up latency and throughput for latency critical tasks, particularly when a short duty cycle load component must compete with CPU bound components.
```sh
sysctl -w kernel.sched_wakeup_granularity_ns=15
```
## Tuning Disks
The recommendations for disk tuning are conveniently packaged in a well commented [shell script](https://github.com/minio/minio/blob/master/docs/deployment/kernel-tuning/disk-tuning.sh). Please review the shell script for our recommendations.

@ -0,0 +1,48 @@
#!/bin/bash
## Minio Cloud Storage, (C) 2017 Minio, Inc.
##
## Licensed under the Apache License, Version 2.0 (the "License");
## you may not use this file except in compliance with the License.
## You may obtain a copy of the License at
##
## http://www.apache.org/licenses/LICENSE-2.0
##
## Unless required by applicable law or agreed to in writing, software
## distributed under the License is distributed on an "AS IS" BASIS,
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
## See the License for the specific language governing permissions and
## limitations under the License.
for i in $(ls -d /sys/block/*/queue/iosched 2>/dev/null); do
iosched_dir=$(echo $i | awk '/iosched/ {print $1}')
[ -z $iosched_dir ] && {
continue
}
## Change each disk ioscheduler to be "deadline"
## Deadline dispatches I/Os in batches. A batch is a
## sequence of either read or write I/Os which are in
## increasing LBA order (the one-way elevator). After
## processing each batch, the I/O scheduler checks to
## see whether write requests have been starved for too
## long, and then decides whether to start a new batch
## of reads or writes
path=$(dirname $iosched_dir)
[ -f $path/scheduler ] && {
echo "deadline" > $path/scheduler
}
## This controls how many requests may be allocated
## in the block layer for read or write requests.
## Note that the total allocated number may be twice
## this amount, since it applies only to reads or
## writes (not the accumulate sum).
[ -f $path/nr_requests ] && {
echo "256" > $path/nr_requests
}
## This is the maximum number of kilobytes
## supported in a single data transfer at
## block layer.
[ -f $path/max_hw_sectors_kb ] && {
echo "1024" > $path/max_hw_sectors_kb
}
done
Loading…
Cancel
Save