Minio server supports Amazon S3 compatible bucket event notification for the following targets [AMQP](https://www.amqp.org/about/what), [Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html) , [Redis](http://redis.io/documentation), [nats.io](http://nats.io/), [PostgreSQL](https://www.postgresql.org) and [kafka](https://kafka.apache.org/).
Minio server supports Amazon S3 compatible bucket event notification for the following targets
## Prerequisites
| Notification Targets|
|:---|
| [`AMQP`](#AMQP) |
| [`Elasticsearch`](#Elasticsearch) |
| [`Redis`](#Redis) |
| [`NATS`](#NATS) |
| [`PostgreSQL`](#PostgreSQL) |
| [`Apache Kafka`](#apache-kafka) |
## Prerequisites
* Install and configure Minio Server from [here](http://docs.minio.io/docs/minio).
* Install and configure Minio Client from [here](https://docs.minio.io/docs/minio-client-quickstart-guide).
| Notification Targets|
|:--
| [`RabbitMQ`](#RabbitMQ) |
| [`Elasticsearch`](#Elasticsearch) |
| [`Redis`](#Redis) |
| [`NATS`](#NATS) |
| [`PostgreSQL`](#PostgreSQL) |
| [`kafka`](#kafka) |
<aname="RabbitMQ"></a>
## Publish Minio events via RabbitMQ
<aname="AMQP"></a>
## Publish Minio events via AMQP
Install RabbitMQ from [here](https://www.rabbitmq.com/).
### Recipe steps
### Step 1: Add AMQP endpoint to Minio
### Step 1: Add RabbitMQ endpoint to Minio
The default location of Minio server configuration file is ``~/.minio/config.json``. Update the AMQP configuration block in ``config.json`` as follows:
The default location of Minio server configuration file is ``~/.minio/config.json``. Update the RabbitMQ configuration block in ``config.json`` as follows:
```
```json
"amqp": {
"1": {
"enable": true,
"url": "amqp://myuser:mypassword@localhost:5672",
"exchange": "bucketevents",
"routingKey": "bucketlogs",
"exchangeType": "fanout",
"mandatory": false,
"immediate": false,
"durable": false,
"internal": false,
"noWait": false,
"autoDeleted": false
}
}
```
Restart Minio server to reflect config changes. Minio supports all the exchange available in [RabbitMQ](https://www.rabbitmq.com/). For this setup, we are using ``fanout`` exchange.
"1": {
"enable": true,
"url": "amqp://myuser:mypassword@localhost:5672",
"exchange": "bucketevents",
"routingKey": "bucketlogs",
"exchangeType": "fanout",
"mandatory": false,
"immediate": false,
"durable": false,
"internal": false,
"noWait": false,
"autoDeleted": false
}
}
```
Restart Minio server to reflect config changes. Minio supports all the exchanges available in [RabbitMQ](https://www.rabbitmq.com/). For this setup, we are using ``fanout`` exchange.
### Step 2: Enable bucket notification using Minio client
@ -52,14 +51,14 @@ We will enable bucket event notification to trigger whenever a JPEG image is upl
```
mc mb myminio/images
mc events add myminio/images arn:minio:sqs:us-east-1:1:amqp --suffix .jpg
mc events add myminio/images arn:minio:sqs:us-east-1:1:amqp --suffix .jpg
The python program below waits on the queue exchange ``bucketevents`` and prints event notifications on the console. We use [Pika Python Client](https://www.rabbitmq.com/tutorials/tutorial-three-python.html) library to do this.
The python program below waits on the queue exchange ``bucketevents`` and prints event notifications on the console. We use [Pika Python Client](https://www.rabbitmq.com/tutorials/tutorial-three-python.html) library to do this.
```py
#!/usr/bin/env python
@ -108,8 +107,9 @@ You should receive the following event notification via RabbitMQ once the upload
Install Elasticsearch 2.4 from [here](https://www.elastic.co/downloads/past-releases/elasticsearch-2-4-0).
@ -119,15 +119,16 @@ Install Elasticsearch 2.4 from [here](https://www.elastic.co/downloads/past-rele
The default location of Minio server configuration file is ``~/.minio/config.json``. Update the Elasticsearch configuration block in ``config.json`` as follows:
```
```json
"elasticsearch": {
"1": {
"enable": true,
"url": "http://127.0.0.1:9200",
"index": "bucketevents"
}
},
"1": {
"enable": true,
"url": "http://127.0.0.1:9200",
"index": "bucketevents"
}
},
```
Restart Minio server to reflect config changes. ``bucketevents`` is the index used by Elasticsearch.
### Step 2: Enable bucket notification using Minio client
Install Redis from [here](http://redis.io/download).
## Recipe steps
### Step 1: Add Redis endpoint to Minio
The default location of Minio server configuration file is ``~/.minio/config.json``. Update the Redis configuration block in ``config.json`` as follows:
```
```json
"redis": {
"1": {
"enable": true,
"address": "127.0.0.1:6379",
"password": "yoursecret",
"key": "bucketevents"
}
}
"1": {
"enable": true,
"address": "127.0.0.1:6379",
"password": "yoursecret",
"key": "bucketevents"
}
}
```
Restart Minio server to reflect config changes. ``bucketevents`` is the key used by Redis in this example.
### Step 2: Enable bucket notification using Minio client
@ -358,7 +360,7 @@ mc cp myphoto.jpg myminio/images
The example ``nats.go`` program prints event notification to console.
```
go run nats.go
go run nats.go
2016/10/12 06:51:26 Connected
2016/10/12 06:51:26 Subscribing to subject 'bucketevents'
2016/10/12 06:51:33 Received message '{"EventType":"s3:ObjectCreated:Put","Key":"images/myphoto.jpg","Records":[{"eventVersion":"2.0","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2016-10-12T13:51:33Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"principalId":"minio"},"requestParameters":{"sourceIPAddress":"[::1]:57106"},"responseElements":{},"s3":{"s3SchemaVersion":"1.0","configurationId":"Config","bucket":{"name":"images","ownerIdentity":{"principalId":"minio"},"arn":"arn:aws:s3:::images"},"object":{"key":"myphoto.jpg","size":56060,"eTag":"1d97bf45ecb37f7a7b699418070df08f","sequencer":"147CCD1AE054BFD0"}}}],"level":"info","msg":"","time":"2016-10-12T06:51:33-07:00"}
@ -369,26 +371,25 @@ go run nats.go
Install PostgreSQL from [here](https://www.postgresql.org/).
## Recipe steps
### Step 1: Add PostgreSQL endpoint to Minio
The default location of Minio server configuration file is ``~/.minio/config.json``. Update the PostgreSQL configuration block in ``config.json`` as follows:
```
"postgresql": {
"1": {
"enable": true,
"connectionString": "",
"table": "bucketevents",
"host": "127.0.0.1",
"port": "5432",
"user": "postgres",
"password": "mypassword",
"database": "bucketevents_db"
}
}
"1": {
"enable": true,
"connectionString": "",
"table": "bucketevents",
"host": "127.0.0.1",
"port": "5432",
"user": "postgres",
"password": "mypassword",
"database": "bucketevents_db"
}
}
```
Restart Minio server to reflect config changes. ``bucketevents`` is the database table used by PostgreSQL in this example.
### Step 2: Enable bucket notification using Minio client
@ -415,32 +416,31 @@ Open PostgreSQL terminal to list the saved event notification logs.
Install kafka from [here](http://kafka.apache.org/).
## Recipe steps
### Step 1: Add kafka endpoint to Minio
The default location of Minio server configuration file is ``~/.minio/config.json``. Update the kafka configuration block in ``config.json`` as follows:
```
"kafka": {
"1": {
"enable": true,
"brokers": ["localhost:9092"],
"topic": "bucketevents"
}
"1": {
"enable": true,
"brokers": ["localhost:9092"],
"topic": "bucketevents"
}
}
```
Restart Minio server to reflect config changes. ``bucketevents`` is the topic used by kafka in this example.
### Step 2: Enable bucket notification using Minio client
Harshavardhana
8 years ago