Kubernetes manages stateless Spark and Hive containers elastically on the compute nodes. Spark has native scheduler integration with Kubernetes. Hive, for legacy reasons, uses YARN scheduler on top of Kubernetes.
Kubernetes manages stateless Spark and Hive containers elastically on the compute nodes. Spark has native scheduler integration with Kubernetes. Hive, for legacy reasons, uses YARN scheduler on top of Kubernetes. \
\
All access to MinIO object storage is via S3/SQL SELECT API. In addition to the compute nodes, MinIO containers are also managed by Kubernetes as stateful containers with local storage (JBOD/JBOF) mapped as persistent local volumes. This architecture enables multi-tenant MinIO, allowing isolation of data between customers.
MinIO also supports multi-cluster, multi-site federation similar to AWS regions and tiers. Using MinIO Information Lifecycle Management (ILM), you can configure data to be tiered between NVMe based hot storage, and HDD based warm storage. All data is encrypted with per-object key. Access Control and Identity Management between the tenants are managed by MinIO using OpenID Connect or Kerberos/LDAP/AD.
@ -21,28 +21,28 @@ MinIO also supports multi-cluster, multi-site federation similar to AWS regions
## **3. Configure Hadoop, Spark, Hive to use MinIO**
After successful installation navigate to the Ambari UI http://<ambari-server>:8080/ and login using the default credentials: [**_username: admin, password: admin_**]
After successful installation navigate to the Ambari UI `http://<ambari-server>:8080/` and login using the default credentials: [**_username: admin, password: admin_**]
Add the following optimal entries for `hive-site.xml` to configure Hive with **MinIO**.
@ -98,11 +98,11 @@ Add the following optimal entries for `hive-site.xml` to configure Hive with **M
For more information about these options please visit [https://www.cloudera.com/documentation/enterprise/5-11-x/topics/admin_hive_on_s3_tuning.html](https://www.cloudera.com/documentation/enterprise/5-11-x/topics/admin_hive_on_s3_tuning.html)
@ -19,9 +19,8 @@ Copy the existing private key and public certificate to the `certs` directory. T
* **Linux:**`${HOME}/.minio/certs`
* **Windows:**`%%USERPROFILE%%\.minio\certs`
> NOTE: Location of custom certs directory can be specified using `--certs-dir` command line option.
**Note:**
* Location of custom certs directory can be specified using `--certs-dir` command line option.
* Inside the `certs` directory, the private key must by named `private.key` and the public key must be named `public.crt`.
* A certificate signed by a CA contains information about the issued identity (e.g. name, expiry, public key) and any intermediate certificates. The root CA is not included.
@ -29,10 +28,10 @@ Copy the existing private key and public certificate to the `certs` directory. T
This section describes how to generate a self-signed certificate using various tools:
3.1 [Use generate_cert.go to Generate a Certificate](#using-go)
3.2 [Use OpenSSL to Generate a Certificate](#using-open-ssl)
3.3 [Use OpenSSL (with IP address) to Generate a Certificate](#using-open-ssl-with-ip)
3.4 [Use GnuTLS (for Windows) to Generate a Certificate](#using-gnu-tls)
* 3.1 [Use generate_cert.go to Generate a Certificate](#using-go)
* 3.2 [Use OpenSSL to Generate a Certificate](#using-open-ssl)
* 3.3 [Use OpenSSL (with IP address) to Generate a Certificate](#using-open-ssl-with-ip)
* 3.4 [Use GnuTLS (for Windows) to Generate a Certificate](#using-gnu-tls)
**Note:**
* MinIO only supports keys and certificates in PEM format on Linux and Windows.
@ -65,7 +64,6 @@ Use one of the following methods to generate a certificate using `openssl`:
* 3.2.2 [Generate a private key with RSA](#generate-private-key-with-rsa)
* 3.2.3 [Generate a self-signed certificate](#generate-a-self-signed-certificate)
#### 3.2.1 <aname="generate-private-key-with-ecdsa"></a>Generate a private key with ECDSA.
Use the following command to generate a private key with ECDSA:
Harshavardhana
6 years ago
committed by