Ozone

Slack Docker Pulls GitHub edit source

This guide describes how to configure Ozone as Alluxio’s under storage system. Ozone is a scalable, redundant, and distributed object store for Hadoop. Apart from scaling to billions of objects of varying sizes, Ozone can function effectively in containerized environments such as Kubernetes and YARN.

Prerequisites

To run an Alluxio cluster on a set of machines, you must deploy Alluxio binaries to each of these machines. You can either download the precompiled binaries directly with the correct Hadoop version (recommended), or compile the binaries from Alluxio source code (for advanced users).

In preparation for using Ozone with Alluxio, follow the Ozone On Premise Installation to install a Ozone cluster, and follow the Volume Commands and Bucket Commands to create volume and bucket for Ozone cluster.

Basic Setup

To configure Alluxio to use Ozone as under storage, you will need to modify the configuration file conf/alluxio-site.properties. If the file does not exist, create the configuration file from the template.

  1. $ cp conf/alluxio-site.properties.template conf/alluxio-site.properties

Edit conf/alluxio-site.properties file to set the under storage address to the Ozone bucket and the Ozone directory you want to mount to Alluxio. For example, the under storage address can be o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/ if you want to mount the whole bucket to Alluxio, or o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/alluxio/data if only the directory /alluxio/data inside the ozone bucket <OZONE_BUCKET> of <OZONE_VOLUME> is mapped to Alluxio.

  1. alluxio.master.mount.table.root.ufs=o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/

Example: Running Alluxio Locally with Ozone

Start the Alluxio servers:

  1. $ ./bin/alluxio format
  2. $ ./bin/alluxio-start.sh local

This will start an Alluxio master and an Alluxio worker. You can see the master UI at http://localhost:19999.

Run a simple example program:

  1. $ ./bin/alluxio runTests

Use the HDFS shell or Ozone shell to Visit your Ozone directory o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/<OZONE_DIRECTORY> to verify the files and directories created by Alluxio exist. For this test, you should see files named like <OZONE_BUCKET>.<OZONE_VOLUME>/<OZONE_DIRECTORY>/default_tests_files/BasicFile_CACHE_PROMOTE_MUST_CACHE.

Stop Alluxio by running:

  1. $ ./bin/alluxio-stop.sh local

Advanced Setup

Mount Ozone

An Ozone location can be mounted at a nested directory in the Alluxio namespace to have unified access to multiple under storage systems. Alluxio’s Mount Command can be used for this purpose. For example, the following command mounts a directory inside an Ozone bucket into Alluxio directory /ozone:

  1. $ ./bin/alluxio fs mount \
  2.   --option alluxio.underfs.hdfs.configuration=<DIR>/ozone-site.xml:<DIR>/core-site.xml \
  3.   /ozone o3fs://<OZONE_BUCKET>.<OZONE_VOLUME>/

Possible core-site.xml and ozone-site.xml

  • core-site.xml
  1. <configuration>
  2.   <property>
  3.     <name>fs.o3fs.impl</name>
  4.     <value>org.apache.hadoop.fs.ozone.BasicOzoneFileSystem</value>
  5.   </property>
  6.   <property>
  7.     <name>fs.AbstractFileSystem.o3fs.impl</name>
  8.     <value>org.apache.hadoop.fs.ozone.BasicOzFs</value>
  9.   </property>
  10. </configuration>
  • ozone-site.xml
  1. <configuration>
  2.   <property>
  3.     <name>ozone.om.address</name>
  4.     <value>localhost</value>
  5.   </property>
  6.   <property>
  7.     <name>scm.container.client.max.size</name>
  8.     <value>256</value>
  9.   </property>
  10.   <property>
  11.     <name>scm.container.client.idle.threshold</name>
  12.     <value>10s</value>
  13.   </property>
  14.   <property>
  15.     <name>hdds.ratis.raft.client.rpc.request.timeout</name>
  16.     <value>60s</value>
  17.   </property>
  18.   <property>
  19.     <name>hdds.ratis.raft.client.async.outstanding-requests.max</name>
  20.     <value>32</value>
  21.   </property>
  22.   <property>
  23.     <name>hdds.ratis.raft.client.rpc.watch.request.timeout</name>
  24.     <value>180s</value>
  25.   </property>
  26. </configuration>

Make sure the related config file is on all servers nodes running Alluxio.

Supported Ozone Versions

Currently, the only tested Ozone version with Alluxio is 1.0.0.

Contributed by the Alluxio Community

Ozone UFS integration is contributed and maintained by the Alluxio community. The source code is located here. Feel free submit pull requests to improve the integration and update the documentation here if any information is missing or out of date.