MapR-FS

This guide describes how to configure Alluxio with MapR-FS as the under storage system.

Compiling Alluxio with MapR Version

Alluxio must be compiled with the correct MapR distribution to integrate with MapR-FS. Here are some values of hadoop.version for different MapR distributions:

Configuring Alluxio for MapR-FS

Once you have compiled Alluxio with the appropriate hadoop.version for your MapR distribution, you may have to configure Alluxio to recognize the MapR-FS scheme and URIs. Alluxio uses the HDFS client to access MapR-FS, and by default is already configured to do so. However, if the configuration has been changed, you can enable the HDFS client to access MapR-FS URIs by adding the URI prefix maprfs:/// to the configuration variable alluxio.underfs.hdfs.prefixes like below:

alluxio.underfs.hdfs.prefixes=hdfs://,maprfs:///

This configuration parameter should be set for all the Alluxio servers (masters, workers). Please read how to configure Alluxio. For Alluxio processes, this parameter can be set in the property file alluxio-site.properties. For more information, please read about configuration of Alluxio with property files.

Configuring Alluxio to use MapR-FS as Under Storage

There are various ways to configure Alluxio to use MapR-FS as under storage. If you want to mount MapR-FS to the root of Alluxio, add the following to conf/alluxio-site.properties:

alluxio.underfs.address=maprfs:///<path in MapR-FS>/

You can also mount a directory in MapR-FS to a directory in the Alluxio namespace.

${ALLUXIO_HOME}/bin/alluxio fs mount /<path in Alluxio>/ maprfs:///<path in MapR-FS>/

Running Alluxio Locally with MapR-FS

Start up Alluxio locally to see that everything works.

./bin/alluxio format
./bin/alluxio-start.sh local

This should start one Alluxio master and one Alluxio worker locally. You can see the master UI at http://localhost:19999.

Run a simple example program:

./bin/alluxio runTests

Visit MapR-FS web UI to verify the files and directories created by Alluxio exist. For this test, you should see files named like: /default_tests_files/BASIC_CACHE_CACHE_THROUGH

Stop Alluxio by running:

./bin/alluxio-stop.sh local