S3 Protocol

Ozone provides S3 compatible REST interface to use the object store data with any S3 compatible tools.

S3 buckets are stored under the /s3v volume.

Getting started

S3 Gateway is a separated component which provides the S3 compatible APIs. It should be started additional to the regular Ozone components.

You can start a docker based cluster, including the S3 gateway from the release package.

Go to the compose/ozone directory, and start the server:

  1. docker-compose up -d --scale datanode=3

You can access the S3 gateway at http://localhost:9878

URL Schema

Ozone S3 gateway supports both the virtual-host-style URL s3 bucket addresses (eg. http://bucketname.host:9878) and the path-style addresses (eg. http://host:9878/bucketname)

By default it uses the path-style addressing. To use virtual host style URLs set your main domain name in your ozone-site.xml:

  1. <property>
  2. <name>ozone.s3g.domain.name</name>
  3. <value>s3g.internal</value>
  4. </property>

Implemented REST endpoints

Operations on S3Gateway service:

EndpointStatus
GET serviceimplemented

Operations on Bucket:

EndpointStatusNotes
GET Bucket (List Objects) Version 2implemented
HEAD Bucketimplemented
DELETE Bucketimplemented
PUT Bucket (Create bucket)implemented
Delete Multiple Objects (POST)implemented

Operation on Objects:

EndpointStatusNotes
PUT Objectimplemented
GET Objectimplemented
Multipart UploadimplementedExcept the listing of the current MultiPartUploads.
DELETE Objectimplemented
HEAD Objectimplemented

Security

If security is not enabled, you can use any AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

If security is enabled, you can get the key and the secret with the ozone s3 getsecret command (*Kerberos based authentication is required).

  1. kinit -kt /etc/security/keytabs/testuser.keytab testuser/scm@EXAMPLE.COM
  2. ozone s3 getsecret
  3. awsAccessKey=testuser/scm@EXAMPLE.COM
  4. awsSecret=c261b6ecabf7d37d5f9ded654b1c724adac9bd9f13e247a235e567e8296d2999

Note: Starting in Ozone 1.4.0, the secret will be shown only once when generated with getsecret. If the secret is lost, the user would have to revokesecret first before regenerating a new secret with getsecret.

Now, you can use the key and the secret to access the S3 endpoint:

  1. export AWS_ACCESS_KEY_ID=testuser/scm@EXAMPLE.COM
  2. export AWS_SECRET_ACCESS_KEY=c261b6ecabf7d37d5f9ded654b1c724adac9bd9f13e247a235e567e8296d2999
  3. aws s3api --endpoint http://localhost:9878 create-bucket --bucket bucket1

To invalidate/revoke the secret, use ozone s3 revokesecret command. Parameter ‘-y’ can be appended to skip the interactive confirmation.

  1. ozone s3 revokesecret
  2. Enter 'y' to confirm S3 secret revocation for 'testuser/scm@EXAMPLE.COM': y
  3. S3 secret revoked.

Ozone Manager administrators can run ozone s3 getsecret and ozone s3 revokesecret command with -u parameter to specify another users.

  1. # Obtained Kerberos TGT for testuser/scm@EXAMPLE.COM with kinit,
  2. # testuser/scm@EXAMPLE.COM is an OM admin.
  3. ozone s3 getsecret -u om/om@EXAMPLE.COM
  4. awsAccessKey=om/om@EXAMPLE.COM
  5. awsSecret=1e9379d0424cce6669b1a501ff14834e46dee004ee868b41a313b49eabcfb68f
  6. ozone s3 revokesecret -u om/om@EXAMPLE.COM -y
  7. S3 secret revoked.

Expose any volume

Ozone has one more element in the name-space hierarchy compared to S3: the volumes. By default, all the buckets of the /s3v volume can be accessed with S3 interface but only the (Ozone) buckets of the /s3v volumes are exposed.

To make any other buckets available with the S3 interface a “symbolic linked” bucket can be created:

  1. ozone sh volume create /s3v
  2. ozone sh volume create /vol1
  3. ozone sh bucket create /vol1/bucket1
  4. ozone sh bucket link /vol1/bucket1 /s3v/common-bucket

This example expose the /vol1/bucket1 Ozone bucket as an S3 compatible common-bucket via the S3 interface.

(Note: the implementation details of the bucket-linking feature can be found in the design doc)

Clients

AWS Cli

aws CLI could be used by specifying the custom REST endpoint.

  1. aws s3api --endpoint http://localhost:9878 create-bucket --bucket buckettest

Or

  1. aws s3 ls --endpoint http://localhost:9878 s3://buckettest

S3 Fuse driver (goofys)

Goofys is a S3 FUSE driver. As Ozone S3 gateway is AWS S3 compatible, it can be used to mount any Ozone buckets as an OS level mounted filesystem.

  1. goofys --endpoint http://localhost:9878 bucket1 /mount/bucket1