Librados (Python)

The rados module is a thin Python wrapper for librados.

Installation

To install Python libraries for Ceph, see Getting librados for Python.

Getting Started

You can create your own Ceph client using Python. The following tutorial willshow you how to import the Ceph Python module, connect to a Ceph cluster, andperform object operations as a client.admin user.

Note

To use the Ceph Python bindings, you must have access to arunning Ceph cluster. To set one up quickly, see Getting Started.

  • First, create a Python source file for your Ceph client. ::
    • linenos
    • sudo vim client.py

Import the Module

To use the rados module, import it into your source file.

  1. 1
  1.      import rados

Configure a Cluster Handle

Before connecting to the Ceph Storage Cluster, create a cluster handle. Bydefault, the cluster handle assumes a cluster named ceph (i.e., the defaultfor deployment tools, and our Getting Started guides too), and aclient.admin user name. You may change these defaults to suit your needs.

To connect to the Ceph Storage Cluster, your application needs to know where tofind the Ceph Monitor. Provide this information to your application byspecifying the path to your Ceph configuration file, which contains the locationof the initial Ceph monitors.

  1. 123456
  1.      import rados, sys     #Create Handle Examples.     cluster = rados.Rados(conffile='ceph.conf')     cluster = rados.Rados(conffile=sys.argv[1])     cluster = rados.Rados(conffile = 'ceph.conf', conf = dict (keyring = '/path/to/keyring'))

Ensure that the conffile argument provides the path and file name of yourCeph configuration file. You may use the sys module to avoid hard-coding theCeph configuration path and file name.

Your Python client also requires a client keyring. For this example, we use theclient.admin key by default. If you would like to specify the keyring whencreating the cluster handle, you may use the conf argument. Alternatively,you may specify the keyring path in your Ceph configuration file. For example,you may add something like the following line to you Ceph configuration file:

  1. keyring = /path/to/ceph.client.admin.keyring

For additional details on modifying your configuration via Python, see Configuration.

Connect to the Cluster

Once you have a cluster handle configured, you may connect to the cluster.With a connection to the cluster, you may execute methods that returninformation about the cluster.

  1.  1 2 3 4 5 6 7 8 9101112131415
  1.      import rados, sys     cluster = rados.Rados(conffile='ceph.conf')     print "\nlibrados version: " + str(cluster.version())     print "Will attempt to connect to: " + str(cluster.conf_get('mon initial members'))     cluster.connect()     print "\nCluster ID: " + cluster.get_fsid()     print "\n\nCluster Statistics"     print "=================="     cluster_stats = cluster.get_cluster_stats()     for key, value in cluster_stats.iteritems():             print key, value

By default, Ceph authentication is on. Your application will need to knowthe location of the keyring. The python-ceph module doesn’t have the defaultlocation, so you need to specify the keyring path. The easiest way to specifythe keyring is to add it to the Ceph configuration file. The following Cephconfiguration file example uses the client.admin keyring you generated withceph-deploy.

  1. 123
  1.      [global]     # … elided configuration     keyring=/path/to/keyring/ceph.client.admin.keyring

Manage Pools

When connected to the cluster, the Rados API allows you to manage pools. Youcan list pools, check for the existence of a pool, create a pool and delete apool.

  1.  1 2 3 4 5 6 7 8 91011121314151617181920212223242526
  1.      print "\n\nPool Operations"     print "==============="     print "\nAvailable Pools"     print "————————"     pools = cluster.list_pools()     for pool in pools:             print pool     print "\nCreate 'test' Pool"     print "—————————"     cluster.create_pool('test')     print "\nPool named 'test' exists: " + str(cluster.pool_exists('test'))     print "\nVerify 'test' Pool Exists"     print "————————————-"     pools = cluster.list_pools()     for pool in pools:             print pool     print "\nDelete 'test' Pool"     print "—————————"     cluster.delete_pool('test')     print "\nPool named 'test' exists: " + str(cluster.pool_exists('test'))

Input/Output Context

Reading from and writing to the Ceph Storage Cluster requires an input/outputcontext (ioctx). You can create an ioctx with the open_ioctx() oropen_ioctx2() method of the Rados class. The ioctx_name parameteris the name of the pool and pool_id is the ID of the pool you wish to use.

  1. 1
  1.      ioctx = cluster.open_ioctx('data')

or

  1. 1
  1.      ioctx = cluster.open_ioctx2(pool_id)

Once you have an I/O context, you can read/write objects, extended attributes,and perform a number of other operations. After you complete operations, ensurethat you close the connection. For example:

  1. 12
  1.      print "\nClosing the connection."     ioctx.close()

Writing, Reading and Removing Objects

Once you create an I/O context, you can write objects to the cluster. If youwrite to an object that doesn’t exist, Ceph creates it. If you write to anobject that exists, Ceph overwrites it (except when you specify a range, andthen it only overwrites the range). You may read objects (and object ranges)from the cluster. You may also remove objects from the cluster. For example:

  1. 12345678
  1. print "\nWriting object 'hw' with contents 'Hello World!' to pool 'data'."ioctx.write_full("hw", "Hello World!")print "\n\nContents of object 'hw'\n————————————\n"print ioctx.read("hw")print "\nRemoving object 'hw'"ioctx.remove_object("hw")

Writing and Reading XATTRS

Once you create an object, you can write extended attributes (XATTRs) tothe object and read XATTRs from the object. For example:

  1. 12345
  1. print "\n\nWriting XATTR 'lang' with value 'en_US' to object 'hw'"ioctx.set_xattr("hw", "lang", "en_US")print "\n\nGetting XATTR 'lang' from object 'hw'\n"print ioctx.get_xattr("hw", "lang")

Listing Objects

If you want to examine the list of objects in a pool, you mayretrieve the list of objects and iterate over them with the object iterator.For example:

  1.  1 2 3 4 5 6 7 8 910
  1. object_iterator = ioctx.list_objects()while True :        try :                rados_object = object_iterator.next()                print "Object contents = " + rados_object.read()        except StopIteration :                break

The Object class provides a file-like interface to an object, allowingyou to read and write content and extended attributes. Object operations usingthe I/O context provide additional functionality and asynchronous capabilities.

Cluster Handle API

The Rados class provides an interface into the Ceph Storage Daemon.

Configuration

The Rados class provides methods for getting and setting configurationvalues, reading the Ceph configuration file, and parsing arguments. Youdo not need to be connected to the Ceph Storage Cluster to invoke the followingmethods. See Storage Cluster Configuration for details on settings.

  • Rados.confget(_option)
  • Rados.confset(_option, val)
  • Rados.confread_file(_path=None)
  • Rados.confparse_argv(_args)
  • Rados.conf_parse_argv(self, args)

Parse known arguments from args, and remove; returnedargs contain only those unknown to ceph

  • Rados.version()
  • Rados.version(self)

Get the version number of the librados C library.

  • Returns
  • a tuple of (major, minor, extra) components of thelibrados version

Connection Management

Once you configure your cluster handle, you may connect to the cluster, checkthe cluster fsid, retrieve cluster statistics, and disconnect (shutdown)from the cluster. You may also assert that the cluster handle is in a particularstate (e.g., “configuring”, “connecting”, etc.).

  • Rados.connect(timeout=0)
  • Rados.connect(self, timeout=0)

Connect to the cluster. Use shutdown() to release resources.

  • Rados.shutdown()
  • Rados.shutdown(self)

Disconnects from the cluster. Call this explicitly when aRados.connect()ed object is no longer used.

  • Rados.get_fsid()
  • Rados.get_fsid(self)

Get the fsid of the cluster as a hexadecimal string.

  • Raises

  • Error

  • Returns

  • str - cluster fsid
  • Rados.get_cluster_stats()
  • Rados.get_cluster_stats(self)

Read usage info about the cluster

This tells you total space, space used, space available, and numberof objects. These are not updated immediately when data is written,they are eventually consistent.

  • Returns

dict - contains the following keys:

  1. -

kb (int) - total space

  1. -

kb_used (int) - space used

  1. -

kb_avail (int) - free space available

  1. -

num_objects (int) - number of objects

  • class rados.Rados
    • requirestate(*args_)

    • Checks if the Rados object is in a special state

      • Parameters

      • args – Any number of states to check as separate arguments

      • Raises

      • RadosStateError

Pool Operations

To use pool operation methods, you must connect to the Ceph Storage Clusterfirst. You may list the available pools, create a pool, check to see if a poolexists, and delete a pool.

  • Rados.list_pools()
  • Rados.list_pools(self)

Gets a list of pool names.

  • Returns
  • list - of pool names.
  • Rados.createpool(_pool_name, crush_rule=None)
  • Rados.pool_exists()
  • Rados.deletepool(_pool_name)

CLI Commands

The Ceph CLI command is internally using the following librados Python binding methods.

In order to send a command, choose the correct method and choose the correct target.

  • Rados.moncommand(_self, cmd, inbuf, timeout=0, target=None)
  • Send a command to the mon.

mon_command_target

  • Parameters

    • cmd – JSON formatted string.

    • inbuf – optional string.

    • timeout – This parameter is ignored.

    • target – name of a specific mon. Optional

  • Returns

  • (int ret, string outbuf, string outs)

Example:

  1. >>> import json
  2. >>> c = Rados(conffile='/etc/ceph/ceph.conf')
  3. >>> c.connect()
  4. >>> cmd = json.dumps({"prefix": "osd safe-to-destroy", "ids": ["2"], "format": "json"})
  5. >>> c.mon_command(cmd, b'')
  • Rados.osdcommand(_self, osdid, cmd, inbuf, timeout=0)

  • osd_command(osdid, cmd, inbuf, outbuf, outbuflen, outs, outslen)

    • Returns
    • (int ret, string outbuf, string outs)
  • Rados.mgrcommand(_self, cmd, inbuf, timeout=0, target=None)
    • Returns
    • (int ret, string outbuf, string outs)
  • Rados.pgcommand(_self, pgid, cmd, inbuf, timeout=0)

  • pg_command(pgid, cmd, inbuf, outbuf, outbuflen, outs, outslen)

    • Returns
    • (int ret, string outbuf, string outs)

Input/Output Context API

To write data to and read data from the Ceph Object Store, you must createan Input/Output context (ioctx). The Rados class provides open_ioctx()_and _open_ioctx2() methods. The remaining ioctx operations involveinvoking methods of the Ioctx and other classes.

  • Rados.openioctx(_ioctx_name)
  • Ioctx.require_ioctx_open()
  • Ioctx.require_ioctx_open(self)

Checks if the rados.Ioctx object state is ‘open’

  • Raises
  • IoctxStateError
  • Ioctx.get_stats()
  • Ioctx.get_stats(self)

Get pool usage statistics

  • Returns

dict - contains the following keys:

  1. -

num_bytes (int) - size of pool in bytes

  1. -

num_kb (int) - size of pool in kbytes

  1. -

num_objects (int) - number of objects in the pool

  1. -

num_object_clones (int) - number of object clones

  1. -

num_object_copies (int) - number of object copies

  1. - 
  2.   - <code>num_objects_missing_on_primary</code> (int) - number of objets
  3.   -

missing on primary

  1. -

num_objects_unfound (int) - number of unfound objects

  1. -

num_objects_degraded (int) - number of degraded objects

  1. -

num_rd (int) - bytes read

  1. -

num_rd_kb (int) - kbytes read

  1. -

num_wr (int) - bytes written

  1. -

num_wr_kb (int) - kbytes written

  • Ioctx.get_last_version()
  • Ioctx.get_last_version(self)

Return the version of the last object read or written to.

This exposes the internal version number of the last object read orwritten via this io context

  • Returns
  • version of the last object used
  • Ioctx.close()
  • Ioctx.close(self)

Close a rados.Ioctx object.

This just tells librados that you no longer need to use the io context.It may not be freed immediately if there are pending asynchronousrequests on it, but you should not use an io context again aftercalling this function on it.

Object Operations

The Ceph Storage Cluster stores data as objects. You can read and write objectssynchronously or asynchronously. You can read and write from offsets. An objecthas a name (or key) and data.

  • Ioctx.aiowrite(_object_name, to_write, offset=0, oncomplete=None, onsafe=None)
  • Ioctx.aiowrite_full(_object_name, to_write, oncomplete=None, onsafe=None)
  • Ioctx.aioappend(_object_name, to_append, oncomplete=None, onsafe=None)
  • Ioctx.write(key, data, offset=0)
  • Ioctx.writefull(_key, data)
  • Ioctx.aio_flush()
  • Ioctx.aio_flush(self)

Block until all pending writes in an io context are safe

  • Raises
  • Error
  • Ioctx.setlocator_key(_loc_key)
  • Ioctx.aioread(_object_name, length, offset, oncomplete)
  • Ioctx.read(key, length=8192, offset=0)
  • Ioctx.stat(key)
  • Ioctx.trunc(key, size)
  • Ioctx.removeobject(_key)

Object Extended Attributes

You may set extended attributes (XATTRs) on an object. You can retrieve a listof objects or XATTRs and iterate over them.

  • Ioctx.setxattr(_key, xattr_name, xattr_value)
  • Ioctx.getxattrs(_oid)
  • XattrIterator.next()

  • Get the next xattr on the object

    • Raises

    • StopIteration

    • Returns

    • pair - of name and value of the next Xattr
  • Ioctx.getxattr(_key, xattr_name)
  • Ioctx.rmxattr(_key, xattr_name)

Object Interface

From an I/O context, you can retrieve a list of objects from a pool and iterateover them. The object interface provide makes each object look like a file, andyou may perform synchronous operations on the objects. For asynchronousoperations, you should use the I/O context methods.

  • Ioctx.list_objects()
  • Ioctx.list_objects(self)

Get ObjectIterator on rados.Ioctx object.

  • Returns
  • ObjectIterator
  • ObjectIterator.next()

  • Get the next object name and locator in the pool

    • Raises

    • StopIteration

    • Returns

    • next rados.Ioctx Object
  • Object.read(length = 1024*1024)
  • Object.write(string_to_write)
  • Object.get_xattrs()
  • Object.getxattr(_xattr_name)
  • Object.setxattr(_xattr_name, xattr_value)
  • Object.rmxattr(_xattr_name)
  • Object.stat()
  • Object.remove()