Ceph File System

The Ceph File System, or CephFS, is a POSIX-compliant file system built ontop of Ceph’s distributed object store, RADOS. CephFS endeavors to providea state-of-the-art, multi-use, highly available, and performant file store fora variety of applications, including traditional use-cases like shared homedirectories, HPC scratch space, and distributed workflow shared storage.

CephFS achieves these goals through the use of some novel architecturalchoices. Notably, file metadata is stored in a separate RADOS pool from filedata and served via a resizable cluster of Metadata Servers, or MDS,which may scale to support higher throughput metadata workloads. Clients ofthe file system have direct access to RADOS for reading and writing file datablocks. For this reason, workloads may linearly scale with the size of theunderlying RADOS object store; that is, there is no gateway or broker mediatingdata I/O for clients.

Access to data is coordinated through the cluster of MDS which serve asauthorities for the state of the distributed metadata cache cooperativelymaintained by clients and MDS. Mutations to metadata are aggregated by each MDSinto a series of efficient writes to a journal on RADOS; no metadata state isstored locally by the MDS. This model allows for coherent and rapidcollaboration between clients within the context of a POSIX file system. CephFS is the subject of numerous academic papers for its novel designs andcontributions to file system research. It is the oldest storage interface inCeph and was once the primary use-case for RADOS. Now it is joined by twoother storage interfaces to form a modern unified storage system: RBD (CephBlock Devices) and RGW (Ceph Object Storage Gateway).

Getting Started with CephFS

For most deployments of Ceph, setting up a CephFS file system is as simple as:

ceph volume create <fs name>

The Ceph Orchestrator will automatically create and configure MDS for yourfile system if the back-end deployment technology supports it (seeOrchestrator deployment table). Otherwise, please deploy MDS manuallyas needed.

Finally, to mount CephFS on your client nodes, setup a FUSE mount or kernel mount. Additionally, acommand-line shell utility is available for interactive access or scripting viathe cephfs-shell.