CephFS Quick Start
To use the CephFS Quick Start guide, you must have executed theprocedures in the Storage Cluster Quick Start guide first. Execute thisquick start on the admin host.
Prerequisites
- Verify that you have an appropriate version of the Linux kernel.See OS Recommendations for details.
- lsb_release -a
- uname -r
- On the admin node, use
ceph-deploy
to install Ceph on yourceph-client
node.
- ceph-deploy install ceph-client
Optionally, if you want a FUSE-mounted file system, you would need toinstall
ceph-fuse
package as well.Ensure that the Ceph Storage Cluster is running and in an
active +clean
state.
- ceph -s [-m {monitor-ip-address}] [-k {path/to/ceph.client.admin.keyring}]
Deploy Metadata Server
All metadata operations in CephFS happen via a metadata server, so you need atleast one metadata server. Execute the following to create a metadata server:
- ceph-deploy mds create {ceph-node}
For example:
- ceph-deploy mds create node1
Now, your Ceph cluster would look like this:
Create a File System
You have already created an MDS (Storage Cluster Quick Start) but it will notbecome active until you create some pools and a file system. SeeCreate a Ceph file system.
- ceph osd pool create cephfs_data 32
- ceph osd pool create cephfs_meta 32
- ceph fs new mycephfs cephfs_meta cephfs_data
Note
In case you have multiple Ceph applications and/or have multipleCephFSs on the same cluster, it would be easier to name your pools as<application>.<fs-name>.<pool-name>. In that case, the above pools wouldbe named as cephfs.mycehfs.data and cephfs.mycehfs.meta.
Quick word about Pools and PGs
Replication Number/Pool Size
Since the default replication number/size is 3, you’d need 3 OSDs to getactive+clean
for all PGs. Alternatively, you may change the replicationnumber for the pool to match the number of OSDs:
- ceph osd pool set cephfs_data size {number-of-osds}
- ceph osd pool set cephfs_meta size {number-of-osds}
Usually, setting pg_num
to 32 gives a perfectly healthy cluster. To pickappropriate value for pg_num
, refer Placement Group. You can also usepg_autoscaler plugin instead. Introduced by Nautilus release, it canautomatically increase/decrease value of pg_num
; refer thePlacement Group to find out more about it.
When all OSDs are on the same node…
And, in case you have deployed all of the OSDs on the same node, you would needto create a new CRUSH rule to replicate data across OSDs and set the rule on theCephFS pools, since the default CRUSH rule is to replicate data acrossdifferent nodes:
- ceph osd crush rule create-replicated rule_foo default osd
- ceph osd pool set cephfs_data crush_rule rule_foo
- ceph osd pool set cephfs_meta crush_rule rule_foo
Using Erasure Coded pools
You may also use Erasure Coded pools which can be more effecient andcost-saving since they allow stripping object data across OSDs andreplicating these stripes with encoded redundancy information. The numberof OSDs across which the data is stripped is k and number of replica is m.You’ll need to pick up these values before creating CephFS pools. Thefollowing commands create a erasure code profile, creates a pool that’lluse it and then enables it on the pool:
- ceph osd erasure-code-profile set ec-42-profile k=4 m=2 crush-failure-domain=host crush-device-class=ssd
- ceph osd pool create cephfs_data_ec42 64 erasure ec-42-profile
- ceph osd pool set cephfs_data_ec42 allow_ec_overwrites true
- ceph fs add_data_pool mycephfs cephfs_data_ec42
You can also mark directories so that they are only stored on certain pools:
- setfattr -n ceph.dir.layout -v pool=cephfs_data_ec42 /mnt/mycephfs/logs
This way you can choose the replication strategy for each directory on yourCeph file system.
Note
Erasure Coded pools can not be used for CephFS metadata pools.
Erasure coded pool were introduced in Firefly and could be used directly byCephFS Luminous onwards. Refer this articleby Sage Weil to understand EC, it’s background, limitations and other detailsin Ceph’s context. Read more about Erasure Code here.
Mounting the File System
Using Kernel Driver
The command to mount CephFS using kernel driver looks like this:
- sudo mount -t ceph :{path-to-mounted} {mount-point} -o name={user-name}
- sudo mount -t ceph :/ /mnt/mycephfs -o name=admin # usable version
{path-to-be-mounted}
is the path within CephFS that will be mounted,{mount-point}
is the point in your file system upon which CephFS will bemounted and {user-name}
is the name of CephX user that has theauthorization to mount CephFS on the machine. Following command is theextended form, however these extra details are automatically figured out byby the mount.ceph helper program:
- sudo mount -t ceph {ip-address-of-MON}:{port-number-of-MON}:{path-to-be-mounted} -o name={user-name},secret={secret-key} {mount-point}
If you have multiple file systems on your cluster you would need to passmds_namespace={fs-name}
option to -o
option to the mount
command:
- sudo mount -t ceph :/ /mnt/kcephfs2 -o name=admin,mds_namespace=mycephfs2
Refer mount.ceph man page and Mount CephFS using Kernel Driver to readmore about this.
Using FUSE
To mount CephFS using FUSE (Filesystem in User Space) run:
- sudo ceph-fuse /mnt/mycephfs
To mount a particular directory within CephFS you can use -r
:
- sudo ceph-fuse -r {path-to-be-mounted} /mnt/mycephfs
If you have multiple file systems on your cluster you would need to pass—client_mds_namespace {fs-name}
to the ceph-fuse
command:
- sudo ceph-fuse /mnt/mycephfs2 --client_mds_namespace mycephfs2
Refer ceph-fuse man page and Mount CephFS using FUSE to read more aboutthis.
Note
Mount the CephFS file system on the admin node, not the server node.
Additional Information
See CephFS for additional information. See Troubleshooting if youencounter trouble.