Ceph Dashboard

Overview

The Ceph Dashboard is a built-in web-based Ceph management and monitoringapplication to administer various aspects and objects of the cluster. It isimplemented as a Ceph Manager Daemon module.

The original Ceph Dashboard that was shipped with Ceph Luminous startedout as a simple read-only view into various run-time information and performancedata of a Ceph cluster. It used a very simple architecture to achieve theoriginal goal. However, there was a growing demand for adding more web-basedmanagement capabilities, to make it easier to administer Ceph for users thatprefer a WebUI over using the command line.

The new Ceph Dashboard module is a replacement of the previous one andadds a built-in web based monitoring and administration application to the CephManager. The architecture and functionality of this new plugin is derived fromand inspired by the openATTIC Ceph management and monitoring tool. The development is actively driven by the teambehind openATTIC at SUSE, with a lot of support fromcompanies like Red Hat and other members of the Cephcommunity.

The dashboard module’s backend code uses the CherryPy framework and a customREST API implementation. The WebUI implementation is based onAngular/TypeScript, merging both functionality from the original dashboard aswell as adding new functionality originally developed for the standalone versionof openATTIC. The Ceph Dashboard module is implemented as a webapplication that visualizes information and statistics about the Ceph clusterusing a web server hosted by ceph-mgr.

Feature Overview

The dashboard provides the following features:

  • Multi-User and Role Management: The dashboard supports multiple useraccounts with different permissions (roles). The user accounts and rolescan be modified on both the command line and via the WebUI.See User and Role Management for details.

  • Single Sign-On (SSO): the dashboard supports authenticationvia an external identity provider using the SAML 2.0 protocol. SeeEnabling Single Sign-On (SSO) for details.

  • SSL/TLS support: All HTTP communication between the web browser and thedashboard is secured via SSL. A self-signed certificate can be created witha built-in command, but it’s also possible to import custom certificatessigned and issued by a CA. See SSL/TLS Support for details.

  • Auditing: the dashboard backend can be configured to log all PUT, POSTand DELETE API requests in the Ceph audit log. See Auditing API Requestsfor instructions on how to enable this feature.

  • Internationalization (I18N): the dashboard can be used in differentlanguages that can be selected at run-time.

Currently, Ceph Dashboard is capable of monitoring and managing the followingaspects of your Ceph cluster:

  • Overall cluster health: Display overall cluster status, performanceand capacity metrics.

  • Embedded Grafana Dashboards: Ceph Dashboard is capable of embeddingGrafana dashboards in many locations, to displayadditional information and performance metrics gathered by thePrometheus Module. See Enabling the Embedding of Grafana Dashboards for details on how toconfigure this functionality.

  • Cluster logs: Display the latest updates to the cluster’s event andaudit log files. Log entries can be filtered by priority, date or keyword.

  • Hosts: Display a list of all hosts associated to the cluster, whichservices are running and which version of Ceph is installed.

  • Performance counters: Display detailed service-specific statistics foreach running service.

  • Monitors: List all MONs, their quorum status, open sessions.

  • Monitoring: Enables creation, re-creation, editing and expiration ofPrometheus’ Silences, lists the alerting configuration of Prometheus andcurrently firing alerts. Also shows notifications for firing alerts. Needsconfiguration.

  • Configuration Editor: Display all available configuration options,their description, type and default values and edit the current values.

  • Pools: List all Ceph pools and their details (e.g. applications,placement groups, replication size, EC profile, CRUSH ruleset, etc.)

  • OSDs: List all OSDs, their status and usage statistics as well asdetailed information like attributes (OSD map), metadata, performancecounters and usage histograms for read/write operations. Mark OSDsup/down/out, purge and reweight OSDs, perform scrub operations, modifyvarious scrub-related configuration options, select different profiles toadjust the level of backfilling activity.

  • iSCSI: List all hosts that run the TCMU runner service, display allimages and their performance characteristics (read/write ops, traffic).Create, modify and delete iSCSI targets (via ceph-iscsi). SeeEnabling iSCSI Management for instructions on how to configure thisfeature.

  • RBD: List all RBD images and their properties (size, objects, features).Create, copy, modify and delete RBD images. Define various I/O or bandwidthlimitation settings on a global, per-pool or per-image level. Create, deleteand rollback snapshots of selected images, protect/unprotect these snapshotsagainst modification. Copy or clone snapshots, flatten cloned images.

  • RBD mirroring: Enable and configure RBD mirroring to a remote Ceph server.Lists all active sync daemons and their status, pools and RBD images includingtheir synchronization state.

  • CephFS: List all active file system clients and associated pools,including their usage statistics.

  • Object Gateway: List all active object gateways and their performancecounters. Display and manage (add/edit/delete) object gateway users and theirdetails (e.g. quotas) as well as the users’ buckets and their details (e.g.owner, quotas). See Enabling the Object Gateway Management Frontend for configurationinstructions.

  • NFS: Manage NFS exports of CephFS file systems and RGW S3 buckets via NFSGanesha. See NFS-Ganesha Management for details on how toenable this functionality.

  • Ceph Manager Modules: Enable and disable all Ceph Manager modules, changethe module-specific configuration settings.

Supported Browsers

Ceph Dashboard is primarily tested and developed using the following webbrowsers:

BrowserVersions
Chrome68+
Firefox61+

While Ceph Dashboard might work in older browsers, we cannot guarantee it andrecommend you to update your browser to the latest version.

Enabling

If you have installed ceph-mgr-dashboard from distribution packages, thepackage management system should have taken care of installing all the requireddependencies.

If you’re installing Ceph from source and want to start the dashboard from yourdevelopment environment, please see the files README.rst and HACKING.rstin directory src/pybind/mgr/dashboard of the source code.

Within a running Ceph cluster, the Ceph Dashboard is enabled with:

  1. $ ceph mgr module enable dashboard

Configuration

SSL/TLS Support

All HTTP connections to the dashboard are secured with SSL/TLS by default.

To get the dashboard up and running quickly, you can generate and install aself-signed certificate using the following built-in command:

  1. $ ceph dashboard create-self-signed-cert

Note that most web browsers will complain about such self-signed certificatesand require explicit confirmation before establishing a secure connection to thedashboard.

To properly secure a deployment and to remove the certificate warning, acertificate that is issued by a certificate authority (CA) should be used.

For example, a key pair can be generated with a command similar to:

  1. $ openssl req -new -nodes -x509 \
  2.   -subj "/O=IT/CN=ceph-mgr-dashboard" -days 3650 \
  3.   -keyout dashboard.key -out dashboard.crt -extensions v3_ca

The dashboard.crt file should then be signed by a CA. Once that is done, youcan enable it for all Ceph manager instances by running the following commands:

  1. $ ceph dashboard set-ssl-certificate -i dashboard.crt
  2. $ ceph dashboard set-ssl-certificate-key -i dashboard.key

If different certificates are desired for each manager instance for some reason,the name of the instance can be included as follows (where $name is the nameof the ceph-mgr instance, usually the hostname):

  1. $ ceph dashboard set-ssl-certificate $name -i dashboard.crt
  2. $ ceph dashboard set-ssl-certificate-key $name -i dashboard.key

SSL can also be disabled by setting this configuration value:

  1. $ ceph config set mgr mgr/dashboard/ssl false

This might be useful if the dashboard will be running behind a proxy which doesnot support SSL for its upstream servers or other situations where SSL is notwanted or required.

Warning

Use caution when disabling SSL as usernames and passwords will be sent to thedashboard unencrypted.

Note

You need to restart the Ceph manager processes manually after changing the SSLcertificate and key. This can be accomplished by either running ceph mgrfail mgr or by disabling and re-enabling the dashboard module (which alsotriggers the manager to respawn itself):

  1. $ ceph mgr module disable dashboard
  2. $ ceph mgr module enable dashboard

Host Name and Port

Like most web applications, dashboard binds to a TCP/IP address and TCP port.

By default, the ceph-mgr daemon hosting the dashboard (i.e., the currentlyactive manager) will bind to TCP port 8443 or 8080 when SSL is disabled.

If no specific address has been configured, the web app will bind to ::,which corresponds to all available IPv4 and IPv6 addresses.

These defaults can be changed via the configuration key facility on acluster-wide level (so they apply to all manager instances) as follows:

  1. $ ceph config set mgr mgr/dashboard/server_addr $IP
  2. $ ceph config set mgr mgr/dashboard/server_port $PORT
  3. $ ceph config set mgr mgr/dashboard/ssl_server_port $PORT

Since each ceph-mgr hosts its own instance of dashboard, it may also benecessary to configure them separately. The IP address and port for a specificmanager instance can be changed with the following commands:

  1. $ ceph config set mgr mgr/dashboard/$name/server_addr $IP
  2. $ ceph config set mgr mgr/dashboard/$name/server_port $PORT
  3. $ ceph config set mgr mgr/dashboard/$name/ssl_server_port $PORT

Replace $name with the ID of the ceph-mgr instance hosting the dashboard webapp.

Note

The command ceph mgr services will show you all endpoints that arecurrently configured. Look for the dashboard key to obtain the URL foraccessing the dashboard.

Username and Password

In order to be able to log in, you need to create a user account and associateit with at least one role. We provide a set of predefined system roles thatyou can use. For more details please refer to the User and Role Managementsection.

To create a user with the administrator role you can use the followingcommands:

  1. $ ceph dashboard ac-user-create <username> <password> administrator

Enabling the Object Gateway Management Frontend

To use the Object Gateway management functionality of the dashboard, you willneed to provide the login credentials of a user with the system flagenabled.

If you do not have a user which shall be used for providing those credentials,you will also need to create one:

  1. $ radosgw-admin user create --uid=<user_id> --display-name=<display_name> \
  2.     --system

Take note of the keys access_key and secret_key in the output of thiscommand.

The credentials of an existing user can also be obtained by usingradosgw-admin:

  1. $ radosgw-admin user info --uid=<user_id>

Finally, provide the credentials to the dashboard:

  1. $ ceph dashboard set-rgw-api-access-key <access_key>
  2. $ ceph dashboard set-rgw-api-secret-key <secret_key>

In a typical default configuration with a single RGW endpoint, this is all youhave to do to get the Object Gateway management functionality working. Thedashboard will try to automatically determine the host and port of the ObjectGateway by obtaining this information from the Ceph Manager’s service map.

If multiple zones are used, it will automatically determine the host within themaster zone group and master zone. This should be sufficient for most setups,but in some circumstances you might want to set the host and port manually:

  1. $ ceph dashboard set-rgw-api-host <host>
  2. $ ceph dashboard set-rgw-api-port <port>

In addition to the settings mentioned so far, the following settings do alsoexist and you may find yourself in the situation that you have to use them:

  1. $ ceph dashboard set-rgw-api-scheme <scheme>  # http or https
  2. $ ceph dashboard set-rgw-api-admin-resource <admin_resource>
  3. $ ceph dashboard set-rgw-api-user-id <user_id>

If you are using a self-signed certificate in your Object Gateway setup, thenyou should disable certificate verification in the dashboard to avoid refusedconnections, e.g. caused by certificates signed by unknown CA or not matchingthe host name:

  1. $ ceph dashboard set-rgw-api-ssl-verify False

If the Object Gateway takes too long to process requests and the dashboard runsinto timeouts, then you can set the timeout value to your needs:

  1. $ ceph dashboard set-rest-requests-timeout <seconds>

The default value is 45 seconds.

Enabling iSCSI Management

The Ceph Dashboard can manage iSCSI targets using the REST API provided by therbd-target-api service of the Ceph iSCSI Gateway. Please make sure that it’sinstalled and enabled on the iSCSI gateways.

Note

The iSCSI management functionality of Ceph Dashboard depends on the latestversion 3 of the ceph-iscsi project.Make sure that your operating system provides the correct version, otherwisethe dashboard won’t enable the management features.

If ceph-iscsi REST API is configured in HTTPS mode and its using a self-signedcertificate, then you need to configure the dashboard to avoid SSL certificateverification when accessing ceph-iscsi API.

To disable API SSL verification run the following command:

  1. $ ceph dashboard set-iscsi-api-ssl-verification false

The available iSCSI gateways must be defined using the following commands:

  1. $ ceph dashboard iscsi-gateway-list
  2. $ ceph dashboard iscsi-gateway-add <scheme>://<username>:<password>@<host>[:port]
  3. $ ceph dashboard iscsi-gateway-rm <gateway_name>

Enabling the Embedding of Grafana Dashboards

Grafana and Prometheus are likely going to be bundled and installed by someorchestration tools along Ceph in the near future, but currently, you will haveto install and configure both manually. After you have installed Prometheus andGrafana on your preferred hosts, proceed with the following steps.

  • Enable the Ceph Exporter which comes as Ceph Manager module by running:
  1. $ ceph mgr module enable prometheus

More details can be found in the documentation of the Prometheus Module.

  • Add the corresponding scrape configuration to Prometheus. This may looklike:
  1. global:
  2.   scrape_interval: 5s
  3.  
  4. scrape_configs:
  5.   - job_name: 'prometheus'
  6.     static_configs:
  7.       - targets: ['localhost:9090']
  8.   - job_name: 'ceph'
  9.     static_configs:
  10.       - targets: ['localhost:9283']
  11.   - job_name: 'node-exporter'
  12.     static_configs:
  13.       - targets: ['localhost:9100']

  • Add Prometheus as data source to Grafana

  • Install the vonage-status-panel and grafana-piechart-panel plugins using:

  1. grafana-cli plugins install vonage-status-panel
  2. grafana-cli plugins install grafana-piechart-panel
  • Add the Dashboards to Grafana:

Dashboards can be added to Grafana by importing dashboard jsons.Following command can be used for downloading json files:

  1. wget https://raw.githubusercontent.com/ceph/ceph/master/monitoring/grafana/dashboards/<Dashboard-name>.json

You can find all the dashboard jsons here .

For Example, for ceph-cluster overview you can use:

  1. wget https://raw.githubusercontent.com/ceph/ceph/master/monitoring/grafana/dashboards/ceph-cluster.json
  • Configure Grafana in /etc/grafana/grafana.ini to adapt anonymous mode:
  1. [auth.anonymous]
  2. enabled = true
  3. org_name = Main Org.
  4. org_role = Viewer

After you have set up Grafana and Prometheus, you will need to configure theconnection information that the Ceph Dashboard will use to access Grafana.

You need to tell the dashboard on which url Grafana instance is running/deployed:

  1. $ ceph dashboard set-grafana-api-url <grafana-server-url>  # default: ''

The format of url is : <protocol>:<IP-address>:<port>

Note

Ceph Dashboard embeds the Grafana dashboards via iframe HTML elements.If Grafana is configured without SSL/TLS support, most browsers will block theembedding of insecure content into a secured web page, if the SSL support inthe dashboard has been enabled (which is the default configuration). If youcan’t see the embedded Grafana dashboards after enabling them as outlinedabove, check your browser’s documentation on how to unblock mixed content.Alternatively, consider enabling SSL/TLS support in Grafana.

If you are using a self-signed certificate in your Grafana setup, then you shoulddisable certificate verification in the dashboard to avoid refused connections,e.g. caused by certificates signed by unknown CA or not matching the host name:

  1. $ ceph dashboard set-grafana-api-ssl-verify False

You can directly access Grafana Instance as well to monitor your cluster.

Enabling Single Sign-On (SSO)

The Ceph Dashboard supports external authentication of users via theSAML 2.0 protocol. You need to createthe user accounts and associate them with the desired roles first, as authorizationis still performed by the Dashboard. However, the authentication process can beperformed by an existing Identity Provider (IdP).

Note

Ceph Dashboard SSO support relies on onelogin’spython-saml library.Please ensure that this library is installed on your system, either by usingyour distribution’s package management or via Python’s pip installer.

To configure SSO on Ceph Dashboard, you should use the following command:

  1. $ ceph dashboard sso setup saml2 <ceph_dashboard_base_url> <idp_metadata> {<idp_username_attribute>} {<idp_entity_id>} {<sp_x_509_cert>} {<sp_private_key>}

Parameters:

  • : Base URL where Ceph Dashboard is accessible (e.g., https://cephdashboard.local)

  • : URL to remote (http://, https://) or local (file://) path or content of the IdP metadata XML (e.g., https://myidp/metadata, file:///home/myuser/metadata.xml).

  • (optional): Attribute that should be used to get the username from the authentication response. Defaults to uid.

  • (optional): Use this when more than one entity id exists on the IdP metadata.

  • / (optional): File path of the certificate that should be used by Ceph Dashboard (Service Provider) for signing and encryption.

Note

The issuer value of SAML requests will follow this pattern: <ceph_dashboard_base_url>/auth/saml2/metadata

To display the current SAML 2.0 configuration, use the following command:

  1. $ ceph dashboard sso show saml2

Note

For more information about onelogin_settings, please check the onelogin documentation.

To disable SSO:

  1. $ ceph dashboard sso disable

To check if SSO is enabled:

  1. $ ceph dashboard sso status

To enable SSO:

  1. $ ceph dashboard sso enable saml2

Enabling Prometheus Alerting

Using Prometheus for monitoring, you have to define alerting rules.To manage them you need to use the Alertmanager.If you are not using the Alertmanager yet, please install it as it’s mandatory inorder to receive and manage alerts from Prometheus.

The Alertmanager capabilities can be consumed by the dashboard in three differentways:

  • Use the notification receiver of the dashboard.

  • Use the Prometheus Alertmanager API.

  • Use both sources simultaneously.

All three methods are going to notify you about alerts. You won’t be notifiedtwice if you use both sources, but you need to consume at least the Alertmanager APIin order to manage silences.

  • Use the notification receiver of the dashboard:

This allows you to get notifications as configured from the Alertmanager.You will get notified inside the dashboard once a notification is send out,but you are not able to manage alerts.

Add the dashboard receiver and the new route to your Alertmanager configuration.This should look like:

  1. route:
  2.   receiver: 'ceph-dashboard'
  3. ...
  4. receivers:
  5.   - name: 'ceph-dashboard'
  6.     webhook_configs:
  7.     - url: '<url-to-dashboard>/api/prometheus_receiver'

Please make sure that the Alertmanager considers your SSL certificate in termsof the dashboard as valid. For more information about the correctconfiguration checkout the documentation.

  • Use the API of Prometheus and the Alertmanager

This allows you to manage alerts and silences. This will enable the “ActiveAlerts”, “All Alerts” as well as the “Silences” tabs in the “Monitoring”section of the “Cluster” menu entry.

Alerts can be sorted by name, job, severity, state and start time.Unfortunately it’s not possible to know when an alertwas sent out through a notification by the Alertmanager based on yourconfiguration, that’s why the dashboard will notify the user on any visiblechange to an alert and will notify the changed alert.

Silences can be sorted by id, creator, status, start, updated and end time.Silences can be created in various ways, it’s also possible to expire them.

  • Create from scratch

  • Based on a selected alert

  • Recreate from expired silence

  • Update a silence (which will recreate and expire it (default Alertmanager behaviour))

To use it, specify the host and port of the Alertmanager server:

  1. $ ceph dashboard set-alertmanager-api-host <alertmanager-host:port>  # default: ''

For example:

  1. $ ceph dashboard set-alertmanager-api-host 'http://localhost:9093'

To be able to see all configured alerts, you will need to configure the URLto the Prometheus API. Using this API, the UI will also help you in verifyingthat a new silence will match a corresponding alert.

  1. $ ceph dashboard set-prometheus-api-host <prometheus-host:port>  # default: ''

For example:

  1. $ ceph dashboard set-prometheus-api-host 'http://localhost:9090'

After setting up the hosts, you have to refresh the dashboard in your browser window.

  • Use both methods

The different behaviors of both methods are configured in a way that theyshould not disturb each other through annoying duplicated notificationspopping up.

Accessing the Dashboard

You can now access the dashboard using your (JavaScript-enabled) web browser, bypointing it to any of the host names or IP addresses and the selected TCP portwhere a manager instance is running: e.g., httpS://<$IP>:<$PORT>/.

You should then be greeted by the dashboard login page, requesting yourpreviously defined username and password. Select the Keep me logged incheckbox if you want to skip the username/password request when accessing thedashboard in the future.

User and Role Management

User Accounts

Ceph Dashboard supports managing multiple user accounts. Each user accountconsists of a username, a password (stored in encrypted form using bcrypt),an optional name, and an optional email address.

User accounts are stored in MON’s configuration database, and are globallyshared across all ceph-mgr instances.

We provide a set of CLI commands to manage user accounts:

  • Show User(s):
  1. $ ceph dashboard ac-user-show [<username>]
  • Create User:
  1. $ ceph dashboard ac-user-create [--force-password] <username> [<password>] [<rolename>] [<name>] [<email>] [--enabled] [<pwd_expiration_date>]
  • Delete User:
  1. $ ceph dashboard ac-user-delete <username>
  • Change Password:
  1. $ ceph dashboard ac-user-set-password [--force-password] <username> <password>
  • Change Password Hash:
  1. $ ceph dashboard ac-user-set-password-hash <username> <hash>

The hash must be a bcrypt hash and salt, e.g. $2b$12$Pt3Vq/rDt2y9glTPSV.VFegiLkQeIpddtkhoFetNApYmIJOY8gau2.This can be used to import users from an external database.

  • Modify User (name, and email):
  1. $ ceph dashboard ac-user-set-info <username> <name> <email>
  • Disable User:
  1. $ ceph dashboard ac-user-disable <username>
  • Enable User:
  1. $ ceph dashboard ac-user-enable <username>

User Roles and Permissions

User accounts are also associated with a set of roles that define whichdashboard functionality can be accessed by the user.

The Dashboard functionality/modules are grouped within a security scope.Security scopes are predefined and static. The current available securityscopes are:

  • hosts: includes all features related to the Hosts menuentry.

  • config-opt: includes all features related to management of Cephconfiguration options.

  • pool: includes all features related to pool management.

  • osd: includes all features related to OSD management.

  • monitor: includes all features related to Monitor management.

  • rbd-image: includes all features related to RBD imagemanagement.

  • rbd-mirroring: includes all features related to RBD-Mirroringmanagement.

  • iscsi: includes all features related to iSCSI management.

  • rgw: includes all features related to Rados Gateway management.

  • cephfs: includes all features related to CephFS management.

  • manager: include all features related to Ceph Managermanagement.

  • log: include all features related to Ceph logs management.

  • grafana: include all features related to Grafana proxy.

  • prometheus: include all features related to Prometheus alert management.

  • dashboard-settings: allows to change dashboard settings.

A role specifies a set of mappings between a security scope and a set ofpermissions. There are four types of permissions:

  • read

  • create

  • update

  • delete

See below for an example of a role specification based on a Python dictionary:

  1. # example of a role
  2. {
  3.   'role': 'my_new_role',
  4.   'description': 'My new role',
  5.   'scopes_permissions': {
  6.     'pool': ['read', 'create'],
  7.     'rbd-image': ['read', 'create', 'update', 'delete']
  8.   }
  9. }

The above role dictates that a user has read and create permissions forfeatures related to pool management, and has full permissions forfeatures related to RBD image management.

The Dashboard already provides a set of predefined roles that we callsystem roles, and can be used right away in a fresh Ceph Dashboardinstallation.

The list of system roles are:

  • administrator: provides full permissions for all security scopes.

  • read-only: provides read permission for all security scopes exceptthe dashboard settings.

  • block-manager: provides full permissions for rbd-image,rbd-mirroring, and iscsi scopes.

  • rgw-manager: provides full permissions for the rgw scope

  • cluster-manager: provides full permissions for the hosts, osd,monitor, manager, and config-opt scopes.

  • pool-manager: provides full permissions for the pool scope.

  • cephfs-manager: provides full permissions for the cephfs scope.

The list of currently available roles can be retrieved by the followingcommand:

  1. $ ceph dashboard ac-role-show [<rolename>]

It is also possible to create new roles using CLI commands. The availablecommands to manage roles are the following:

  • Create Role:
  1. $ ceph dashboard ac-role-create <rolename> [<description>]
  • Delete Role:
  1. $ ceph dashboard ac-role-delete <rolename>
  • Add Scope Permissions to Role:
  1. $ ceph dashboard ac-role-add-scope-perms <rolename> <scopename> <permission> [<permission>...]
  • Delete Scope Permission from Role:
  1. $ ceph dashboard ac-role-del-scope-perms <rolename> <scopename>

To associate roles to users, the following CLI commands are available:

  • Set User Roles:
  1. $ ceph dashboard ac-user-set-roles <username> <rolename> [<rolename>...]
  • Add Roles To User:
  1. $ ceph dashboard ac-user-add-roles <username> <rolename> [<rolename>...]
  • Delete Roles from User:
  1. $ ceph dashboard ac-user-del-roles <username> <rolename> [<rolename>...]

Example of User and Custom Role Creation

In this section we show a full example of the commands that need to be usedin order to create a user account, that should be able to manage RBD images,view and create Ceph pools, and have read-only access to any other scopes.

  • Create the user:
  1. $ ceph dashboard ac-user-create bob mypassword
  • Create role and specify scope permissions:
  1. $ ceph dashboard ac-role-create rbd/pool-manager
  2. $ ceph dashboard ac-role-add-scope-perms rbd/pool-manager rbd-image read create update delete
  3. $ ceph dashboard ac-role-add-scope-perms rbd/pool-manager pool read create
  • Associate roles to user:
  1. $ ceph dashboard ac-user-set-roles bob rbd/pool-manager read-only

Proxy Configuration

In a Ceph cluster with multiple ceph-mgr instances, only the dashboard runningon the currently active ceph-mgr daemon will serve incoming requests. Accessingthe dashboard’s TCP port on any of the other ceph-mgr instances that arecurrently on standby will perform a HTTP redirect (303) to the currently activemanager’s dashboard URL. This way, you can point your browser to any of theceph-mgr instances in order to access the dashboard.

If you want to establish a fixed URL to reach the dashboard or if you don’t wantto allow direct connections to the manager nodes, you could set up a proxy thatautomatically forwards incoming requests to the currently active ceph-mgrinstance.

Configuring a URL Prefix

If you are accessing the dashboard via a reverse proxy configuration,you may wish to service it under a URL prefix. To get the dashboardto use hyperlinks that include your prefix, you can set theurl_prefix setting:

  1. ceph config set mgr mgr/dashboard/url_prefix $PREFIX

so you can access the dashboard at http://$IP:$PORT/$PREFIX/.

Disable the redirection

If the dashboard is behind a load-balancing proxy like HAProxyyou might want to disable the redirection behaviour to prevent situations thatinternal (unresolvable) URL’s are published to the frontend client. Use thefollowing command to get the dashboard to respond with a HTTP error (500 by default)instead of redirecting to the active dashboard:

  1. $ ceph config set mgr mgr/dashboard/standby_behaviour "error"

To reset the setting to the default redirection behaviour, use the following command:

  1. $ ceph config set mgr mgr/dashboard/standby_behaviour "redirect"

Configure the error status code

When the redirection behaviour is disabled, then you want to customize the HTTP statuscode of standby dashboards. To do so you need to run the command:

  1. $ ceph config set mgr mgr/dashboard/standby_error_status_code 503

HAProxy example configuration

Below you will find an example configuration for SSL/TLS pass through usingHAProxy.

Please note that the configuration works under the following conditions.If the dashboard fails over, the front-end client might receive a HTTP redirect(303) response and will be redirected to an unresolvable host. This happens whenthe failover occurs during two HAProxy health checks. In this situation thepreviously active dashboard node will now respond with a 303 which points tothe new active node. To prevent that situation you should consider to disablethe redirection behaviour on standby nodes.

  1. defaults
  2.   log global
  3.   option log-health-checks
  4.   timeout connect 5s
  5.   timeout client 50s
  6.   timeout server 450s
  7.  
  8. frontend dashboard_front
  9.   mode http
  10.   bind *:80
  11.   option httplog
  12.   redirect scheme https code 301 if !{ ssl_fc }
  13.  
  14. frontend dashboard_front_ssl
  15.   mode tcp
  16.   bind *:443
  17.   option tcplog
  18.   default_backend dashboard_back_ssl
  19.  
  20. backend dashboard_back_ssl
  21.   mode tcp
  22.   option httpchk GET /
  23.   http-check expect status 200
  24.   server x <HOST>:<PORT> check-ssl check verify none
  25.   server y <HOST>:<PORT> check-ssl check verify none
  26.   server z <HOST>:<PORT> check-ssl check verify none

Auditing API Requests

The REST API is capable of logging PUT, POST and DELETE requests to the Cephaudit log. This feature is disabled by default, but can be enabled with thefollowing command:

  1. $ ceph dashboard set-audit-api-enabled <true|false>

If enabled, the following parameters are logged per each request:

  • from - The origin of the request, e.g. https://[::1]:44410

  • path - The REST API path, e.g. /api/auth

  • method - e.g. PUT, POST or DELETE

  • user - The name of the user, otherwise ‘None’

The logging of the request payload (the arguments and their values) is enabledby default. Execute the following command to disable this behaviour:

  1. $ ceph dashboard set-audit-api-log-payload <true|false>

A log entry may look like this:

  1. 2018-10-22 15:27:01.302514 mgr.x [INF] [DASHBOARD] from='https://[::ffff:127.0.0.1]:37022' path='/api/rgw/user/klaus' method='PUT' user='admin' params='{"max_buckets": "1000", "display_name": "Klaus Mustermann", "uid": "klaus", "suspended": "0", "email": "klaus.mustermann@ceph.com"}'

NFS-Ganesha Management

Ceph Dashboard can manage NFS Ganesha exports that useCephFS or RadosGW as their backstore.

To enable this feature in Ceph Dashboard there are some assumptions that needto be met regarding the way NFS-Ganesha services are configured.

The dashboard manages NFS-Ganesha config files stored in RADOS objects on the Ceph Cluster.NFS-Ganesha must store part of their configuration in the Ceph cluster.

These configuration files must follow some conventions.Each export block must be stored in its own RADOS object namedexport-<id>, where <id> must match the Export_ID attribute of theexport configuration. Then, for each NFS-Ganesha service daemon there shouldexist a RADOS object named conf-<daemon_id>, where <daemon_id> is anarbitrary string that should uniquely identify the daemon instance (e.g., thehostname where the daemon is running).Each conf-<daemon_id> object contains the RADOS URLs to the exports thatthe NFS-Ganesha daemon should serve. These URLs are of the form:

  1. %url rados://<pool_name>[/<namespace>]/export-<id>

Both the conf-<daemon_id> and export-<id> objects must be stored in thesame RADOS pool/namespace.

Configuring NFS-Ganesha in the Dashboard

To enable the management of NFS-Ganesha exports in Ceph Dashboard, we onlyneed to tell the Dashboard, in which RADOS pool and namespace theconfiguration objects are stored. Then, Ceph Dashboard can access the objectsby following the naming convention described above.

The Dashboard command to configure the NFS-Ganesha configuration objectslocation is:

  1. $ ceph dashboard set-ganesha-clusters-rados-pool-namespace <pool_name>[/<namespace>]

After running the above command, Ceph Dashboard is able to find the NFS-Ganeshaconfiguration objects and we can start manage the exports through the Web UI.

Support for Multiple NFS-Ganesha Clusters

Ceph Dashboard also supports the management of NFS-Ganesha exports belongingto different NFS-Ganesha clusters. An NFS-Ganesha cluster is a group ofNFS-Ganesha service daemons sharing the same exports. Different NFS-Ganeshaclusters are independent and don’t share the exports configuration between eachother.

Each NFS-Ganesha cluster should store its configuration objects in adifferent RADOS pool/namespace to isolate the configuration from each other.

To specify the locations of the configuration of each NFS-Ganesha cluster wecan use the same command as above but with a different value pattern:

  1. $ ceph dashboard set-ganesha-clusters-rados-pool-namespace <cluster_id>:<pool_name>[/<namespace>](,<cluster_id>:<pool_name>[/<namespace>])*

The <cluster_id> is an arbitrary string that should uniquely identify theNFS-Ganesha cluster.

When configuring the Ceph Dashboard with multiple NFS-Ganesha clusters, theWeb UI will automatically allow to choose to which cluster an export belongs.

Plug-ins

Dashboard Plug-ins extend the functionality of the dashboard in a modularand loosely coupled fashion.

Feature Toggles

This plug-in allows to enable or disable some features from the Ceph Dashboardon-demand. When a feature becomes disabled:

  • Its front-end elements (web pages, menu entries, charts, etc.) will become hidden.

  • Its associated REST API endpoints will reject any further requests (404, Not Found Error).

The main purpose of this plug-in is to allow ad-hoc customizations of the workflows exposedby the dashboard. Additionally, it could allow for dynamically enabling experimentalfeatures with minimal configuration burden and no service impact.

The list of features that can be enabled/disabled is:

    • Block (RBD):

      • Image Management: rbd

      • Mirroring: mirroring

      • iSCSI: iscsi

  • Filesystem (Cephfs): cephfs

  • Objects (RGW): rgw (including daemon, user and bucket management).

By default all features come enabled.

To retrieve a list of features and their current statuses:

  1. $ ceph dashboard feature status
  2. Feature 'cephfs': 'enabled'
  3. Feature 'iscsi': 'enabled'
  4. Feature 'mirroring': 'enabled'
  5. Feature 'rbd': 'enabled'
  6. Feature 'rgw': 'enabled'

To enable or disable the status of a single or multiple features:

  1. $ ceph dashboard feature disable iscsi mirroring
  2. Feature 'iscsi': disabled
  3. Feature 'mirroring': disabled

After a feature status has changed, the API REST endpoints immediately respond tothat change, while for the front-end UI elements, it may take up to 20 seconds toreflect it.

Debug

This plugin allows to customize the behaviour of the dashboard according to thedebug mode. It can be enabled, disabled or checked with the following command:

  1. $ ceph dashboard debug status
  2. Debug: 'disabled'
  3. $ ceph dashboard debug enable
  4. Debug: 'enabled'
  5. $ ceph dashboard debug disable
  6. Debug: 'disabled'

By default, it’s disabled. This is the recommended setting for productiondeployments. If required, debug mode can be enabled without need of restarting.Currently, disabled debug mode equals to CherryPy production environment,while when enabled, it uses test_suite defaults (please refer toCherryPy Environments for moredetails).

It also adds request uuid (unique_id) to Cherrypy on versions that don’tsupport this. It additionally prints the unique_id to error responses andlog messages.