Documenting Ceph

The easiest way to help the Ceph project is to contribute to thedocumentation. As the Ceph user base grows and the development pace quickens, anincreasing number of people are updating the documentation and adding newinformation. Even small contributions like fixing spelling errors or clarifyinginstructions will help the Ceph project immensely.

The Ceph documentation source resides in the ceph/doc directory of the Cephrepository, and Python Sphinx renders the source into HTML and manpages. Thehttp://ceph.com/docs link currently displays the master branch by default,but you may view documentation for older branches (e.g., argonaut) or futurebranches (e.g., next) as well as work-in-progress branches by substitutingmaster with the branch name you prefer.

Making Contributions

Making a documentation contribution generally involves the same proceduralsequence as making a code contribution, except that you must build documentationsource instead of compiling program source. The sequence includes the followingsteps:

Get the Source

Ceph documentation lives in the Ceph repository right alongside the Ceph sourcecode under the ceph/doc directory. For details on github and Ceph,see Get Involved in the Ceph Community!.

The most common way to make contributions is to use the Fork and Pullapproach. You must:

  • Install git locally. For Debian/Ubuntu, execute:
  1. sudo apt-get install git

For Fedora, execute:

  1. sudo yum install git

For CentOS/RHEL, execute:

  1. sudo yum install git
  • Ensure your .gitconfig file has your name and email address.
  1. [user]
  2.    email = {your-email-address}
  3.    name = {your-name}

For example:

  1. git config --global user.name "John Doe"
  2. git config --global user.email johndoe@example.com

Ceph organizes documentation into an information architecture primarily by itsmain components.

  • Ceph Storage Cluster: The Ceph Storage Cluster documentation residesunder the doc/rados directory.

  • Ceph Block Device: The Ceph Block Device documentation resides underthe doc/rbd directory.

  • Ceph Object Storage: The Ceph Object Storage documentation resides underthe doc/radosgw directory.

  • Ceph File System: The Ceph File System documentation resides under thedoc/cephfs directory.

  • Installation (Quick): Quick start documentation resides under thedoc/start directory.

  • Installation (Manual): Manual installation documentation resides underthe doc/install directory.

  • Manpage: Manpage source resides under the doc/man directory.

  • Developer: Developer documentation resides under the doc/devdirectory.

  • Images: If you include images such as JPEG or PNG files, you shouldstore them under the doc/images directory.

Select a Branch

When you make small changes to the documentation, such as fixing typographicalerrors or clarifying explanations, use the master branch (default). Youshould also use the master branch when making contributions to features thatare in the current release. master is the most commonly used branch.

  1. git checkout master

When you make changes to documentation that affect an upcoming release, usethe next branch. next is the second most commonly used branch.

  1. git checkout next

When you are making substantial contributions such as new features that are notyet in the current release; if your contribution is related to an issue with atracker ID; or, if you want to see your documentation rendered on the Ceph.comwebsite before it gets merged into the master branch, you should create abranch. To distinguish branches that include only documentation updates, weprepend them with wip-doc by convention, following the formwip-doc-{your-branch-name}. If the branch relates to an issue filed inhttp://tracker.ceph.com/issues, the branch name incorporates the issue number.For example, if a documentation branch is a fix for issue #4000, the branch nameshould be wip-doc-4000 by convention and the relevant tracker URL will behttp://tracker.ceph.com/issues/4000.

Note

Please do not mingle documentation contributions and source codecontributions in a single commit. When you keep documentationcommits separate from source code commits, it simplifies the reviewprocess. We highly recommend that any pull request that adds a feature ora configuration option, should also include a documentation commit,describing the relevant changes/options.

Before you create your branch name, ensure that it doesn’t already exist in thelocal or remote repository.

  1. git branch -a | grep wip-doc-{your-branch-name}

If it doesn’t exist, create your branch:

  1. git checkout -b wip-doc-{your-branch-name}

Make a Change

Modifying a document involves opening a restructuredText file, changingits contents, and saving the changes. See Documentation Style Guide fordetails on syntax requirements.

Adding a document involves creating a new restructuredText file under thedoc directory or its subdirectories and saving the file with a *.rstfile extension. You must also include a reference to the document: a hyperlinkor a table of contents entry. The index.rst file of a top-level directoryusually contains a TOC, where you can add the new file name. All documents musthave a title. See Headings for details.

Your new document doesn’t get tracked by git automatically. When you wantto add the document to the repository, you must use git add {path-to-filename}. For example, from the top level directory of therepository, adding an example.rst file to the rados subdirectory wouldlook like this:

  1. git add doc/rados/example.rst

Deleting a document involves removing it from the repository with git rm {path-to-filename}. For example:

  1. git rm doc/rados/example.rst

You must also remove any reference to a deleted document from other documents.

Build the Source

To build the documentation, navigate to the ceph repository directory:

  1. cd ceph

To build the documentation on Debian/Ubuntu, Fedora, or CentOS/RHEL, execute:

  1. admin/build-doc

To scan for the reachability of external links, execute:

  1. admin/build-doc linkcheck

Executing admin/build-doc will create a build-doc directory under ceph.You may need to create a directory under ceph/build-doc for output of Javadocfiles.

  1. mkdir -p output/html/api/libcephfs-java/javadoc

The build script build-doc will produce an output of errors and warnings.You MUST fix errors in documents you modified before committing a change, and youSHOULD fix warnings that are related to syntax you modified.

Important

You must validate ALL HYPERLINKS. If a hyperlink is broken,it automatically breaks the build!

Once you build the documentation set, you may start an HTTP server athttp://localhost:8080/ to view it:

  1. admin/serve-doc

You can also navigate to build-doc/output to inspect the built documents.There should be an html directory and a man directory containingdocumentation in HTML and manpage formats respectively.

Build the Source (First Time)

Ceph uses Python Sphinx, which is generally distribution agnostic. The firsttime you build Ceph documentation, it will generate a doxygen XML tree, whichis a bit time consuming.

Python Sphinx does have some dependencies that vary across distributions. Thefirst time you build the documentation, the script will notify you if you do nothave the dependencies installed. To run Sphinx and build documentation successfully,the following packages are required:

Debian/Ubuntu

  • gcc

  • python-dev

  • python-pip

  • python-virtualenv

  • python-sphinx

  • libxml2-dev

  • libxslt1-dev

  • doxygen

  • graphviz

  • ant

  • ditaa

Fedora

  • gcc

  • python-devel

  • python-pip

  • python-virtualenv

  • python-docutils

  • python-jinja2

  • python-pygments

  • python-sphinx

  • libxml2-devel

  • libxslt1-devel

  • doxygen

  • graphviz

  • ant

  • ditaa

CentOS/RHEL

  • gcc

  • python-devel

  • python-pip

  • python-virtualenv

  • python-docutils

  • python-jinja2

  • python-pygments

  • python-sphinx

  • libxml2-dev

  • libxslt1-dev

  • doxygen

  • graphviz

  • ant

Install each dependency that is not installed on your host. For Debian/Ubuntudistributions, execute the following:

  1. sudo apt-get install gcc python-dev python-pip python-virtualenv libxml2-dev libxslt-dev doxygen graphviz ant ditaa
  2. sudo apt-get install python-sphinx

For Fedora distributions, execute the following:

  1. sudo yum install gcc python-devel python-pip python-virtualenv libxml2-devel libxslt-devel doxygen graphviz ant
  2. sudo pip install html2text
  3. sudo yum install python-jinja2 python-pygments python-docutils python-sphinx
  4. sudo yum install jericho-html ditaa

For CentOS/RHEL distributions, it is recommended to have epel (ExtraPackages for Enterprise Linux) repository as it provides some extra packageswhich are not available in the default repository. To install epel, executethe following:

  1. sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

For CentOS/RHEL distributions, execute the following:

  1. sudo yum install gcc python-devel python-pip python-virtualenv libxml2-devel libxslt-devel doxygen graphviz ant
  2. sudo pip install html2text

For CentOS/RHEL distributions, the remaining python packages are not available inthe default and epel repositories. So, use http://rpmfind.net/ to find thepackages. Then, download them from a mirror and install them. For example:

  1. wget http://rpmfind.net/linux/centos/7/os/x86_64/Packages/python-jinja2-2.7.2-2.el7.noarch.rpm
  2. sudo yum install python-jinja2-2.7.2-2.el7.noarch.rpm
  3. wget http://rpmfind.net/linux/centos/7/os/x86_64/Packages/python-pygments-1.4-9.el7.noarch.rpm
  4. sudo yum install python-pygments-1.4-9.el7.noarch.rpm
  5. wget http://rpmfind.net/linux/centos/7/os/x86_64/Packages/python-docutils-0.11-0.2.20130715svn7687.el7.noarch.rpm
  6. sudo yum install python-docutils-0.11-0.2.20130715svn7687.el7.noarch.rpm
  7. wget http://rpmfind.net/linux/centos/7/os/x86_64/Packages/python-sphinx-1.1.3-11.el7.noarch.rpm
  8. sudo yum install python-sphinx-1.1.3-11.el7.noarch.rpm

Ceph documentation makes extensive use of ditaa, which is not presently builtfor CentOS/RHEL7. You must install ditaa if you are making changes toditaa diagrams so that you can verify that they render properly before youcommit new or modified ditaa diagrams. You may retrieve compatible requiredpackages for CentOS/RHEL distributions and install them manually. To run ditaaon CentOS/RHEL7, following dependencies are required:

  • jericho-html

  • jai-imageio-core

  • batik

Use http://rpmfind.net/ to find compatible ditaa and the dependencies.Then, download them from a mirror and install them. For example:

  1. wget http://rpmfind.net/linux/fedora/linux/releases/22/Everything/x86_64/os/Packages/j/jericho-html-3.3-4.fc22.noarch.rpm
  2. sudo yum install jericho-html-3.3-4.fc22.noarch.rpm
  3. wget http://rpmfind.net/linux/centos/7/os/x86_64/Packages/jai-imageio-core-1.2-0.14.20100217cvs.el7.noarch.rpm
  4. sudo yum install jai-imageio-core-1.2-0.14.20100217cvs.el7.noarch.rpm
  5. wget http://rpmfind.net/linux/centos/7/os/x86_64/Packages/batik-1.8-0.12.svn1230816.el7.noarch.rpm
  6. sudo yum install batik-1.8-0.12.svn1230816.el7.noarch.rpm
  7. wget http://rpmfind.net/linux/fedora/linux/releases/22/Everything/x86_64/os/Packages/d/ditaa-0.9-13.r74.fc21.noarch.rpm
  8. sudo yum install ditaa-0.9-13.r74.fc21.noarch.rpm

Once you have installed all these packages, build the documentation by followingthe steps given in Build the Source.

Commit the Change

Ceph documentation commits are simple, but follow a strict convention:

  • A commit SHOULD have 1 file per commit (it simplifies rollback). You MAYcommit multiple files with related changes. Unrelated changes SHOULD NOTbe put into the same commit.

  • A commit MUST have a comment.

  • A commit comment MUST be prepended with doc:. (strict)

  • The comment summary MUST be one line only. (strict)

  • Additional comments MAY follow a blank line after the summary,but should be terse.

  • A commit MAY include Fixes: #{bug number}.

  • Commits MUST include Signed-off-by: Firstname Lastname <email>. (strict)

Tip

Follow the foregoing convention particularly where it says(strict) or you will be asked to modify your commit to comply withthis convention.

The following is a common commit comment (preferred):

  1. doc: Fixes a spelling error and a broken hyperlink.
  2.  
  3. Signed-off-by: John Doe <john.doe@gmail.com>

The following comment includes a reference to a bug.

  1. doc: Fixes a spelling error and a broken hyperlink.
  2.  
  3. Fixes: #1234
  4.  
  5. Signed-off-by: John Doe <john.doe@gmail.com>

The following comment includes a terse sentence following the comment summary.There is a carriage return between the summary line and the description:

  1. doc: Added mon setting to monitor config reference
  2.  
  3. Describes 'mon setting', which is a new setting added
  4. to config_opts.h.
  5.  
  6. Signed-off-by: John Doe <john.doe@gmail.com>

To commit changes, execute the following:

  1. git commit -a

An easy way to manage your documentation commits is to use visual tools forgit. For example, gitk provides a graphical interface for viewing therepository history, and git-gui provides a graphical interface for viewingyour uncommitted changes, staging them for commit, committing the changes andpushing them to your forked Ceph repository.

For Debian/Ubuntu, execute:

  1. sudo apt-get install gitk git-gui

For Fedora/CentOS/RHEL, execute:

  1. sudo yum install gitk git-gui

Then, execute:

  1. cd {git-ceph-repo-path}
  2. gitk

Finally, select File->Start git gui to activate the graphical user interface.

Push the Change

Once you have one or more commits, you must push them from the local copy of therepository to github. A graphical tool like git-gui provides a userinterface for pushing to the repository. If you created a branch previously:

  1. git push origin wip-doc-{your-branch-name}

Otherwise:

  1. git push

Make a Pull Request

As noted earlier, you can make documentation contributions using the Fork andPull approach.

Notify Us

After you make a pull request, please email ceph-docs@redhat.com.

Documentation Style Guide

One objective of the Ceph documentation project is to ensure the readability ofthe documentation in both native restructuredText format and its renderedformats such as HTML. Navigate to your Ceph repository and view a document inits native format. You may notice that it is generally as legible in a terminalas it is in its rendered HTML format. Additionally, you may also notice thatdiagrams in ditaa format also render reasonably well in text mode.

  1. less doc/architecture.rst

Review the following style guides to maintain this consistency.

Headings

  • Document Titles: Document titles use the = character overline andunderline with a leading and trailing space on the title text line.See Document Title for details.

  • Section Titles: Section tiles use the = character underline with noleading or trailing spaces for text. Two carriage returns should precede asection title (unless an inline reference precedes it). See Sections fordetails.

  • Subsection Titles: Subsection titles use the _ character underlinewith no leading or trailing spaces for text. Two carriage returns shouldprecede a subsection title (unless an inline reference precedes it).

Text Body

As a general rule, we prefer text to wrap at column 80 so that it is legible ina command line interface without leading or trailing white space. Wherepossible, we prefer to maintain this convention with text, lists, literal text(exceptions allowed), tables, and ditaa graphics.

  • Paragraphs: Paragraphs have a leading and a trailing carriage return,and should be 80 characters wide or less so that the documentation can beread in native format in a command line terminal.

  • Literal Text: To create an example of literal text (e.g., command lineusage), terminate the preceding paragraph with :: or enter a carriagereturn to create an empty line after the preceding paragraph; then, enter:: on a separate line followed by another empty line. Then, begin theliteral text with tab indentation (preferred) or space indentation of 3characters.

  • Indented Text: Indented text such as bullet points(e.g., - some text) may span multiple lines. The text of subsequentlines should begin at the same character position as the text of theindented text (less numbers, bullets, etc.).

Indented text may include literal text examples. Whereas, text indentationshould be done with spaces, literal text examples should be indented withtabs. This convention enables you to add an additional indented paragraphfollowing a literal example by leaving a blank line and beginning thesubsequent paragraph with space indentation.

  • Numbered Lists: Numbered lists should use autonumbering by startinga numbered indent with #. instead of the actual number so thatnumbered paragraphs can be repositioned without requiring manualrenumbering.

  • Code Examples: Ceph supports the use of the.. code-block::<language> role, so that you can add highlighting tosource examples. This is preferred for source code. However, use of thistag will cause autonumbering to restart at 1 if it is used as an examplewithin a numbered list. See Showing code examples for details.

Paragraph Level Markup

The Ceph project uses paragraph level markup to highlight points.

  • Tip: Use the .. tip:: directive to provide additional informationthat assists the reader or steers the reader away from trouble.

  • Note: Use the .. note:: directive to highlight an important point.

  • Important: Use the .. important:: directive to highlight importantrequirements or caveats (e.g., anything that could lead to data loss). Usethis directive sparingly, because it renders in red.

  • Version Added: Use the .. versionadded:: directive for new featuresor configuration settings so that users know the minimum release for usinga feature.

  • Version Changed: Use the .. versionchanged:: directive for changesin usage or configuration settings.

  • Deprecated: Use the .. deprecated:: directive when CLI usage,a feature or a configuration setting is no longer preferred or will bediscontinued.

  • Topic: Use the .. topic:: directive to encapsulate text that isoutside the main flow of the document. See the topic directive foradditional details.

All documents must be linked from another document or a table of contents,otherwise you will receive a warning when building the documentation.

The Ceph project uses the .. toctree:: directive. See The TOC treefor details. When rendering a TOC, consider specifying the :maxdepth:parameter so the rendered TOC is reasonably terse.

Document authors should prefer to use the :ref: syntax where a link targetcontains a specific unique identifier (e.g., .. _unique-target-id:), and areference to the target specifically references the target (e.g.,:ref:unique-target-id) so that if source files are moved or theinformation architecture changes, the links will still work. SeeCross referencing arbitrary locations for details.

Ceph documentation also uses the backtick (accent grave) character followed bythe link text, another backtick and an underscore. Sphinx allows you toincorporate the link destination inline; however, we prefer to use the use the.. _Link Text: ../path convention at the bottom of the document, because itimproves the readability of the document in a command line interface.