Developing Modules

Ansible modules are reusable units of magic that can be used by the Ansible API,or by the ansible or ansible-playbook programs.

See 模块相关 for a list of various ones developed in core.

Modules can be written in any language and are found in the path specifiedby ANSIBLE_LIBRARY or the —module-path command line option.

By default, everything that ships with ansible is pulled from its source tree, butadditional paths can be added.

The directory ”./library”, alongside your top level playbooks, is also automaticallyadded as a search directory.

Should you develop an interesting Ansible module, consider sending a pull request to themodules-extras project. There’s also a corerepo for more established and widely used modules. “Extras” modules may be promoted to core periodically,but there’s no fundamental difference in the end - both ship with ansible, all in one package, regardlessof how you acquire ansible.

Tutorial

Let’s build a very-basic module to get and set the system time. For starters, let’s builda module that just outputs the current time.

We are going to use Python here but any language is possible. Only File I/O and outputting to standardout are required. So, bash, C++, clojure, Python, Ruby, whatever you wantis fine.

Now Python Ansible modules contain some extremely powerful shortcuts (that all the core modules use)but first we are going to build a module the very hard way. The reason we do this is because moduleswritten in any language OTHER than Python are going to have to do exactly this. We’ll show the easyway later.

So, here’s an example. You would never really need to build a module to set the system time,the ‘command’ module could already be used to do this. Though we’re going to make one.

Reading the modules that come with ansible (linked above) is a great way to learn how to writemodules. Keep in mind, though, that some modules in ansible’s source tree are internalisms,so look at service or yum, and don’t stare too close into things like async_wrapper oryou’ll turn to stone. Nobody ever executes async_wrapper directly.

Ok, let’s get going with an example. We’ll use Python. For starters, save this as a file named timetest.py:

  1. #!/usr/bin/python
  2.  
  3. import datetime
  4. import json
  5.  
  6. date = str(datetime.datetime.now())
  7. print json.dumps({
  8.     "time" : date
  9. })

Testing Modules

There’s a useful test script in the source checkout for ansible:

  1. git clone git@github.com:ansible/ansible.git --recursive
  2. source ansible/hacking/env-setup
  3. chmod +x ansible/hacking/test-module

Let’s run the script you just wrote with that:

  1. ansible/hacking/test-module -m ./timetest.py

You should see output that looks something like this:

  1. {u'time': u'2012-03-14 22:13:48.539183'}

If you did not, you might have a typo in your module, so recheck it and try again.

Reading Input

Let’s modify the module to allow setting the current time. We’ll do this by seeingif a key value pair in the form time=<string> is passed in to the module.

Ansible internally saves arguments to an arguments file. So we must read the fileand parse it. The arguments file is just a string, so any form of arguments are legal.Here we’ll do some basic parsing to treat the input as key=value.

The example usage we are trying to achieve to set the time is:

  1. time time="March 14 22:10"

If no time parameter is set, we’ll just leave the time as is and return the current time.

Note

This is obviously an unrealistic idea for a module. You’d most likely justuse the shell module. However, it probably makes a decent tutorial.

Let’s look at the code. Read the comments as we’ll explain as we go. Note that thisis highly verbose because it’s intended as an educational example. You can write modulesa lot shorter than this:

  1. #!/usr/bin/python
  2.  
  3. # import some python modules that we'll use.  These are all
  4. # available in Python's core
  5.  
  6. import datetime
  7. import sys
  8. import json
  9. import os
  10. import shlex
  11.  
  12. # read the argument string from the arguments file
  13. args_file = sys.argv[1]
  14. args_data = file(args_file).read()
  15.  
  16. # for this module, we're going to do key=value style arguments
  17. # this is up to each module to decide what it wants, but all
  18. # core modules besides 'command' and 'shell' take key=value
  19. # so this is highly recommended
  20.  
  21. arguments = shlex.split(args_data)
  22. for arg in arguments:
  23.  
  24.     # ignore any arguments without an equals in it
  25.     if "=" in arg:
  26.  
  27.         (key, value) = arg.split("=")
  28.  
  29.         # if setting the time, the key 'time'
  30.         # will contain the value we want to set the time to
  31.  
  32.         if key == "time":
  33.  
  34.             # now we'll affect the change.  Many modules
  35.             # will strive to be 'idempotent', meaning they
  36.             # will only make changes when the desired state
  37.             # expressed to the module does not match
  38.             # the current state.  Look at 'service'
  39.             # or 'yum' in the main git tree for an example
  40.             # of how that might look.
  41.  
  42.             rc = os.system("date -s \"%s\"" % value)
  43.  
  44.             # always handle all possible errors
  45.             #
  46.             # when returning a failure, include 'failed'
  47.             # in the return data, and explain the failure
  48.             # in 'msg'.  Both of these conventions are
  49.             # required however additional keys and values
  50.             # can be added.
  51.  
  52.             if rc != 0:
  53.                 print json.dumps({
  54.                     "failed" : True,
  55.                     "msg"    : "failed setting the time"
  56.                 })
  57.                 sys.exit(1)
  58.  
  59.             # when things do not fail, we do not
  60.             # have any restrictions on what kinds of
  61.             # data are returned, but it's always a
  62.             # good idea to include whether or not
  63.             # a change was made, as that will allow
  64.             # notifiers to be used in playbooks.
  65.  
  66.             date = str(datetime.datetime.now())
  67.             print json.dumps({
  68.                 "time" : date,
  69.                 "changed" : True
  70.             })
  71.             sys.exit(0)
  72.  
  73. # if no parameters are sent, the module may or
  74. # may not error out, this one will just
  75. # return the time
  76.  
  77. date = str(datetime.datetime.now())
  78. print json.dumps({
  79.     "time" : date
  80. })

Let’s test that module:

  1. ansible/hacking/test-module -m ./time -a time=\"March 14 12:23\"

This should return something like:

  1. {"changed": true, "time": "2012-03-14 12:23:00.000307"}

Module Provided ‘Facts’

The ‘setup’ module that ships with Ansible provides many variables about a system that can be used in playbooksand templates. However, it’s possible to also add your own facts without modifying the system module. To dothis, just have the module return a ansible_facts key, like so, along with other return data:

  1. {
  2.     "changed" : True,
  3.     "rc" : 5,
  4.     "ansible_facts" : {
  5.         "leptons" : 5000,
  6.         "colors" : {
  7.             "red"   : "FF0000",
  8.             "white" : "FFFFFF"
  9.         }
  10.     }
  11. }

These ‘facts’ will be available to all statements called after that module (but not before) in the playbook.A good idea might be make a module called ‘site_facts’ and always call it at the top of each playbook, thoughwe’re always open to improving the selection of core facts in Ansible as well.

Common Module Boilerplate

As mentioned, if you are writing a module in Python, there are some very powerful shortcuts you can use.Modules are still transferred as one file, but an arguments file is no longer needed, so these are notonly shorter in terms of code, they are actually FASTER in terms of execution time.

Rather than mention these here, the best way to learn is to read some of the source of the modules that come with Ansible.

The ‘group’ and ‘user’ modules are reasonably non-trivial and showcase what this looks like.

Key parts include always ending the module file with:

  1. from ansible.module_utils.basic import *
  2. if __name__ == '__main__':
  3.     main()

And instantiating the module class like:

  1. module = AnsibleModule(
  2.     argument_spec = dict(
  3.         state     = dict(default='present', choices=['present', 'absent']),
  4.         name      = dict(required=True),
  5.         enabled   = dict(required=True, choices=BOOLEANS),
  6.         something = dict(aliases=['whatever'])
  7.     )
  8. )

The AnsibleModule provides lots of common code for handling returns, parses your argumentsfor you, and allows you to check inputs.

Successful returns are made like this:

  1. module.exit_json(changed=True, something_else=12345)

And failures are just as simple (where ‘msg’ is a required parameter to explain the error):

  1. module.fail_json(msg="Something fatal happened")

There are also other useful functions in the module class, such as module.sha1(path). Seelib/ansible/module_common.py in the source checkout for implementation details.

Again, modules developed this way are best tested with the hacking/test-module script in the gitsource checkout. Because of the magic involved, this is really the only way the scriptscan function outside of Ansible.

If submitting a module to ansible’s core code, which we encourage, use of the AnsibleModuleclass is required.

Check Mode

New in version 1.1.

Modules may optionally support check mode. If the user runs Ansible in checkmode, the module should try to predict whether changes will occur.

For your module to support check mode, you must pass supports_check_mode=Truewhen instantiating the AnsibleModule object. The AnsibleModule.check_mode attributewill evaluate to True when check mode is enabled. For example:

  1. module = AnsibleModule(
  2.     argument_spec = dict(...),
  3.     supports_check_mode=True
  4. )
  5.  
  6. if module.check_mode:
  7.     # Check if any changes would be made but don't actually make those changes
  8.     module.exit_json(changed=check_if_system_state_would_be_changed())

Remember that, as module developer, you are responsible for ensuring that nosystem state is altered when the user enables check mode.

If your module does not support check mode, when the user runs Ansible in checkmode, your module will simply be skipped.

Common Pitfalls

You should also never do this in a module:

  1. print "some status message"

Because the output is supposed to be valid JSON.

Modules must not output anything on standard error, because the system will mergestandard out with standard error and prevent the JSON from parsing. Capturing standarderror and returning it as a variable in the JSON on standard out is fine, and is, in fact,how the command module is implemented.

If a module returns stderr or otherwise fails to produce valid JSON, the actual outputwill still be shown in Ansible, but the command will not succeed.

Always use the hacking/test-module script when developing modules and it will warnyou about these kind of things.

Conventions/Recommendations

As a reminder from the example code above, here are some basic conventionsand guidelines:

  • If the module is addressing an object, the parameter for that object should be called ‘name’ whenever possible, or accept ‘name’ as an alias.
  • If you have a company module that returns facts specific to your installations, a good name for this module is site_facts.
  • Modules accepting boolean status should generally accept ‘yes’, ‘no’, ‘true’, ‘false’, or anything else a user may likely throw at them. The AnsibleModule common code supports this with “choices=BOOLEANS” and a module.boolean(value) casting function.
  • Include a minimum of dependencies if possible. If there are dependencies, document them at the top of the module file, and have the module raise JSON error messages when the import fails.
  • Modules must be self-contained in one file to be auto-transferred by ansible.
  • If packaging modules in an RPM, they only need to be installed on the control machine and should be dropped into /usr/share/ansible. This is entirely optional and up to you.
  • Modules must output valid JSON only. The toplevel return type must be a hash (dictionary) although they can be nested. Lists or simple scalar values are not supported, though they can be trivially contained inside a dictionary.
  • In the event of failure, a key of ‘failed’ should be included, along with a string explanation in ‘msg’. Modules that raise tracebacks (stacktraces) are generally considered ‘poor’ modules, though Ansible can deal with these returns and will automatically convert anything unparseable into a failed result. If you are using the AnsibleModule common Python code, the ‘failed’ element will be included for you automatically when you call ‘fail_json’.
  • Return codes from modules are not actually not significant, but continue on with 0=success and non-zero=failure for reasons of future proofing.
  • As results from many hosts will be aggregated at once, modules should return only relevant output. Returning the entire contents of a log file is generally bad form.

Documenting Your Module

All modules included in the CORE distribution must have aDOCUMENTATION string. This string MUST be a valid YAML documentwhich conforms to the schema defined below. You may find it easier tostart writing your DOCUMENTATION string in an editor with YAMLsyntax highlighting before you include it in your Python file.

Example

See an example documentation string in the checkout under examples/DOCUMENTATION.yml.

Include it in your module file like this:

  1. #!/usr/bin/python
  2. # Copyright header....
  3.  
  4. DOCUMENTATION = '''
  5. ---
  6. module: modulename
  7. short_description: This is a sentence describing the module
  8. # ... snip ...
  9. '''

The description, and notes fieldssupport formatting with some special macros.

These formatting functions are U(), M(), I(), and C()for URL, module, italic, and constant-width respectively. It is suggestedto use C() for file and option names, and I() when referencingparameters; module names should be specified as M(module).

Examples (which typically contain colons, quotes, etc.) are difficultto format with YAML, so these must bewritten in plain text in an EXAMPLES string within the modulelike this:

  1. EXAMPLES = '''
  2. - action: modulename opt1=arg1 opt2=arg2
  3. '''

The EXAMPLES section, just like the documentation section, is required inall module pull requests for new modules.

Building & Testing

Put your completed module file into the ‘library’ directory and thenrun the command: make webdocs. The new ‘modules.html’ file will bebuilt and appear in the ‘docsite/’ directory.

Tip

If you’re having a problem with the syntax of your YAML you canvalidate it on the YAML Lint website.

Tip

You can set the environment variable ANSIBLE_KEEP_REMOTE_FILES=1 on the controlling host to prevent ansible fromdeleting the remote files so you can debug your module.

Module Paths

If you are having trouble getting your module “found” by ansible, besure it is in the ANSIBLE_LIBRARY environment variable.

If you have a fork of one of the ansible module projects, do something like this:

  1. ANSIBLE_LIBRARY=~/ansible-modules-core:~/ansible-modules-extras

And this will make the items in your fork be loaded ahead of what ships with Ansible. Just be sureto make sure you’re not reporting bugs on versions from your fork!

To be safe, if you’re working on a variant on something in Ansible’s normal distribution, it’s nota bad idea to give it a new name while you are working on it, to be sure you know you’re pullingyour version.

Getting Your Module Into Ansible

High-quality modules with minimal dependenciescan be included in Ansible, but modules (just due to the programmingpreferences of the developers) will need to be implemented in Python and usethe AnsibleModule common code, and should generally use consistent arguments with the rest ofthe program. Stop by the mailing list to inquire about requirements if you like, and submita github pull request to the extras project.Included modules will ship with ansible, and also have a chance to be promoted to ‘core’ status, whichgives them slightly higher development priority (though they’ll work in exactly the same way).

Module checklist

  • The shebang should always be #!/usr/bin/python, this allows ansible_python_interpreter to work

    • Documentation: Make sure it exists
      • required should always be present, be it true or false
      • If required is false you need to document default, even if the default is ‘None’ (which is the default if no parameter is supplied). Make sure default parameter in docs matches default parameter in code.
      • default is not needed for required: true
      • Remove unnecessary doc like aliases: [] or choices: []
      • The version is not a float number and value the current development version
      • Verify that arguments in doc and module spec dict are identical
      • For password / secret arguments no_log=True should be set
      • Requirements should be documented, using the requirements=[] field
      • Author should be set, name and github id at least
      • Made use of U() for urls, C() for files and options, I() for params, M() for modules?
      • GPL 3 License header
      • Does module use check_mode? Could it be modified to use it? Document it
      • Examples: make sure they are reproducible
      • Return: document the return structure of the module
    • Exceptions: The module must handle them. (exceptions are bugs)
      • Give out useful messages on what you were doing and you can add the exception message to that.
      • Avoid catchall exceptions, they are not very useful unless the underlying API gives very good error messages pertaining the attempted action.

  • The module must not use sys.exit() –> use fail_json() from the module object

  • Import custom packages in try/except and handled with fail_json() in main() e.g.:

  1. try:
  2.     import foo
  3.     HAS_LIB=True
  4. except:
  5.     HAS_LIB=False

  • The return structure should be consistent, even if NA/None are used for keys normally returned under other options.

  • Are module actions idempotent? If not document in the descriptions or the notes

  • Import module snippets from ansible.module_utils.basic import * at the bottom, conserves line numbers for debugging.

  • Call your main() from a conditional so that it would be possible totest them in the future example:

  1. if __name__ == '__main__':
  2.     main()

  • Try to normalize parameters with other modules, you can have aliases for when user is more familiar with underlying API name for the option

  • Being pep8 compliant is nice, but not a requirement. Specifically, the 80 column limit now hinders readability more that it improves it

  • Avoid ‘action/command‘, they are imperative and not declarative, there are other ways to express the same thing

  • Sometimes you want to split the module, specially if you are adding a list/info state, you want a _facts version

  • If you are asking ‘how can I have a module execute other modules’ … you want to write a role

  • Return values must be able to be serialized as json via the python stdlibjson library. basic python types (strings, int, dicts, lists, etc) areserializable. A common pitfall is to try returning an object viaexit_json(). Instead, convert the fields you need from the object into thefields of a dictionary and return the dictionary.

  • When fetching URLs, please use either fetch_url or open_url from ansible.module_utils.urlsrather than urllib2; urllib2 does not natively verify TLS certificates and so is insecure for https.

Windows modules checklist

  • Favour native powershell and .net ways of doing things over calls to COM libraries or calls to native executables which may or may not be present in all versions of windows

  • modules are in powershell (.ps1 files) but the docs reside in same name python file (.py)

  • look at ansible/lib/ansible/module_utils/powershell.ps1 for commmon code, avoid duplication

  • start with:

  1. #!powershell
  • then::
  • then::

  • WANT_JSON# POWERSHELL_COMMON

    • Arguments:

      • Try and use state present and state absent like other modules

      • You need to check that all your mandatory args are present:

  1. If ($params.state) {
  2.     $state = $params.state.ToString().ToLower()
  3.     If (($state -ne 'started') -and ($state -ne 'stopped') -and ($state -ne 'restarted')) {
  4.         Fail-Json $result "state is '$state'; must be 'started', 'stopped', or 'restarted'"
  5.     }
  6. }
  1. -

Look at existing modules for more examples of argument checking.

    • Results

      • The result object should allways contain an attribute called changed set to either $true or $false

      • Create your result object like this:

  1. $result = New-Object psobject @{
  2. changed = $false
  3. other_result_attribute = $some_value
  4. };
  5.  
  6. If all is well, exit with a
  7. Exit-Json $result
  1. -

Ensure anything you return, including errors can be converted to json.

  1. -

Be aware that because exception messages could contain almost anything.

  1. -

ConvertTo-Json will fail if it encounters a trailing in a string.

  1. -

If all is not well use Fail-Json to exit.

  • Have you tested for powershell 3.0 and 4.0 compliance?

Deprecating and making module aliases

Starting in 1.8 you can deprecate modules by renaming them with a preceding _, i.e. old_cloud.py to_old_cloud.py, This will keep the module available but hide it from the primary docs and listing.

You can also rename modules and keep an alias to the old name by using a symlink that starts with _.This example allows the stat module to be called with fileinfo, making the following examples equivalent

EXAMPLES = ‘’’ln -s stat.py _fileinfo.pyansible -m stat -a “path=/tmp” localhostansible -m fileinfo -a “path=/tmp” localhost‘’‘

See also