Debugging modules

Debugging (local)

To break into a module running on localhost and step through with the debugger:

  • Set a breakpoint in the module: import pdb; pdb.set_trace()
  • Run the module on the local machine: $ python -m pdb ./my_new_test_module.py ./args.json

Debugging (remote)

To debug a module running on a remote target (i.e. not localhost):

  • On your controller machine (running Ansible) set ANSIBLE_KEEP_REMOTE_FILES=1 to tell Ansible to retain the modules it sends to the remote machine instead of removing them after you playbook runs.
  • Run your playbook targeting the remote machine and specify -vvvv (verbose) to display the remote location Ansible is using for the modules (among many other things).
  • Take note of the directory Ansible used to store modules on the remote host. This directory is usually under the home directory of your ansible_ssh_user, in the form ~/.ansible/tmp/ansible-tmp-….
  • SSH into the remote target after the playbook runs.
  • Navigate to the directory you noted in step 3.
  • Extract the module you want to debug from the zipped file that Ansible sent to the remote host: $ python my_test_module.py explode. Ansible will expand the module into ./debug-dir. You can optionally run the zipped file by specifying python my_test_module.py.
  • Navigate to the debug directory: $ cd debug-dir.
  • Modify or set a breakpoint in ansible_module_my_test_module.py.
  • Ensure that the unzipped module is executable: $ chmod 755 ansible_module_my_test_module.py.
  • Run the unzipped module directly, passing the args file that contains the params that were originally passed: $ ./ansible_module_my_test_module.py args. This approach is good for reproducing behavior as well as modifying the parameters for debugging.

Debugging AnsibleModule-based modules

Tip

If you’re using the hacking/test-module script then most of thisis taken care of for you. If you need to do some debugging of the moduleon the remote machine that the module will actually run on or when themodule is used in a playbook then you may need to use this informationinstead of relying on test-module.

Starting with Ansible 2.1, AnsibleModule-based modules are put together asa zip file consisting of the module file and the various python moduleboilerplate inside of a wrapper script instead of as a single file with all ofthe code concatenated together. Without some help, this can be harder todebug as the file needs to be extracted from the wrapper in order to seewhat’s actually going on in the module. Luckily the wrapper script providessome helper methods to do just that.

If you are using Ansible with the ANSIBLE_KEEP_REMOTE_FILESenvironment variables to keep the remote module file, here’s a sample of howyour debugging session will start:

  1. $ ANSIBLE_KEEP_REMOTE_FILES=1 ansible localhost -m ping -a 'data=debugging_session' -vvv
  2. <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: badger
  3. <127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595 `" && echo "` echo $HOME/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595 `" )'
  4. <127.0.0.1> PUT /var/tmp/tmpjdbJ1w TO /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping
  5. <127.0.0.1> EXEC /bin/sh -c 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping'
  6. localhost | SUCCESS => {
  7.     "changed": false,
  8.     "invocation": {
  9.         "module_args": {
  10.             "data": "debugging_session"
  11.         },
  12.         "module_name": "ping"
  13.     },
  14.     "ping": "debugging_session"
  15. }

Setting ANSIBLE_KEEP_REMOTE_FILES to 1 tells Ansible to keep theremote module files instead of deleting them after the module finishesexecuting. Giving Ansible the -vvv option makes Ansible more verbose.That way it prints the file name of the temporary module file for you to see.

If you want to examine the wrapper file you can. It will show a small pythonscript with a large, base64 encoded string. The string contains the modulethat is going to be executed. Run the wrapper’s explode command to turn thestring into some python files that you can work with:

  1. $ python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping explode
  2. Module expanded into:
  3. /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/debug_dir

When you look into the debug_dir you’ll see a directory structure like this:

  1. ├── ansible_module_ping.py
  2. ├── args
  3. └── ansible
  4.     ├── __init__.py
  5.     └── module_utils
  6.         ├── basic.py
  7.         └── __init__.py
  • ansible_module_ping.py is the code for the module itself. The nameis based on the name of the module with a prefix so that we don’t clash withany other python module names. You can modify this code to see what effectit would have on your module.
  • The args file contains a JSON string. The string is a dictionarycontaining the module arguments and other variables that Ansible passes intothe module to change its behaviour. If you want to modify the parametersthat are passed to the module, this is the file to do it in.
  • The ansible directory contains code fromansible.module_utils that is used by the module. Ansible includesfiles for any ansible.module_utils imports in the module but notany files from any other module. So if your module usesansible.module_utils.url Ansible will include it for you, but ifyour module includes requests then you’ll have to make sure thatthe python requests library is installed on the system before running themodule. You can modify files in this directory if you suspect that themodule is having a problem in some of this boilerplate code rather than inthe module code you have written.

Once you edit the code or arguments in the exploded tree you need some way torun it. There’s a separate wrapper subcommand for this:

  1. $ python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping execute
  2. {"invocation": {"module_args": {"data": "debugging_session"}}, "changed": false, "ping": "debugging_session"}

This subcommand takes care of setting the PYTHONPATH to use the explodeddebug_dir/ansible/module_utils directory and invoking the script usingthe arguments in the args file. You can continue to run it like thisuntil you understand the problem. Then you can copy it back into your realmodule file and test that the real module works via ansible oransible-playbook.

Note

The wrapper provides one more subcommand, excommunicate. Thissubcommand is very similar to execute in that it invokes the explodedmodule on the arguments in the args. The way it does this isdifferent, however. excommunicate imports the mainfunction from the module and then calls that. This makes excommunicateexecute the module in the wrapper’s process. This may be useful forrunning the module under some graphical debuggers but it is very differentfrom the way the module is executed by Ansible itself. Some modules maynot work with excommunicate or may behave differently than when usedwith Ansible normally. Those are not bugs in the module; they’relimitations of excommunicate. Use at your own risk.