Debugging Ansible

19 minute read

Ansible

It is critical to understand how to debug issues with Ansible. These are the common methods used.

Verbose

The first step is generally to extract more verbose information from the tasks so you can see what is going on. This can tell you information like what user is used to perform the task, what SSH arguments are used, and lots more.

First level of verbosity:

ansible-playbook -v

Second level:

ansible-playbook -vv

Third level:

ansible-playbook -vvv

Fourth level - connection debugging:

ansible-playbook -vvvv

Syntax Check (–syntax-check)

Performs a syntax check on your Playbook, but does not execute the Playbook.

ansible-playbook --syntax-check

Check Mode (–check)

Execute a playbook in check mode (also called dry run mode). In this mode Ansible will not make any changes to your host, but will simply report what changes would have been made if the playbook was run without this flag.

$ ansible-playbook --check playbook.yml

NOTE: Be aware that some modules might not support Check Mode, in which case the task will be skipped.

Differences --diff

The --diff option to works great with --check (detailed above) but can also be used by itself. When this flag is supplied and the module supports this, Ansible will report back the changes made or, if used with --check, the changes that would have been made. This is mostly used in modules that manipulate files (i.e. template) but other modules might also show ‘before and after’ information (i.e. user). Since the diff feature produces a large amount of output, it is best used with the --limit argument to check a single host at a time.

$ ansible-playbook foo.yml --check --diff --limit foo.example.com

For example, let us perform a copy of content to a file and run it in Check Mode and Difference Mode.

$ ansible localhost -c local -m copy -a "content='test' dest=~/test.txt" --diff --check

Results in the following output:

--- before
+++ after: /var/folders/sl/1x9_8zx93qdgyj9w1jnnb2980000gn/T/tmpKFp3zb
@@ -0,0 +1 @@
+test
localhost | SUCCESS => {
    "changed": true,
    "dest": "test.txt",
    "src": "/var/folders/sl/1x9_8zx93qdgyj9w1jnnb2980000gn/T/tmpKFp3zb"
}

List Tasks

List all tasks that would be executed. Note that this will only list the tasks that will be executed. It therefore may not list all tasks.

ansible-playbook --list-tasks

TIP: You can also use this command for auto-generated documentation purposes if you have a central server such as Confluence.

List Hosts

List all of the hosts that will be targeted by the playbook.

ansible-playbook --list-hosts

TIP: You can also use this command for auto-generated documentation purposes if you have a central server such as Confluence.

List Tags

List all the tags available for the playbook.

ansible-playbook --list-tags

Tags

Limit the tasks executed by using one or more tags.

ansible-playbook --tags=tag1,tag2,tag3

Limit Hosts

Limit the hosts targeted by the playbook. You might have one host that is failing, so you can target that one.

ansible-playbook --limit

Task Navigation

Restart the playbook at a specific task. This is great to debug a specific task that is failing or to simply test a specific section of your playbook/role. Of course you can simply use tags to do the same thing but then you might forget to remove those debug tags.

ansible-playbook --start-at-task

Step through each task interactively.

ansible-playbook --step

Fail Intentionally

Use the fail module to force Ansible to fail at a certain point in your code. This is helpful in many ways but simply when you are testing and debugging a particular piece of code and you do not want Ansible to continue if or when a certain task is successful.

- fail:
    msg: "fail now and show "

Examine Variables

  • Using the set_fact module
  • Using register
  • Using the debug module

Pause

  • Using the pause module

Retry

When playbooks fail Ansible generates a retry file: playbook.retry.

  • using retry files
  • disable the generation of retry files
  • location of retry files

Remote Files

Generally, when you run an Ansible Playbook, Ansible will push any necessary modules to a temporary location on the managed nodes in order to execute them. After the task is completed, Ansible removes these temporary files from the managed node.

The environment variable ‘ANSIBLE_KEEP_REMOTE_FILES’ enables/disables the cleaning up of the temporary files Ansible used to execute the tasks on the remote.

http://docs.ansible.com/ansible/2.4/config.html#default-keep-remote-files

If something didn’t work as expected during our playbook execution, we can enter into the managed system and view the Ansible module itself and the parameters that were received when executing the Python source file with the explode argument.

Before running the playbook, be sure to set the environment variable to 1 to enable this feature.

To understand this better, let us examine a simple playbook that will fail on our localhost. Again, for the purposes of a simple example we will use localhost but it can be any target you wish.

NOTE: What verbose level is needed to determine the temporary path where the modules exist? That is required information before you can use the explode python feature.

- name: failing playbook
  hosts: localhost

  tasks:
    - name: failing task
      fail:
        msg: "I have some bad news..."

NOTE: Be aware that running Ansible local tasks results in the temporary files being stored at ~/.ansible/tmp. You can customize this using the environment variable ANSIBLE_LOCAL_TEMP.

ls -l ~/.ansible/tmp

Now let’s prepare to debug the playbook by enabling the environment variable and then running the playbook.

$ ANSIBLE_KEEP_REMOTE_FILES=1
$ ansible-playbook local test.yml
<results>

$ cd ~/.ansible/temp
$ cd <most recent temp folder>
$ python fail.py explode
$ ls -l debug_dir
$ cat debug_dir/args

You see the argument that was passed to the fail module.

Source Code

When all else fails, it is sometimes necessary to search within the source code to understand things a bit deeper.

You can download the source code from Github.

Search for specific keywords within the source code.

cat $(find /usr/local/lib/python2.7/site-packages/ansible/ -name *.py) | grep mykeywords

Depending on how you install Ansible, typically the Ansible modules are located at:

$ find /usr/local/lib/python2.7/site-packages/ansible/modules/ -name *.py
$ find /usr/local/lib/python2.7/site-packages/ansible/module_utils/ -name *.py