Journey of the development of Ansible Collection? ( Part 2)

I hope you have read the first article for this series.

Journey of the development of Ansible Collection? ( Part 1 )

https://twitter.com/ProgrammersMeme

September 2018

First thing, we need to know about the module and its working, let’s take an example code and try to explain what it does for you?
Sample Code: mapr_service.py
you can treat the ansible module as a python script that does a very specific thing and it is exactly working as a script. Instead of running script manually with python3 <test-module>.py , we going to run the script with Ansible.

Ansible will handle a lot of things for us like

1. Input / Outputs
2. Formatting
3. Compatibility across various OS
4. Input / Output Validation & Verification
5. No Need to create/manage SSH or WINRM or API Sessions

Part 1 ( Header )

#!/usr/bin/python3
# -*- coding: utf-8 -*-

# Copyright 2021 Davinder Pal <dpsangwal@gmail.com>

from __future__ import absolute_import, division, print_function
__metaclass__ = type

It includes various things like
1. Shebang to include Python3
2. Copyright Content ( if applicable )
3. Imports of Some default classes for compatibility.

Part 2 ( Documentation + Example )

# Continue Part 1....
DOCUMENTATION = '''
---
module: mapr_service
version_added: 0.0.1
author: "Davinder Pal (@116davinder)"
short_description: Manage MapR Services by rest api.
description:
   - Manage MapR Services
options:
  username:
    description:
      - username for MapR MCS
    required: true
    type: string
...................
'''

EXAMPLES = '''
- mapr_service:
    username: mapr
...............
'''

We have to write to optional things called documentation and examples code snippets in YAML format in this part.

Part 3 ( Beginning of Code + Inputs )

# Continue Part 2....
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.urls import fetch_url

def main():

    module = AnsibleModule(
        argument_spec=dict(
            username=dict(type='str', required=True),
            password=dict(type='str', required=True, no_log=True),
            service_name=dict(type='str', required=True),
            mcs_url=dict(type='str', required=True),
            mcs_port=dict(type='str', default='8443', required=False),
            state=dict(type='str', required=True),
            validate_certs=dict(type='bool', default='False'),
        )
    )
.....

Now, we need to import Ansible libraries to make our script compatible with Ansible and Ansible based input handling.
AnsibleModule provides various ansible functions/classes which handle inputs and their validation for the script.

Part 4 ( Inbuilt Helper Functions )

# Continue Part 3 .... 
def get_current_hostname():
        cmd = module.get_bin_path('hostname', True)
        rc, out, err = module.run_command(cmd)
        if rc != 0:
            module.fail_json(
                msg="Command failed rc=%d, out=%s, err=%s" % (rc, out, err))
        return out.strip()

we can write helper functions or classes as well inside our script but it will much better to keep them in separate helper libraries later I will explain where to put them and how to use them but for now, you can do them inside the same module. The above function only runs a command in the Linux shell and will return the hostname of a given machine.

Part 5 ( Input handling )

# Continue Part 4.... 
    maprUsername = module.params['username']
    maprPassword = module.params['password']
    serviceName = module.params['service_name'].lower()
    serviceState = module.params['state'].lower()
    mcsUrl = module.params['mcs_url']
    mcsPort = module.params['mcs_port']
    mapr_default_service_state = ['start', 'stop', 'restart']
    # Hack to add basic auth username and password the way fetch_url expects
    module.params['url_username'] = maprUsername
    module.params['url_password'] = maprPassword
if not maprUsername or not maprPassword:
        module.fail_json(msg="Username and Password should be defined")
    elif not serviceName or not serviceState:
        module.fail_json(msg="Service Name & Service State should be defined")
    elif not mcsUrl:
        module.fail_json(msg="MCS Url Should be Defined")
    elif serviceState not in mapr_default_service_state:
        module.fail_json(msg="state should be start/stop/restart only")
    else:

This part is optional, it’s not required at least in this module but I never updated it :D.
1. Variable assignment is just to make code a bit beautiful.
2. If-Else for maprUsername, maprPassword, serviceName, serviceState, mcsUrl, and serviceState should be handled by using Ansible built-in functions like required_if / etc. and that will reduce our codebase.

Part 6 ( Actual Job Code + Outputs )

# Continue Part 5 ....
        host = get_current_hostname()
        url_parameters = "?action=" + serviceState + "&nodes=" + \
            str(host) + "&name=" + serviceName
        complete_url = "https://" + mcsUrl + ":" + mcsPort + \
            "/rest/node/services" + url_parameters
        headers = {'Content-Type': 'application/json'}
        (resp, info) = fetch_url(module,
                                 complete_url,
                                 headers=headers,
                                 method='GET')
        if info['status'] >= 400:
            module.fail_json(msg="Unauthorized Access to MapR Services")
        elif info['status'] == 200:
            body = json.loads(resp.read())
            if body['status'] == 'ERROR':
                module.fail_json(msg=body['errors'][0]['desc'])
            else:
                module.exit_json(changed=True)
        else:
            module.fail_json(
                msg="Unknown Response from MapR API: %s" % resp.read())

Now, the actual magic code that will do our actual job. let’s try to understand it.
First, I get the hostname of the current machine with helper function then I create a string of parameters then API Url and last headers for API call.
Once we have all these values we just use another Ansible Function called fetch_url because it provides wrapper on urllib2/similar library and returns results in JSON as required by Ansible.
At last, we need to parse API response so we can show them in Ansible output as per response like if status is not 200 then we should fail ansible with fail_jsonand print error and response of API call so user can understand what thing failed and fix it.

Part 7 ( Call to Main Function )

# Continue Part 6....
if __name__ == '__main__':
    main()

At last, we should run our main function in a #pythonic way.

Part 8 ( Testing )

- mapr_service:
    username: mapr
    password: mapr
    service_name: nfs
    mcs_url: demo.mapr.com
    mcs_port: 8443
    state: restart
    validate_certs: false

Let’s write a sample task as well to test our code and run against one of our test servers where you can see the response and debug it further.

Please do use -vvvv parameter with ansible so you can see much more details for ansible task and while development of module, set no_log=False so you can see masked fields as well.

Now you know a bit more about Ansible Module Creation Process. this part got a bit longer because it’s easy to understand in one shot than to break it down across multiple articles.

I will write the next part very soon as well about making a real collection. :)