network_resource module failing against specific resource modules

[Gittins]

SUMMARY

Using the ansible.netcommon.network_resource module against a working, from Ansible's point of view, network device (Cisco IOS, CSR1000V) fails when specific resource modules, that are known to be available for the device, are referenced.

The resource modules causing the failure have been identified as:

bgp_address_family
ospf_interfaces
ospfv2
ospfv3

ISSUE TYPE

• Bug Report

COMPONENT NAME

ansible.netcommon.network_resource

ANSIBLE VERSION

ansible --version
ansible [core 2.14.2]
  config file = /home/anthonygittins/venv_ansible/ansible/ansible.cfg
  configured module search path = ['/home/anthonygittins/venv_ansible/ansible/modules', '/home/anthonygittins/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/anthonygittins/venv_ansible/lib/python3.9/site-packages/ansible
  ansible collection location = /home/anthonygittins/venv_ansible/ansible/collections:/home/anthonygittins/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/anthonygittins/venv_ansible/bin/ansible
  python version = 3.9.5 (default, Nov 23 2021, 15:27:38) [GCC 9.3.0] (/home/anthonygittins/venv_ansible/bin/python)
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

ansible-galaxy collection list

# /home/anthonygittins/venv_ansible/lib/python3.9/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    5.2.0  
ansible.netcommon             4.1.0  
ansible.posix                 1.5.1  
ansible.utils                 2.9.0  
ansible.windows               1.13.0 
arista.eos                    6.0.0  
awx.awx                       21.11.0
azure.azcollection            1.14.0 
check_point.mgmt              4.0.0  
chocolatey.chocolatey         1.4.0  
cisco.aci                     2.3.0  
cisco.asa                     4.0.0  
cisco.dnac                    6.6.3  
cisco.intersight              1.0.23 
cisco.ios                     4.3.1  
cisco.iosxr                   4.1.0  
cisco.ise                     2.5.12 
cisco.meraki                  2.15.0 
cisco.mso                     2.2.1  
cisco.nso                     1.0.3  
cisco.nxos                    4.0.1  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.4  
community.aws                 5.2.0  
community.azure               2.0.0  
community.ciscosmb            1.0.5  
community.crypto              2.10.0 
community.digitalocean        1.23.0 
community.dns                 2.5.0  
community.docker              3.4.0  
community.fortios             1.0.0  
community.general             6.3.0  
community.google              1.0.0  
community.grafana             1.5.3  
community.hashi_vault         4.1.0  
community.hrobot              1.7.0  
community.libvirt             1.2.0  
community.mongodb             1.4.2  
community.mysql               3.5.1  
community.network             5.0.0  
community.okd                 2.2.0  
community.postgresql          2.3.2  
community.proxysql            1.5.1  
community.rabbitmq            1.2.3  
community.routeros            2.7.0  
community.sap                 1.0.0  
community.sap_libs            1.4.0  
community.skydive             1.0.0  
community.sops                1.6.0  
community.vmware              3.3.0  
community.windows             1.12.0 
community.zabbix              1.9.1  
containers.podman             1.10.1 
cyberark.conjur               1.2.0  
cyberark.pas                  1.0.17 
dellemc.enterprise_sonic      2.0.0  
dellemc.openmanage            6.3.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
dellemc.powerflex             1.5.0  
dellemc.unity                 1.5.0  
f5networks.f5_modules         1.22.0 
fortinet.fortimanager         2.1.7  
fortinet.fortios              2.2.2  
frr.frr                       2.0.0  
gluster.gluster               1.0.2  
google.cloud                  1.1.2  
grafana.grafana               1.1.0  
hetzner.hcloud                1.9.1  
hpe.nimble                    1.1.4  
ibm.qradar                    2.1.0  
ibm.spectrum_virtualize       1.11.0 
infinidat.infinibox           1.3.12 
infoblox.nios_modules         1.4.1  
inspur.ispim                  1.2.0  
inspur.sm                     2.3.0  
junipernetworks.junos         4.1.0  
kubernetes.core               2.3.2  
lowlydba.sqlserver            1.3.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.22.0
netapp.elementsw              21.7.0 
netapp.ontap                  22.2.0 
netapp.storagegrid            21.11.1
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.4.0  
netbox.netbox                 3.10.0 
ngine_io.cloudstack           2.3.0  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.3  
openstack.cloud               1.10.0 
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   2.4.1  
purestorage.flasharray        1.16.2 
purestorage.flashblade        1.10.0 
purestorage.fusion            1.3.0  
sensu.sensu_go                1.13.2 
splunk.es                     2.1.0  
t_systems_mms.icinga_director 1.32.0 
theforeman.foreman            3.8.0  
vmware.vmware_rest            2.2.0  
vultr.cloud                   1.7.0  
vyos.vyos                     4.0.0  
wti.remote                    1.0.4  

# /home/anthonygittins/venv_ansible/ansible/collections/ansible_collections
Collection        Version
----------------- -------
ansible.netcommon 4.1.0  
ansible.utils     2.9.0  
cisco.ios         4.3.1  
cisco.nxos        4.0.1

CONFIGURATION

$ ansible-config dump --only-changed -t all
ansible-config dump --only-changed -t all
ACTION_WARNINGS(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = True
COLLECTIONS_PATHS(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = ['/home/anthonygittins/venv_ansible/ansible/collections', '/home/anthonygittins/.ansible/collections', '/usr/s>
CONFIG_FILE() = /home/anthonygittins/venv_ansible/ansible/ansible.cfg
DEFAULT_FORKS(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = 5
DEFAULT_GATHERING(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = explicit
DEFAULT_HOST_LIST(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = ['/home/anthonygittins/venv_ansible/ansible/inventory/inventory.yml']
DEFAULT_MODULE_PATH(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = ['/home/anthonygittins/venv_ansible/ansible/modules', '/home/anthonygittins/.ansible/plugins/modules', '/usr>
DEFAULT_TIMEOUT(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = 10
HOST_KEY_CHECKING(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = False
INTERPRETER_PYTHON(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = auto
PERSISTENT_COMMAND_TIMEOUT(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = 60
PERSISTENT_CONNECT_RETRY_TIMEOUT(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = 15
PERSISTENT_CONNECT_TIMEOUT(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = 60

CONNECTION:
==========

paramiko_ssh:
____________
host_key_checking(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = False

ssh:
___
host_key_checking(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = False
timeout(/home/anthonygittins/venv_ansible/ansible/ansible.cfg) = 10

OS / ENVIRONMENT

Targeted network device info:

The below table collates the device's hardware information:

Manufacture	Model	Platform
Cisco	CSR1000V	IOS-XE

The following table collates the device's software information

OS	Image file
17.03.01a	bootflash:packages.conf

STEPS TO REPRODUCE

Run the below playbook against the Cisco Always-On latest IOSXE device:

ciscoao_latest_iosxe:
ansible_host: sandbox-iosxe-latest-1.cisco.com
ansible_connection: ansible.netcommon.network_cli
ansible_network_os: cisco.ios.ios
ansible_user: developer
ansible_password: C1sco12345

---

- name: Network Resource Manager testing Playbook
  hosts: ciscoao_latest_iosxe

  tasks:
    - name: Collect available network resources module information
      ansible.netcommon.network_resource:
      register: network_resource_info

    - name: Debug network_resource_info
      ansible.builtin.debug:
        var: network_resource_info

    - name: Fetch config for working modules
      ansible.netcommon.network_resource:
        name: "{{ item }}"
        state: gathered
      loop:
        - acl_interfaces
        - acls
        - vlans
      ignore_errors: true
      register: working_module_info

    - name: Debug network_resource_info
      ansible.builtin.debug:
        var: working_module_info

    - name: Fetch config for failing modules
      ansible.netcommon.network_resource:
        name: "{{ item }}"
        state: gathered
      loop:
        - bgp_address_family
        - ospf_interfaces
        - ospfv2
        - ospfv3
      ignore_errors: true
      register: failing_module_info

    - name: Debug network_resource_info
      ansible.builtin.debug:
        var: failing_module_info

EXPECTED RESULTS

The first task in the playbook registers all the available modules for the device/host being targeted, with the subsequent task outputting them so we can see what's available.

The "Fetch config for working modules" task contains a partial list of the modules found to be available, which it runs through, as expected. The first 2 modules in the list work without issue and their outputs are collected. The last module "vlans" fails with an appropriate error message, which I have no real issue with (there are no vlans on the device, so this could be the cause).

However, the "Fetch config for failing modules" task contains the list of modules I've isolated down which cause the task to fail with a python based error, when targeting either 1 or all of them. This is the issue that is completely breaking things for me and I'd like help understanding and, if possible, fixing.

ACTUAL RESULTS

The "Fetch config for failing modules" task fails with a python error message as seen below:

ansible-playbook network_resource_manager_test_playbook.yml

PLAY [Network Resource Manager testing Playbook] *******************************

TASK [Collect available network resources module information] ******************
ok: [ciscoao_latest_iosxe]

TASK [Debug network_resource_info] *********************************************
ok: [ciscoao_latest_iosxe] => {
    "network_resource_info": {
        "ansible_connection": "ansible.netcommon.network_cli",
        "ansible_network_os": "cisco.ios.ios",
        "changed": false,
        "failed": false,
        "modules": [
            "acl_interfaces",
            "acls",
            "bgp_address_family",
            "bgp_global",
            "hostname",
            "interfaces",
            "l2_interfaces",
            "l3_interfaces",
            "lacp",
            "lacp_interfaces",
            "lag_interfaces",
            "lldp_global",
            "lldp_interfaces",
            "logging_global",
            "ntp_global",
            "ospf_interfaces",
            "ospfv2",
            "ospfv3",
            "prefix_lists",
            "route_maps",
            "snmp_server",
            "static_routes",
            "vlans"
        ]
    }
}

TASK [Fetch config for working modules] ****************************************
ok: [ciscoao_latest_iosxe] => (item=acl_interfaces)
ok: [ciscoao_latest_iosxe] => (item=acls)
failed: [ciscoao_latest_iosxe] (item=vlans) => {"ansible_connection": "ansible.netcommon.network_cli", "ansible_loop_var": "item", "ansible_network_os": "cisco.ios.ios", "changed": false, "item": "vlans", "msg": "Resource VLAN is not valid for the target device.", "resource_module_name": "cisco.ios.vlans"}
...ignoring

TASK [Debug network_resource_info] *********************************************
ok: [ciscoao_latest_iosxe] => {
    "working_module_info": {
        "changed": false,
        "failed": true,
        "msg": "One or more items failed",
        "results": [
            {
                "ansible_connection": "ansible.netcommon.network_cli",
                "ansible_loop_var": "item",
                "ansible_network_os": "cisco.ios.ios",
                "changed": false,
                "failed": false,
                "gathered": {},
                "invocation": {
                    "module_args": {
                        "config": null,
                        "running_config": null,
                        "state": "gathered"
                    }
                },
                "item": "acl_interfaces",
                "resource_module_name": "cisco.ios.acl_interfaces"
            },
            {
                "ansible_connection": "ansible.netcommon.network_cli",
                "ansible_loop_var": "item",
                "ansible_network_os": "cisco.ios.ios",
                "changed": false,
                "failed": false,
                "gathered": [
                    {
                        "acls": [
                            {
                                "acl_type": "extended",
                                "name": "meraki-fqdn-dns"
                            }
                        ],
                        "afi": "ipv4"
                    }
                ],
                "invocation": {
                    "module_args": {
                        "config": null,
                        "running_config": null,
                        "state": "gathered"
                    }
                },
                "item": "acls",
                "resource_module_name": "cisco.ios.acls"
            },
            {
                "ansible_connection": "ansible.netcommon.network_cli",
                "ansible_loop_var": "item",
                "ansible_network_os": "cisco.ios.ios",
                "changed": false,
                "failed": true,
                "invocation": {
                    "module_args": {
                        "config": null,
                        "running_config": null,
                        "state": "gathered"
                    }
                },
                "item": "vlans",
                "msg": "Resource VLAN is not valid for the target device.",
                "resource_module_name": "cisco.ios.vlans"
            }
        ],
        "skipped": false
    }
}

TASK [Fetch config for failing modules] ****************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'PluginLoader' object has no attribute 'find_plugin_with_name'
fatal: [ciscoao_latest_iosxe]: FAILED! => {"msg": "Unexpected failure during module execution: 'PluginLoader' object has no attribute 'find_plugin_with_name'", "stdout": ""}
...ignoring

TASK [Debug network_resource_info] *********************************************
ok: [ciscoao_latest_iosxe] => {
    "failing_module_info": {
        "exception": "Traceback (most recent call last):\n  File \"/home/anthonygittins/venv_ansible/ansible/collections/ansible_collections/ansible/netcommon/plugins/action/network.py\", line 280, in _find_load_module\n    module = importlib.import_module(context.plugin_resolved_name)\n  File \"/usr/lib64/python3.9/importlib/__init__.py\", line 118, in import_module\n    if name.startswith('.'):\nAttributeError: 'NoneType' object has no attribute 'startswith'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/usr/lib/python3.9/site-packages/ansible/executor/task_executor.py\", line 119, in run\n    item_results = self._run_loop(items)\n  File \"/usr/lib/python3.9/site-packages/ansible/executor/task_executor.py\", line 335, in _run_loop\n    res = self._execute(variables=task_vars)\n  File \"/usr/lib/python3.9/site-packages/ansible/executor/task_executor.py\", line 629, in _execute\n    result = self._handler.run(task_vars=vars_copy)\n  File \"/home/anthonygittins/venv_ansible/ansible/collections/ansible_collections/ansible/netcommon/plugins/action/network_resource.py\", line 76, in run\n    result = self._run_resource_module()\n  File \"/home/anthonygittins/venv_ansible/ansible/collections/ansible_collections/ansible/netcommon/plugins/action/network_resource.py\", line 122, in _run_resource_module\n    result = action.run(task_vars=self._task_vars)\n  File \"/home/anthonygittins/venv_ansible/ansible/collections/ansible_collections/cisco/ios/plugins/action/ios.py\", line 50, in run\n    result = super(ActionModule, self).run(task_vars=task_vars)\n  File \"/home/anthonygittins/venv_ansible/ansible/collections/ansible_collections/ansible/netcommon/plugins/action/network.py\", line 42, in run\n    filename, module = self._find_load_module()\n  File \"/home/anthonygittins/venv_ansible/ansible/collections/ansible_collections/ansible/netcommon/plugins/action/network.py\", line 283, in _find_load_module\n    fullname, filename = mloadr.find_plugin_with_name(\nAttributeError: 'PluginLoader' object has no attribute 'find_plugin_with_name'\n",
        "failed": true,
        "msg": "Unexpected failure during module execution: 'PluginLoader' object has no attribute 'find_plugin_with_name'",
        "stdout": ""
    }
}

PLAY RECAP *********************************************************************
ciscoao_latest_iosxe       : ok=6    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=2

[Gittins]

Had a play at debugging the playbook with "-vvv" and spotted that the working resource modules were being redirected to the actual underlying network provider module name:

TASK [Fetch config for working modules] *****************************************************************************************************
task path: /home/anthonygittins/venv_ansible/ansible/network_resource_manager_test_playbook.yml:29
redirecting (type: modules) cisco.ios.acl_interfaces to cisco.ios.ios_acl_interfaces
ok: [ciscoao_latest_iosxe] => (item=acl_interfaces) => {
    "ansible_connection": "ansible.netcommon.network_cli",
    "ansible_loop_var": "item",
    "ansible_network_os": "cisco.ios.ios",
    "changed": false,
    "gathered": {},
    "invocation": {
        "module_args": {
            "config": null,
            "running_config": null,
            "state": "gathered"
        }
    },
    "item": "acl_interfaces",
    "resource_module_name": "cisco.ios.acl_interfaces"
}

Specifically redirecting cisco.ios.acl_interfaces to cisco.ios.ios_acl_interfaces (the actual name of the module), which got me wondering if the redirecting wasn't working for one or all of the failing resource modules. To check this I changed the resource name provided in the loop to be the actual module it should go to, namely ios_bgp_address_family, like this:

    - name: Fetch config for failing modules
      ansible.netcommon.network_resource:
        # os_name: "{{ network_resource_info.ansible_network_os }}"
        name: "{{ item }}"
        state: gathered
      loop:
        - ios_bgp_address_family
        - ios_ospf_interfaces
        - ios_ospfv2
        - ios_ospfv3
      ignore_errors: true
      register: failing_module_info

And what do you know, the task no longer fails:

TASK [Fetch config for failing modules] *****************************************************************************************************
task path: /home/anthonygittins/venv_ansible/ansible/network_resource_manager_test_playbook.yml:73
ok: [ciscoao_latest_iosxe] => (item=ios_bgp_address_family) => {
    "ansible_connection": "ansible.netcommon.network_cli",
    "ansible_loop_var": "item",
    "ansible_network_os": "cisco.ios.ios",
    "changed": false,
    "gathered": {
        "address_family": [
            {
                "afi": "ipv4"
            }
        ],
        "as_number": "65"
    },
    "invocation": {
        "module_args": {
            "config": null,
            "running_config": null,
            "state": "gathered"
        }
    },
    "item": "ios_bgp_address_family",
    "resource_module_name": "cisco.ios.ios_bgp_address_family"
}
ok: [ciscoao_latest_iosxe] => (item=ios_ospf_interfaces) => {
    "ansible_connection": "ansible.netcommon.network_cli",
    "ansible_loop_var": "item",
    "ansible_network_os": "cisco.ios.ios",
    "changed": false,
    "gathered": [
        {
            "name": "GigabitEthernet1"
        },
        {
            "name": "GigabitEthernet2"
        },
        {
            "name": "GigabitEthernet3"
        }
    ],
    "invocation": {
        "module_args": {
            "config": null,
            "running_config": null,
            "state": "gathered"
        }
    },
    "item": "ios_ospf_interfaces",
    "resource_module_name": "cisco.ios.ios_ospf_interfaces"
}
ok: [ciscoao_latest_iosxe] => (item=ios_ospfv2) => {
    "ansible_connection": "ansible.netcommon.network_cli",
    "ansible_loop_var": "item",
    "ansible_network_os": "cisco.ios.ios",
    "changed": false,
    "gathered": {},
    "invocation": {
        "module_args": {
            "config": null,
            "running_config": null,
            "state": "gathered"
        }
    },
    "item": "ios_ospfv2",
    "resource_module_name": "cisco.ios.ios_ospfv2"
}
ok: [ciscoao_latest_iosxe] => (item=ios_ospfv3) => {
    "ansible_connection": "ansible.netcommon.network_cli",
    "ansible_loop_var": "item",
    "ansible_network_os": "cisco.ios.ios",
    "changed": false,
    "gathered": {},
    "invocation": {
        "module_args": {
            "config": null,
            "running_config": null,
            "state": "gathered"
        }
    },
    "item": "ios_ospfv3",
    "resource_module_name": "cisco.ios.ios_ospfv3"
}

TASK [Debug network_resource_info] **********************************************************************************************************
task path: /home/anthonygittins/venv_ansible/ansible/network_resource_manager_test_playbook.yml:86
ok: [ciscoao_latest_iosxe] => {
    "failing_module_info": {
        "changed": false,
        "msg": "All items completed",
        "results": [
            {
                "ansible_connection": "ansible.netcommon.network_cli",
                "ansible_loop_var": "item",
                "ansible_network_os": "cisco.ios.ios",
                "changed": false,
                "failed": false,
                "gathered": {
                    "address_family": [
                        {
                            "afi": "ipv4"
                        }
                    ],
                    "as_number": "65"
                },
                "invocation": {
                    "module_args": {
                        "config": null,
                        "running_config": null,
                        "state": "gathered"
                    }
                },
                "item": "ios_bgp_address_family",
                "resource_module_name": "cisco.ios.ios_bgp_address_family"
            },
            {
                "ansible_connection": "ansible.netcommon.network_cli",
                "ansible_loop_var": "item",
                "ansible_network_os": "cisco.ios.ios",
                "changed": false,
                "failed": false,
                "gathered": [
                    {
                        "name": "GigabitEthernet1"
                    },
                    {
                        "name": "GigabitEthernet2"
                    },
                    {
                        "name": "GigabitEthernet3"
                    }
                ],
                "invocation": {
                    "module_args": {
                        "config": null,
                        "running_config": null,
                        "state": "gathered"
                    }
                },
                "item": "ios_ospf_interfaces",
                "resource_module_name": "cisco.ios.ios_ospf_interfaces"
            },
            {
                "ansible_connection": "ansible.netcommon.network_cli",
                "ansible_loop_var": "item",
                "ansible_network_os": "cisco.ios.ios",
                "changed": false,
                "failed": false,
                "gathered": {},
                "invocation": {
                    "module_args": {
                        "config": null,
                        "running_config": null,
                        "state": "gathered"
                    }
                },
                "item": "ios_ospfv2",
                "resource_module_name": "cisco.ios.ios_ospfv2"
            },
            {
                "ansible_connection": "ansible.netcommon.network_cli",
                "ansible_loop_var": "item",
                "ansible_network_os": "cisco.ios.ios",
                "changed": false,
                "failed": false,
                "gathered": {},
                "invocation": {
                    "module_args": {
                        "config": null,
                        "running_config": null,
                        "state": "gathered"
                    }
                },
                "item": "ios_ospfv3",
                "resource_module_name": "cisco.ios.ios_ospfv3"
            }
        ],
        "skipped": false
    }
}

PLAY RECAP **********************************************************************************************************************************
ciscoao_latest_iosxe       : ok=6    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=1 

So there seems to be some sort of issue with the mapping and searching of the resource names that are pulled back using the ansible.netcommon.network_resource module.

Hope this helps identify and fix the issue.