aci_rest needs to support pagination

[nurseandthenerd]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

When querying an api endpoint with a large number of results the request needs to be paginated when required.

{
"msg": "APIC Error 400: Unable to process the query, result dataset is too big",
"changed": false,
"status": -1,
"error": {
"code": "400",
"text": "Unable to process the query, result dataset is too big"
},
"invocation": {
"module_args": {
"hostname": "apic",
"username": "ansible",
"private_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"certificate_name": "ansible_tower",
"validate_certs": false,
"use_proxy": false,
"method": "get",
"path": "/api/node/class/fvAEPg.json?order-by=fvAEPg.dn&rsp-subtree=full",
"host": "apic",
"output_level": "normal",
"timeout": 30,
"use_ssl": true,
"port": null,
"password": null,
"output_path": null,
"annotation": null,
"owner_key": null,
"owner_tag": null,
"src": null,
"content": null,
"protocol": "https"
}
},
"_ansible_no_log": false
}

Affected Module Name(s):

aci_rest

APIC version and APIC Platform

V 4.2(3l) on-prem

Collection versions

cisco.aci latest

Output/ Error message

Unable to process the query, result dataset is too big

Expected Behavior

all results returned

Actual Behavior

nothing. module returned error from the apic api

Playbook tasks to Reproduce

• name: get fvAEPg
  cisco.aci.aci_rest:    
  hostname: "{{ apic_address }}"    
  username: "{{ apic_user }}"    
  private_key: "{{ apic_key_path }}"    
  certificate_name: "{{ apic_cert_name }}"    
  validate_certs: no    
  use_proxy: no    
  method: get    
  path: /api/node/class/fvAEPg.json?order-by=fvAEPg.dn&rsp-subtree=full
  register: response  changed_when: False

Important Factoids

module was working fine. issue just started happening. there have been no ansible code changes. i think the number of AEPGs just grew so the resultant data returned from the API is just too large now.

References

from https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/3-x/rest-api-config/b_Cisco_APIC_REST_API_Configuration_Guide_3x/b_Cisco_APIC_REST_API_Configuration_Guide_chapter_01.html

Paginating the Results
By adding the page-size operator to the query URI, you can divide the query results into groups (pages) of objects using the following syntax. The operand specifies the number of objects in each group.

page-size = number-of-objects-per-page

By adding the page operator in the query URI, you can specify a single group to be returned using the following syntax. The pages start from number 0.

page = page-number

This example shows you how to specify 15 fault instances per page in descending order, returning only the first page:

GET https://apic-ip-address/api/class/faultInfo.json?order-by=faultInst.severity|desc&page=0&page-size=15

[lhercot]

@nurseandthenerd aci_rest gives you the ability to implement pagination by yourself in your playbook. You have full control on the URL and are able to pass the pagination attributes as needed.

I would also recommend to reduce the amount of child classes your retrieve by not using rsp-subtree=full or filtering it by using rsp-subtree-class

We will consider this request an enhancement request and will prioritize it accordingly.