ztp error and errorMessage":"403 Forbidden after upgrade AVD 4.4

[seungtaek]

After upgrading AVD and CVP, two problems have appeared.
One is a ZTP error and the other is an issue with change control not working from CVP to the switch.
It seems to be a permissions issue, but how can it be resolved?

version eariler : AVD(3.8.1), CVP(2022.2.1)
version now : AVD(4.4), CVP(2023.2.0)

[ClausHolbechArista]

Re the DHCP change, that seems like some permission issue on the linux part of the CVP server. The only valid user is root but maybe you added some other user in the previous version, that got lost during the upgrade?

Re the Change Control error, that is an authentication error between the CVP server and the device. CVP uses the user that did the last change to the change-control, to authenticate to the device, so please take a look at that. I don't think this relates to the AVD update.

If you don't find a solution to the change control issue, you should open a TAC case on that.

[seungtaek]

Thank you for your replayI
haven't changed any settings regarding ztp I have always used it as root. as you can see below
and no problem to connect using ssh from ansible to cvp

> all:
>   children:
>     CVP:
>       hosts:
>         ztp:
>           ansible_host: 10.12.6.31
>           ansible_user: root
>           ansible_password: cvpadmin

Regarding the change control issue, I have a question about the answer you wrote:
When cvp push a configuration to the switch, does it use terminattr or a specific user?
To log in cvp, I always use just one account local, there are no radius server.

I don't quite understand this part

CVP uses the user that did the last change to the change-control, to authenticate to the device

[ClausHolbechArista]

When CVP pushes config to a device, it will use the TerminAttr connection to tunnel the EAPI requests to EOS. The requests are authorized using the user that is logged in to cloudvision. So it is important that your local user on cloudvision is also a local user on the switch, with the required permissions (network-admin role).

For DHCP role you can try running with -vvv to get more debug output. Maybe we can figure out why it is failing.

[seungtaek]

Thank you for your explanation.
There's something I don't understand, though.
You mentioned that when pushing a configuration to a switch, it uses the user for the CloudVision login. However, initially, there are no users on the switch, so which user executes the initial config? for example just change a hostname first time.
To create a local user on the switch through change control, how CVP connect to the switch? As I understand it, CVP connects to the switch using TerminAttr using token or ingestkey so don't need to any user to connect to switch, am I misunderstanding something?

Regarding the ZTP issue, as you can see from the screenshot above, it seems to be an issue with Paramiko not being able to connect to ZTP with authentication issue.
But when I test using SSH from Ansible machine directly, there doesn't seem to be an authentication problem...

Regards

[ClausHolbechArista]

Re first time config, if the device is in ZTP mode, it will create a temporary local user, and let CVP know which username to use. But if it is not in ZTP mode, CVP will not be able to configure it.
(There are workarounds like -disableaaa on TerminAttr config on the switch, but that should not be used outside a lab).

Please run your ZTP tests with more verbosity. Otherwise we cannot dig deeper.