F5 LTM and APM CLI ARP support

[nicolatron]

Hello,
I'm working on the code to get ARP tables from F5 LTM (load balancer) and APM (proxy).
First, I notice there is a difference on how this devices are treated from my production netdot version and the last git version.
On production, on the interface tab i see the physical interface and one entry for each vlan configured with the name "/Common/VLAN_XX" (that Common represents a F5 partition, Common is like "default" where vlans have to be created).

On last git, same device has only the entries for physical interfaces (2 and management on my LTM subject), and on each physical interface a combobox with all the VLANs.

Screenshots attached.
Production "old" 1.0.7

Latest GIT 1.0.7

Another thing, is that physical interfaces in my subject case are in reality members of an LACP, but f5 presents them as separate physical ports (with the same vlans in the combobox as they share configuration), anyway that's an f5 thing, not Netdot.

When I get the ARP from the device (command is "show net arp/show net ndp").
It is presented with relation to VLANs, not physical interfaces, so if I load $iname with the "/Common/VLAN_XX" value, validate_arp will not find a matching interface.

Sample ARP line:
10.1.2.3%4 10.1.2.3%4 00:11:22:33:44:56 /Common/VLAN_15 19 resolved
Any tip on how to map that /Common/VLAN_15 to the interfaces Netdot knows (talking about latest git version)?. The (physical interface are there (interface table) and the VLANs associated to that interfaces exists in the table interfacevlan, but I guess there must be a better approach than directly accessing the database from the CLI F5.pm.
If you notice an ARP tab in the latest git screenshot, that's becouse I cheated and set $iname to '0.9' to see if it load (and IGNORE_IPS_FROM_ARP_NOT_WITHIN_SUBNET => 0 as F5 have nothing in the "IP info" tab). Again, this has to do with how F5 presents information via SNMP, the IP the F5 has on a given vlan, is not presented as another devices do, it can be located on selfip oids, but i guess a fix to this should be addressed on SNMP::Info::Layer3::F5 (if there's something to be fixed, i don't feel qualified to decide on that)).
Thanks

[nicolatron]

Got the interface names with this (from validate arp):
my %devints;
%devints = $self->interfaces;
foreach my $int ( $self->interfaces ){
$devints{$int->number} = $int->name;
}
Now i need to see how can i get the vlans. How can i list all the methods of $self?.

[nicolatron]

Got this sorted out with:

    #Create a hash with vlan=>interface mapping
    my %vlanints;
    foreach my $if ( $self->interfaces ){
		if ( my @ifv = $if->vlans ){
			foreach my $ifv ( @ifv ){
				#This will store only the last physical interface a vlan belongs to, no use to have multiple interfaces in the same LACP storing the same arp tables
				$vlanints{$ifv->vlan->vid} = $if->name;
			}
    	}

}

And later during the ARP entries parsing:
$iname = $vlanints{$vlan};
But this works in the assumption that now Netdot will only list physical interfaces on f5, as opposed to the behaviour I have on my production Netdot, so I'd like to clarify if that is the intended behavior prior to submitting any pull request.

[nicolatron]

Another approach would be to get information about vlans and IPs the F5 has on that vlan in the CLI/F5.pm I'm working on, and populate from there the vlan interfaces with proper network and ips (selfip in f5), something f5 should expose on their SNMP but seems to not do. Then load the arp entries to the appropriate interface, and at the same time get rid of differences (if they exists) between git 1.0.7 and my production 1.0.7.

[nicolatron]

Still wondering why my production netdot get different information about f5 interfaces.
I'm having errors about interface sort. Because of interface number being like "4.48.46.49.48".
Using a test script to retrieve my $interfaces = $f5->interfaces(); directly using SNMP::Info (3.70) both my production and dev machines get this (using print Dumper($interfaces);):
$VAR1 = {
'4.48.46.49.48' => '0.10',
'4.109.103.109.116' => 'mgmt',
'3.48.46.57' => '0.9'
};
But in my production machine netdot seems to have "translated" this weird number information, and interface number looks like:

Number	Name
32	mgmt
64	/Common/http-tunnel
80	/Common/socks-tunnel
96	0.9
112	0.10
128	/Common/VLAN_15
144	/Common/VLAN_19

Whereas in my dev machine it looks like:

Number	Name
3.48.46.57	0.9
4.48.46.49.48	0.10
4.109.103.109.116	mgmt

That kind of number values are causing various errors, and are maybe the root cause that vlan interface are not added on dev machine.

Argument "3.48.46.57" isn't numeric in sort at /opt/netdot/netdot/prod/netdot-1.0.7/lib/Netdot/Model/Device.pm line 5583.
Argument "4.48.46.49.48" isn't numeric in sort at /opt/netdot/netdot/prod/netdot-1.0.7/lib/Netdot/Model/Device.pm line 5583.
Argument "4.109.103.109.116" isn't numeric in sort at /opt/netdot/netdot/prod/netdot-1.0.7/lib/Netdot/Model/Device.pm line 5583.
Use of uninitialized value in numeric lt (<) at /opt/netdot/netdot/prod/netdot-1.0.7/lib/Netdot/Model.pm line 1179.
Use of uninitialized value in numeric lt (<) at /opt/netdot/netdot/prod/netdot-1.0.7/lib/Netdot/Model.pm line 1179.

But i can't find where that number is changed, or why it is not changed on my dev machine.

In my F5.pm i'm trying to add code to populate vlan interfaces, but I think it'll be better to fix the numbering, and maybe interfaces will be created.
F5.txt
Comparing Model.pm on my production and dev machines there is no difference, but there are several changes on Device.pm and Interface.pm (attached diffs).
difss.txt

[nicolatron]

I was checking things, when I run an update on my production machine it sees the same information as the dev one. At least that is consistent with relevant code being the same.

Argument "3.48.46.57" isn't numeric in sort at /opt/netdot/netdot/prod/netdot-1.0.7/lib/Netdot/Model/Device.pm line 5643.
Argument "4.109.103.109.116" isn't numeric in sort at /opt/netdot/netdot/prod/netdot-1.0.7/lib/Netdot/Model/Device.pm line 5643.
Argument "4.48.46.49.48" isn't numeric in sort at /opt/netdot/netdot/prod/netdot-1.0.7/lib/Netdot/Model/Device.pm line 5643.
DEBUG - ov01lbsv01.princast.org: Using existing 020123450008 as base bridge address
DEBUG - ov01lbsv01.princast.org: Old Ifs: 44, New Ifs: 3
WARN - ov01lbsv01.princast.org: new/old interface ratio: 0.07 is below INT_COUNT_THRESHOLD Skipping interface update. Re-discover manually if needed.

So i suspect that if I delete the device, and rediscover it, i'll see exactly the same information as on my dev machine, 3 interfaces with not-numeric numbers. Now i wonder where did that change from numeric numbering and vlan added as interface to actual state and why. The database on production machine is really old. I guess it is due to SNMP::Info version changes (or maybe some other dependencies).
Netdot "make testdeps" checks for SNMP::Info 2.06, which is from 2011. Checking SNMP::Info changelog, F5 support was added on version 3.01 (2013-04-13), with bugfixes for it on version 3.35 (2017-06-28). I'll try to remove SNMP::Info, install that versions and see what i get.

[nicolatron]

Could not find a tarball with 3.01 version, but with 3.31 the behaviour is the same. Also with 3.35 which contains bugfixes for F5 module.
SNMP::Info 3.31
SNMP::Info::device_type() layers:01001110 id:3375 sysDescr:"BIG-IP vCMP Guest : Linux 3.10.0-514.26.2.el7.x86_64 : BIG-IP software release 13.1.0.8, build 0.0.3"
SNMP::Info::specify() - Changed Class to SNMP::Info::Layer3::F5.
SNMP::Info determined this device to fall under subclass : SNMP::Info::Layer3::F5
SNMP::Info::_load_attr i_index : F5-BIGIP-SYSTEM-MIB::sysInterfaceName : .1.3.6.1.4.1.3375.2.1.2.4.1.2.1.1
$VAR1 = {
'4.48.46.49.48' => '0.10',
'3.48.46.57' => '0.9',
'4.109.103.109.116' => 'mgmt'
};
Maybe what changed was the F5 version (we did upgrade some version time ago) and with it it's SNMP engine.

[nicolatron]

Figured out what changed, the vlan interfaces are still there at 1.3.6.1.2.1.2.2.1 (ifEntry) and with nice numeric numbering. It is the addition of F5 support to snmp-info what makes it look for interfaces at 1.3.6.1.4.1.3375.2.1.2.4.1.2.1 (F5-BIGIP-SYSTEM-MIB.sysInterfaceEntry) where only physical interfaces are retrieved (and with the weird numbering).
The "new" way introduced by SNMP/Info/Layer3/F5.pm is nice in a way, in that it retrieves physical interfaces and their vlan interfaces, but as far as I know, in netdot I can't add an IP to a interfaceVlan object. So i think I need a Interface for that.

[nicolatron]

Got it working. However it needs you to don't be using SNMP::Info::Layer3::F5.pm.

So either you can use an SNMP::Info version prior to 3.01, or (this is what I did), use latest and comment out the lines that load F5 module, in file Info.pm, at lines 1690 and 1746 (in my system (debian 10) it is located at /usr/local/share/perl/5.28.1/SNMP/Info.pm).
3375 => 'SNMP::Info::Layer3::F5',
to
# 3375 => 'SNMP::Info::Layer3::F5',
and
I've write to snmp-info mailing list abour the possibility to change the Layer3::F5 module so that it loads all interfaces (including vlans) and ip info from selfips.

In the meantime, and in absent of SNMP::Info::Layer3::F5, SNMP::Info::Layer3 takes care of adding the vlan interfaces and my script loads the ip information it finds for each vlan. If a vlan is not created i also creates it.

But I don't know if you want me do a pull request for this while the snmp-info part is not fixed. Anyway with the SNMP-Info version the netdot installation recommends, that shouldn't be a problem, as it is prior to 3.01. But I wonder if an older system like that might be missing the f5bigip pb file that the underlying CLI libraries uses (located at /usr/local/share/perl/5.28.1/auto/share/dist/Net-CLI-Interact/phrasebook/f5/f5bigip in my system).

Here's the code:
F5.txt

In the absent of a pull request, if anyone is interested, besides copying the file to lib/Netdot/Model/Device/CLI/ and renaming it from F5.txt to F5.pm, there's only a few "manual" changes you need few changes to make this work. In a nutshell:
In etc/Site.conf add 'f5' to the list of mibs, and configure the relevant CLI section. and add
'^bigip.*' => 'F5',
to the FETCH_DEVICE_INFO_VIA_CLI section.
In lib/Netdot/Meta.pm add to %DERIVED_CLASSES hash the line:
'''
'F5' => ['Netdot::Model::Device::CLI::F5', 'Netdot::Model::Device'],
'''
I think that's all.
Edit: One last thing. My F5 are guest running on an F5 host. The sysobject id is iso.3.6.1.4.1.3375.2.1.3.4.94 and that value is missing in current netdisco-mibs, so i manually added it. Values from 89 to 99 in line 357 of F5-BIGIP-SYSTEM-MIB.txt
bigip10200S OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 89 }
bigiq7000 OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 90 }
bigip5250F OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 91 }
bigip12050 OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 92 }
bigip10350N OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 93 }
bigipVcmpGuest OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 94 }
bigipVprC2200 OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 95 }
bigip7055 OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 96 }
bigip7255 OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 97 }
bigip10055 OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 98 }
bigip10255 OBJECT IDENTIFIER ::= { sysDeviceModelOIDs 99 }

And now some pics of how it looks.

Best regards