fifo_write_vrrp_states_on_reload not always working

[Yncensus]

Describe the bug
Situation:

Two servers in keepalived vrrp instance; to add or remove IPs from virtual_ipaddress_excluded, reloading occurs regularly.
State changes are written to a statefile by notify_fifo_script, if the script exits, it writes "not running" to the file.

Problem:

Sometimes, after a reload, the statefile shows "not running", but keepalived is working and well. If reloading again, the statefile stays the same.

To Reproduce
Any steps necessary to reproduce the behaviour:

• Install keepalived v2.2.8 from unstable branch to Debian 12.2 stable
• Configure the VRRP instance including notify_fifo_script, virtual_ipaddress_excluded and fifo_write_vrrp_states_on_reload
• Reload keepalived with or without new or removed vIPs
• check if the statefile shows "not running"

Expected behavior
fifo_write_vrrp_states_on_reload should result in the current state being written to the statefile on reload, always. Even if some obscure situation results in the statefile failing, a repeated reload should correct the issue.

Keepalived version

Keepalived v2.2.8 (04/04,2023), git commit v2.2.7-154-g292b299e+

Copyright(C) 2001-2023 Alexandre Cassen, <[email protected]>

Built with kernel headers for Linux 6.3.7
Running on Linux 6.1.0-13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.55-1 (2023-09-29)
Distro: Debian GNU/Linux 12 (bookworm)

configure options: --build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-option-checking --disable-silent-rules --libdir=${prefix}/lib/x86_64-linux-gnu --runstatedir=/run --disable-maintainer-mode --disable-dependency-tracking --enable-snmp --enable-sha1 --enable-snmp-rfcv2 --enable-snmp-rfcv3 --enable-dbus --enable-json --enable-bfd --enable-regex --with-init=systemd build_alias=x86_64-linux-gnu CFLAGS=-g -O2  -fstack-protector-strong -Wformat -Werror=format-security LDFLAGS=-Wl,-z,relro CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2

Config options:  NFTABLES LVS REGEX VRRP VRRP_AUTH VRRP_VMAC JSON BFD OLD_CHKSUM_COMPAT SNMP_V3_FOR_V2 SNMP_VRRP SNMP_CHECKER SNMP_RFCV2 SNMP_RFCV3 DBUS INIT=systemd SYSTEMD_NOTIFY

System options:  VSYSLOG MEMFD_CREATE IPV6_MULTICAST_ALL IPV4_DEVCONF LIBNL3 RTA_ENCAP RTA_EXPIRES RTA_NEWDST RTA_PREF FRA_SUPPRESS_PREFIXLEN FRA_SUPPRESS_IFGROUP FRA_TUN_ID RTAX_CC_ALGO RTAX_QUICKACK RTEXT_FILTER_SKIP_STATS FRA_L3MDEV FRA_UID_RANGE RTAX_FASTOPEN_NO_COOKIE RTA_VIA FRA_PROTOCOL FRA_IP_PROTO FRA_SPORT_RANGE FRA_DPORT_RANGE RTA_TTL_PROPAGATE IFA_FLAGS LWTUNNEL_ENCAP_MPLS LWTUNNEL_ENCAP_ILA NET_LINUX_IF_H_COLLISION LIBIPVS_NETLINK IPVS_DEST_ATTR_ADDR_FAMILY IPVS_SYNCD_ATTRIBUTES IPVS_64BIT_STATS IPVS_TUN_TYPE IPVS_TUN_CSUM IPVS_TUN_GRE VRRP_IPVLAN IFLA_LINK_NETNSID GLOB_BRACE GLOB_ALTDIRFUNC INET6_ADDR_GEN_MODE VRF SO_MARK

Distro (please complete the following information):

• Name: Debian
• Version: 12.2
• Architecture: x86_64

Details of any containerisation or hosted service (e.g. AWS)
VMware on-premises

Configuration file:

global_defs {
    notification_email {
        [email protected]
    }
    @server1 notification_email_from [email protected]
    @server2 notification_email_from [email protected]
    smtp_server smtp-srv.domain.example 25
    smtp_connect_timeout 60
    enable_script_security
    script_user keepalived
    notify_fifo /tmp/keepalived_haproxy-dmz_fifo
    notify_fifo_script /etc/keepalived/scripts/notify_fifo_state.sh
    fifo_write_vrrp_states_on_reload
    max_auto_priority
}

vrrp_track_process track_haproxy {
    process haproxy
}

vrrp_instance vrrp_inst {
    version 2
    state BACKUP
    nopreempt
    interface ens224
    # gratuitous ARP message on all interfaces to maintain MAC on switch cache (every 100s)
    garp_extra_if [all] 100
    # above setting results in: Keepalived_vrrp[155160]: (/etc/keepalived/keepalived.conf: Line 27) WARNING - invalid number '[all]'
    virtual_router_id 1
    @server1 priority 200
    @server2 priority 100
    advert_int 1

    # VIPs
    virtual_ipaddress {
        192.168.1.10 dev ens192 label ens192:main
    }
    virtual_ipaddress_excluded {
        include ./config-active/*.conf
    }

    # tracking
    track_interface {
        #ens224 (not necessary, tracked anyway)
        ens192
        dummy0
    }
    track_process {
        track_haproxy
    }
}

Notify and track scripts

notify_fifo_state.sh.txt

System Log entries

logfile.txt

Did keepalived coredump?

No.

Additional context
I was pleased to see the added functionality for fifo notifying in v2.2.8 and installed it even though it was not considered debian stable yet. For the most part, it works and I am happy to have this feature, unfortunately, sometimes it fails.

[Yncensus]

After further testing, the following pattern has shown itself:

A reload in BACKUP state works as expected, at least no change of the statefile is observed.
A reload in MASTER state changes the statefile to "not running", every single time.