[BR]: Fail2ban for mssql not work filtering my log from docker containers log

[bistungki]

Environment:

• Fail2Ban version : 0.11.1
• OS, including release name/version : Ubuntu 20.04.3 LTS
• Fail2Ban installed via OS/distribution mechanisms
• You have not applied any additional foreign patches to the codebase
• Some customizations were done to the configuration (provide details below is so)

In my case, I think all the configurations have gone well and I found no errors when running fail2ban, whether for fail2ban regex or fail2ban client restart, please enlighten me regarding this.
here is my mssql.conf config jail:

[mssqld]
enabled = true
logpath = /var/lib/docker/containers/20802bff83d79a801ae6e282a5cd2b827a18df7643fc8e92247e866b9ed3a88a/20802bff83d79a801ae6e282a5cd2b827a18df7643fc8e92247e866b9ed3a88a-json.log
findtime = 10m
maxretry = 1
bantime = 10m
filter = mssqld
port = 1433
action = iptables-allports

and my filter config:

[Definition]
datepattern = ^\{\"log\":\"\\r%%Y-%%m-%%d %%H:%%M:%%S(?:\.%%f)?
_daemon = mssqld
failregex = ^%(__prefix_line)s.*Login failed for user '[A-Za-z ]*'. .*provided. \[CLIENT: <HOST>\].*
ignoreregex =

docker log value:

{"log":"\r2024-01-15 01:44:57.08 Logon       Login failed for user 'sa'. Reason: Could not find a login matching the name provided. [CLIENT: 173.212.226.244]\r\n","stream":"stdout","time":"2024-01-15T01:44:57.091802906Z"}
{"log":"\r2024-01-15 01:44:57.24 Logon       Error: 18456, Severity: 14, State: 5.\r\n","stream":"stdout","time":"2024-01-15T01:44:57.25033964Z"}
{"log":"\r2024-01-15 01:44:57.24 Logon       Login failed for user 'sa'. Reason: Could not find a login matching the name provided. [CLIENT: 80.66.76.30]\r\n","stream":"stdout","time":"2024-01-15T01:44:57.253764779Z"}
{"log":"\r2024-01-15 01:44:57.31 Logon       Error: 18456, Severity: 14, State: 5.\r\n","stream":"stdout","time":"2024-01-15T01:44:57.316004224Z"}
{"log":"\r2024-01-15 01:44:57.31 Logon       Login failed for user 'sa'. Reason: Could not find a login matching the name provided. [CLIENT: 87.251.75.20]\r\n","stream":"stdout","time":"2024-01-15T01:44:57.3196235Z"}
{"log":"\r2024-01-15 01:44:57.36 Logon       Error: 18456, Severity: 14, State: 5.\r\n","stream":"stdout","time":"2024-01-15T01:44:57.366363673Z"}
{"log":"\r2024-01-15 01:44:57.36 Logon       Login failed for user 'sa'. Reason: Could not find a login matching the name provided. [CLIENT: 80.66.76.30]\r\n","stream":"stdout","time":"2024-01-15T01:44:57.370183786Z"}
{"log":"\r2024-01-15 01:44:57.36 Logon       Error: 18456, Severity: 14, State: 5.\r\n","stream":"stdout","time":"2024-01-15T01:44:57.372326741Z"}
{"log":"\r2024-01-15 01:44:57.36 Logon       Login failed for user 'sa'. Reason: Could not find a login matching the name provided. [CLIENT: 173.212.226.244]\r\n","stream":"stdout","time":"2024-01-15T01:44:57.37422179Z"}
{"log":"\r2024-01-15 01:44:57.39 Logon       Error: 18456, Severity: 14, State: 5.\r\n","stream":"stdout","time":"2024-01-15T01:44:57.403006742Z"}
{"log":"\r2024-01-15 01:44:57.39 Logon       Login failed for user 'sa'. Reason: Could not find a login matching the name provided. [CLIENT: 173.212.226.244]\r\n","stream":"stdout","time":"2024-01-15T01:44:57.405919035Z"}
{"log":"\r2024-01-15 01:44:57.44 Logon       Error: 18456, Severity: 14, State: 5.\r\n","stream":"stdout","time":"2024-01-15T01:44:57.453760098Z"}
{"log":"\r2024-01-15 01:44:57.44 Logon       Login failed for user 'sa'. Reason: Could not find a login matching the name provided. [CLIENT: 173.212.226.244]\r\n","stream":"stdout","time":"2024-01-15T01:44:57.456644058Z"}

fail2ban-regex:

Running tests
=============

Use   failregex filter file : mssqld, basedir: /etc/fail2ban
Use      datepattern : ^\{\"log\":\"\\rYear-Month-Day 24hour:Minute:Second(?:\.Microseconds)?
Use         log file : /var/lib/docker/containers/20802bff83d79a801ae6e282a5cd2b827a18df7643fc8e92247e866b9ed3a88a/20802bff83d79a801ae6e282a5cd2b827a18df7643fc8e92247e866b9ed3a88a-json.log
Use         encoding : UTF-8


Results
=======

Failregex: 28279 total
|-  #) [# of hits] regular expression
|   1) [28279] ^(?:\[\])?\s*(?:<[^.]+\.[^.]+>\s+)?(?:\S+\s+)?(?:kernel:\s?\[ *\d+\.\d+\]:?\s+)?(?:@vserver_\S+\s+)?(?:(?:(?:\[\d+\])?:\s+[\[\(]?\S*(?:\(\S+\))?[\]\)]?:?|[\[\(]?\S*(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:?)\s+)?(?:\[ID \d+ \S+\]\s+)?.*Login failed for user '[A-Za-z ]*'. .*provided. \[CLIENT: <HOST>\].*
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [60122] ^\{\"log\":\"\\rYear-Month-Day 24hour:Minute:Second(?:\.Microseconds)?
`-

Lines: 60122 lines, 0 ignored, 28279 matched, 31843 missed
[processed in 4.35 sec]

Missed line(s): too many to print.  Use --print-all-missed to print all 31843 lines

root@pc# tail -f /var/log/fail2ban.log

2024-01-15 08:48:02,678 fail2ban.datedetector   [1532225]: INFO      date pattern `'^\\{\\"log\\":\\"\\\\r%Y-%m-%d %H:%M:%S(?:\\.%f)?'`: `^\{\"log\":\"\\rYear-Month-Day 24hour:Minute:Second(?:\.Microseconds)?`
2024-01-15 08:48:02,678 fail2ban.filter         [1532225]: INFO      maxRetry: 1
2024-01-15 08:48:02,679 fail2ban.filter         [1532225]: INFO      findtime: 600
2024-01-15 08:48:02,679 fail2ban.actions        [1532225]: INFO      banTime: 600
2024-01-15 08:48:02,679 fail2ban.filter         [1532225]: INFO      encoding: UTF-8
2024-01-15 08:48:02,680 fail2ban.filter         [1532225]: INFO    Added logfile: '/var/lib/docker/containers/20802bff83d79a801ae6e282a5cd2b827a18df7643fc8e92247e866b9ed3a88a/20802bff83d79a801ae6e282a5cd2b827a18df7643fc8e92247e866b9ed3a88a-json.log' (pos = 11205766, hash = 28bf70c8a0bfb3106b8f848889779500d2849006)
2024-01-15 08:48:02,834 fail2ban.jail           [1532225]: INFO    Jail 'sshd' started
2024-01-15 08:48:02,835 fail2ban.jail           [1532225]: INFO    Jail 'mssqld' started
2024-01-15 08:48:02,884 fail2ban.actions        [1532225]: NOTICE  [sshd] Restore Ban 159.75.122.191
2024-01-15 08:48:02,912 fail2ban.actions        [1532225]: NOTICE  [sshd] Restore Ban 195.3.147.81

Does all of the above work correctly? because after several hours of waiting, no one was jailed in the filtering

root@pc:/etc/fail2ban# fail2ban-client status mssqld
Status for the jail: mssqld
|- Filter
|  |- Currently failed:	0
|  |- Total failed:	0
|  `- File list:	/var/lib/docker/containers/20802bff83d79a801ae6e282a5cd2b827a18df7643fc8e92247e866b9ed3a88a/20802bff83d79a801ae6e282a5cd2b827a18df7643fc8e92247e866b9ed3a88a-json.log
`- Actions
   |- Currently banned:	0
   |- Total banned:	0
   `- Banned IP list:	
root@pc:/etc/fail2ban# 

Please help with this, and I would be very grateful in this matter.

[sebres]

Basically it would be enough to set a precise datepattern matching everything before Logon, and then remove end-anchor by failregex:

fail2ban/config/filter.d/mssql-auth.conf

Line 5 in 9bedc3c

failregex = ^\s*Logon\s+Login failed for user '<F-USER>(?:[^']*|.*)</F-USER>'\. [^'\[]+\[CLIENT: <ADDR>\]$

But because of catch-all .* in user-name, the RE becomes immediately vulnerable.
To avoid that either one would use something like that:

- failregex = ^\s*Logon\s+Login failed for user '<F-USER>(?:[^']*|.*)</F-USER>'\. [^'\[]+\[CLIENT: <ADDR>\]$
+ failregex = ^\s*Logon\s+Login failed for user '<F-USER>[^']*</F-USER>'\. [^'\[]+\[CLIENT: <ADDR>\]
datepattern = ^\{"log":"\\r%%Y-%%m-%%d %%H:%%M:%%S(?:\.%%f)?

No idea what is with user names like o'connor etc.

Alternatively (at least as long as RFE #3526 not yet implemented) switch back to normal logging (from json format).

Or fix it somehow like:

datepattern = ^\{"log":"\\r%%Y-%%m-%%d %%H:%%M:%%S(?:\.%%f)?
_groupre = (?:"\w[^"]+":(?:"[^"]+"|\w+)\s*[,\}]\s*)
failregex = ^\s*Logon\s+Login failed for user '<F-USER>(?:[^']*|.*)</F-USER>'\. [^'\[]+\[CLIENT: <ADDR>\]\\r\\n",?\s*%(_groupre)s*$