[loki] more events than what happened based on log #2843
Comments
|
@pschiffe: Thanks for opening an issue, it is currently awaiting triage. In the meantime, you can:
DetailsI am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
|
Hello, I saw on your discord thread that you had multiple loki data sources configured. Do you have a rough idea of how many logs per second are pushed to loki ? You can also set the data source to trace (by setting |
|
Hi @blotus, thank you for your reply. I do have multiple loki data sources configured, but none of them reads the same logs, the queries are distinct. You can see the acquis log files in the details in the issue description above. Roughly there's 10 to 25 logs per second pushed to loki now in total, but crowdsec is reading maybe half of that. I'll check the trace log if I can see something there. |
|
Here's a trace log from single loki datasource: crowdsec_trace.txt $ cat /etc/container-crowdsec/acquis.d/loki_mail.yaml
source: loki
log_level: trace
url: http://loki:3100/
headers:
X-Scope-OrgID: someorg
query: |
{container=~"mailcowdockerized-postfix-mailcow-1|mailcowdockerized-dovecot-mailcow-1"} | json | line_format `{{.MESSAGE}}`
labels:
type: syslogYou can try to search for a log line - |
|
Let me know if you need anything else from my side. |
Hi 👋🏻 we just need find time to replicate, currently we have internal projects and upcoming release of |
|
Hello, I have an update that may or may not be related. Roughly after a day or two of crowdsec running, the loki integration dies with: However loki is working fine, receiving data and I can browse the logs in Grafana. After crowdsec restart, it works again for a couple of days and then dies... Previously when it died, the metrics said that the loki acquisition read around 8 millions of lines. |
Hmm there is an incremental backoff features but personally I would not expect the acquisition to end if it has already made successful connections in the past since it knows the configuration is valid. 🤔 |
What happened?
For example, this IP was banned:
91.73.194.178. In the logs I only see:But the alert says there was 6 events:
This is causing me false positives with scenarios such as
crowdsecurity/postfix-spam,crowdsecurity/http-crawl-non_statics,LePresidente/http-generic-403-bf. Other scenarios are working fine.Acquisition for postfix logs:
What did you expect to happen?
Less events based on logs.
How can we reproduce it (as minimally and precisely as possible)?
See above.
Anything else we need to know?
No response
Crowdsec version
OS version
Enabled collections and parsers
Acquisition config
Prometheus metrics
Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.
No response
The text was updated successfully, but these errors were encountered: