-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Log messages exceeding size limits lost #921
Comments
Hi @jgtr-blis thanks for opening an issue. I am actively working on this. I don't know if we have a public issue tracker for it at the moment, but I will keep this issue updated with progress. |
Hi @braydonk, is this issue still being worked on? I've been running into a similar issue for some time now. However we're only using "parse_json" and "modify_fields", not the "parse_regex_ processor OP mentioned.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
When parsing long/multiline log entries (usually due to stack traces and debug information) they often exceed the maximum size resulting in the message being rejected with the following error:
When using a look ahead regex expression with the parse_regex processor there isn't an easy way that I'm aware of to limit the size of the message.
To Reproduce
Steps to reproduce the behaviour:
^(?<time>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2},\d{3}) (?<severity>[^ ]*) (?<msg>[\s\S]*?)(?=\g<time>|\Z)
2.Read in a log entry that exceeds the 256KB limit:
Expected behaviour
A message should be written to stackdriver to indicate that a log had been dropped rather than it just be silently lost and/or a processor should be provided to allow specific fields in the payload to be truncated based on a maximum length so we can at least get a log entry and part of the message.
The text was updated successfully, but these errors were encountered: