-
Notifications
You must be signed in to change notification settings - Fork 576
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Timesketch auto-renames timeline uploads with the same timeline name #3052
Comments
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
I'm having this issue, too. It occurs whenever the client makes a request to upload data to a sketch while the search index for the sketch is in use. This makes it common when uploading asynchronously, but possible even when uploading synchronously (as long as the upload rate is faster than the OpenSearch indexing rate). If the index for a given timeline is in use, Timesketch will create a new index and timeline for the data. The new timeline is given the "original name plus 5 random characters" name. If the user uploads more data and the original search index is still in use, but the secondary index is not, Timesketch is able to find and use the secondary index, but still always creates a new timeline. On my team, we create our own JSONL timelines and send them to Timesketch in batches, of which there can be thousands. This bug leads us to have several hundred timelines, even when there are only 2-3 indexes on the sketch. |
Describe the bug
When uploading mutliple timelines with the same name to a sketch, Timesketch renames the timeline name by appending 4 random characters.
When uploading timelines with the same name, i was assuming the additional uploads with just get added to the timeline data source instead of a new name
To Reproduce
This seems to happen mainly when using async uploads.
However, i was able to trigger using sync uploads by running the upload multiple times
I was able to trigger it using the python code below (replacing with Timesketch instance, and with home directory
I also had to spawn it several times to "mimic" async a bit
its alot easier to trigger with async code is seems
Expected behavior
Timesketch continues to upload data to the same Timeline name and increment the data source
Screenshots
Image below is from the python code. I had to spawn multiple instances to trigger the rename
Image below is from async code i was using (mixture of TypeScript and Rust)
Desktop (please complete the following information):
Running Timesketch on Ubuntu 22.04 VM
Additional context
I brought this up in Timesketch Slack channel, and it was mentioned to open a Github issue.
Could this possibly happen because mutliple uploads are being submitted at once (or too quickly) and there is some kind of brief lock on the timeline name and when the second upload occurs the lock triggers timesketch/opensearch to rename it?
Let me know if additional info is required
Thanks
The text was updated successfully, but these errors were encountered: