Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add formats parameter to parse_timestamp #503

Open
mattnthat opened this issue Oct 10, 2023 · 2 comments
Open

Add formats parameter to parse_timestamp #503

mattnthat opened this issue Oct 10, 2023 · 2 comments
Labels
vrl: stdlib Changes to the standard library

Comments

@mattnthat
Copy link

As per conversation on this thread

With the removal of the to_timestamp function, it's become harder to handle / normalise unknown timestamps in input data.

The feature as suggested by @jszwedko is to extend parse_timestamp to accept a 'format' parameter which can be an array of timestamp formats to try to match.
This puts control of the timestamp format array onto the user (vector.dev team don't have to hardcode and constantly extend acceptable formats) but gives the user a simple way to match a wide variery of timestamp formats in the incoming data.

Work around suggested in the comment thread above gives some good structure around how the feature could work:

formats = ["%+c", "some other format"]
t1 = now() # default
for_each(formats) -> |_index, format| {
  t2, err = parse_timestamp(.timestamp, "some other format")
  if err == null {
    t1 = t2
  }
}
.timestamp = t1
@mattnthat mattnthat changed the title Add formats parameter to parse_timestamp Add format parameter to parse_timestamp Oct 10, 2023
@jszwedko
Copy link
Member

Thanks @mattnthat ! Note the function already takes a format parameter to specify the format, as seen in the example you shared, but the change I think would be useful to have here is an optional formats parameter that accepts an array of valid formats.

@jszwedko jszwedko added the vrl: stdlib Changes to the standard library label Oct 11, 2023
@mattnthat mattnthat changed the title Add format parameter to parse_timestamp Add formats parameter to parse_timestamp Oct 11, 2023
@mattnthat
Copy link
Author

Thanks @mattnthat ! Note the function already takes a format parameter to specify the format, as seen in the example you shared, but the change I think would be useful to have here is an optional formats parameter that accepts an array of valid formats.

Noted! Updated Issue title to reflect formats

@mattnthat mattnthat reopened this Oct 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
vrl: stdlib Changes to the standard library
Projects
None yet
Development

No branches or pull requests

2 participants