-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Replies: 1 comment · 3 replies
-
Hi @Manniac07 , I'm not sure I completely understand. Can you share your configuration? |
Beta Was this translation helpful? Give feedback.
All reactions
3 replies
-
yes @jszwedko
transform
sink
|
Beta Was this translation helpful? Give feedback.
All reactions
-
@jszwedko what it keeps doing is loading the whole file for every line instead of spinning through the table i made and going on to next file |
Beta Was this translation helpful? Give feedback.
All reactions
-
Thanks! That behavior looks like what I would expect based on this configuration. The |
Beta Was this translation helpful? Give feedback.
All reactions
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
The complete csv file arrives in source folder and I am using lua to load the entire csv into an array then I emit each cell that matches of each line that matches my criteria until EOF. The issue is that vector reprocesses the source csv file for every row in the csv file.
I have attempted to use multiline from BOF to EOF and it still processes line by line.
I have also tried to remove file using remove_after_secs and that didn't work either.
Beta Was this translation helpful? Give feedback.
All reactions