enhancement(reduce transform): New setting for reduce transform: end_every_period_ms

[charlesconnell]

I propose a new setting available for use on reduce transforms: end_every_period_ms.

In my organization's use of vector, we are tailing Java log files to look for garbage collection events. We want to know the duration of the longest garbage collection that happened lately, for a rolling "lately," and report that value to Prometheus. I don't believe we can accomplish this with vector at present, but this PR will enable it. We can collect values and reduce them with a max merge strategy, then flush them every 60 seconds.

I am sure that this will be useful to many users in the long run. Periodically flushing a reducer surely has many applications.

I have compiled vector with this change and deployed it within my organization, and confirmed that it works for us. For example, here is a snippet of our config:

      "gc-log/parse0/g1Pauses0/pauseTimeMax0": {
         "end_every_period_ms": 60000,
         "group_by": [
            "reason",
            "subtype",
            "type"
         ],
         "inputs": [
            "gc-log/parse0/g1Pauses0"
         ],
         "merge_strategies": {
            "gc_millis": "max"
         },
         "type": "reduce"
      },

This PR does not include new test coverage. I am new to Rust and not familiar with best testing practices. My new code has behavior dependent on the system clock, and that seems hard to test.

[bits-bot]

All committers have signed the CLA.

[jszwedko]

Thanks for this @charlesconnell ! We actually had a previous attempt at this that we had to revert: #16146. I think your PR resolves the issue that led to the revert by adding a new option rather than changing the behavior of an existing one.

I think this would close #16695.