Support Regex-based Label \ Metadata matching

[Daniel-Vaz]

Request

This FR is similar to the following issue - grafana/loki#4987

The loki.process component should allow regex pattern matching on the structured_metadata and pack stages.

Use case

In a Alloy pipeline, when discovering k8s targets I can easily perform a labelmap to have access to all the discoverable __meta labels and keep the ones I need.

Now with these, in a loki.process component, I want to transform them into Loki Structured Metadata fields. This seems to be only possible by specifying a list of static names (Source).

With k8s discovered targets, they tend to have the __meta_kubernetes_*_annotation_* and __meta_kubernetes_*_label_* Labels. These values vary, and knowing all possible values, so that they get specified statically in the stage.structured_metadata block stage of the component doesn't seem feasible. Making me lose some k8s labels and annotations that I want to keep.

[akevdmeer]

You refer to service discovery as supporting flexible relabeling. In fact so does the Alloy component loki.relabel?

But selecting labels to move to structured metadata is currently left to the structured_metadata stage of loki.process that doesn't support regular expressions nor an analogue of labeldrop/labelkeep (which here would mean to move to structured metadata or not).

Could it be possible to move the structured metadata selection to loki.relabel by implementing an extra pair of actions? (Or make the structured_metadata stage of loki.process more expressive, of course.)