Thanos Receive doesn't announce external_labels which are set in hashrings.json when it works in routing and ingesting mode.

[pvlltvk]

Thanos, Prometheus and Golang version used:

Thanos: 0.34.1; go1.21.7, Prometheus 2.48.1

Object Storage Provider:

AWS S3

What happened:

Thanos Receive doesn't announce external_labels set in hashrings.json when it's working in routing and ingesting mode.
I explicitly set need external labels for the tenant in the hashrings.json:

[
  {
    "endpoints": [
      {
        "address": "endpoint-1:10901"
      },
      {
        "address": "endpoint-2:10901"
      },
      {
        "address": "endpoint-3:10901"
      },
      {
        "address": "endpoint-4:10901"
      },
      {
        "address": "endpoint-5:10901"
      }
    ],
    "external_labels": {
      "team": "frontend",
      "prometheus": "prometheus-1"
    },
    "hashring": "frontend",
    "tenants": [
      "frontend-prometheus-1"
    ]
  }
]

When I used the default mode of the receive deployment (without splitting to route and ingester) this external_labels were announced by the receive instances (I could check it via Query UI), but after switching to the split mode I noticed that this labels are gone in Query UI.

What you expected to happen:
I expect that external_labels functionality works correctly in route/ingester mode.

How to reproduce it (as minimally and precisely as possible):

Run Thanos Receive in route/ingester mode and try to use external_labels in hashrings configuration.

[haanhvu]

Yes, currently this is just supported in RouterIngester mode. But we do have plan to support it in the RouterOnly and IngesterOnly mode too. Here's the proposal: https://github.com/thanos-io/thanos/blob/main/docs/proposals-accepted/202209-receive-tenant-external-labels.md

Actually, before supporting this in these two modes, it's best to split their implementation logics. I remember there's a PR that's very close to finish here: #6043

PRs are welcome to finish these^^