Simplify custom resource state metrics API using CEL #2059

CatherineF-dev · 2023-05-07T20:55:03Z

What this PR does / why we need it: Simplify custom resource state metrics API

How does this change affect the cardinality of KSM: (increases, decreases or does not change cardinality) No

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1978

k8s-ci-robot · 2023-05-07T20:55:09Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: CatherineF-dev
Once this PR has been reviewed and has the lgtm label, please assign mrueg for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2023-05-07T20:55:11Z

This issue is currently awaiting triage.

If kube-state-metrics contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mrueg · 2023-06-14T14:58:56Z

docs/design/simplify-custom-resource-metrics-api.md

+ version: "v1"
+ mode: for_loop # or merged
+ metrics:
+ - name: "ready_count"


how is the metrics type defined?

I presume we'll still need fields with hardcoded values, eg., type.

Noting here that we still have non-path-based constructs, like so:

type (used to check OpenMetrics conformance, and if value field is applicable)

path (relative paths when used with labelsFrom*, these can be CEL expressions with the following subsections)

non-relative

relative

multi-relative (relative to different paths) (KSM does not support this right now)

commonLabels

list

Something we might want to persue under the configuration revamp is to keep such constructs to a minimum (and try to replace most of these using CEL, wherever possible), as we may find ourselves implementing custom logic and introducing complexity as such constructs grow over time.

mrueg · 2023-06-14T15:01:15Z

docs/design/simplify-custom-resource-metrics-api.md

+ type-a: 1
+ type-b: 3
+ conditions:
+ - name: a


Is it possible to have a configuration example using CEL that loops over this list and generates the following metrics?

ready_count{phase="Pending", condition="a"} 45 ready_count{phase="Pending", condition="b"} 66

Just to add on to what Manuel asked, would it be possible to reproduce this example using CEL?

rexagod

Thank you for working on this!

rexagod · 2023-06-15T08:06:25Z

docs/design/simplify-custom-resource-metrics-api.md

+## Goal
+
+- Simplify 8 operations into one operation to reduce maintaining work.
+- A complete API, so that can support more cases. For example, querying number of CRs under one CRD.


I'm sure what a complete API means here? AFAICT the proposal targets redefining a subset of configuration constructs, specifically ones that work with path resolutions. Is that not the case?

rexagod · 2023-06-15T08:17:42Z

docs/design/simplify-custom-resource-metrics-api.md

+
+| Existing operation | CEL |
+| :--- | :--- |
+| path: [status, sub] \n labelFromKey: type | x.status.sub.map(y, {"name": y}) |


Can you document what's x here? Is this a representation for all CRs that come under the specified GVK?

rexagod · 2023-06-15T08:20:54Z

docs/design/simplify-custom-resource-metrics-api.md

+ type-a: 1
+ type-b: 3
+ conditions:
+ - name: a


Just to add on to what Manuel asked, would it be possible to reproduce this example using CEL?

rexagod · 2023-06-15T09:00:43Z

docs/design/simplify-custom-resource-metrics-api.md

+ version: "v1"
+ mode: for_loop # or merged
+ metrics:
+ - name: "ready_count"


Noting here that we still have non-path-based constructs, like so:

type (used to check OpenMetrics conformance, and if value field is applicable)

path (relative paths when used with labelsFrom*, these can be CEL expressions with the following subsections)

non-relative

relative

multi-relative (relative to different paths) (KSM does not support this right now)

commonLabels

list

Something we might want to persue under the configuration revamp is to keep such constructs to a minimum (and try to replace most of these using CEL, wherever possible), as we may find ourselves implementing custom logic and introducing complexity as such constructs grow over time.

k8s-triage-robot · 2024-01-20T09:08:58Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

CatherineF-dev · 2024-01-20T12:56:05Z

/remove-lifecycle stale

rexagod · 2024-01-21T19:06:37Z

While this is easily one of the features we'd like to see implemented, tested, and polished enough throughout v2.x releases to finally make its way into v3, I just want to mention the primary concern around this, before we commit to the move, was if CEL could replace the existing native metrics. If we could verify the same first, I believe we could then move forward confidently which this change.

chrischdi · 2024-01-30T18:05:00Z

docs/design/simplify-custom-resource-metrics-api.md

+
+
+```
+kind: CustomResourceStateMetricsV2


Suggested change

kind: CustomResourceStateMetricsV2

kind: CustomResourceStateMetrics

apiVersion: v2

I'd propose to use the common versioning keys :-) We could even provide conversion from the old to the new version if we want

k8s-triage-robot · 2024-04-29T18:16:48Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

CatherineF-dev · 2024-04-29T18:22:09Z

/remove-lifecycle stale

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 7, 2023

k8s-ci-robot requested review from dgrisonnet and fpetkovski May 7, 2023 20:55

k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 7, 2023

CatherineF-dev force-pushed the cel branch 3 times, most recently from 0e9c7c1 to 059ed38 Compare May 8, 2023 13:26

Simplify custom resource state metrics API

a3befae

CatherineF-dev force-pushed the cel branch from 059ed38 to a3befae Compare May 8, 2023 13:28

CatherineF-dev mentioned this pull request May 8, 2023

Simplify custom resource metrics API by leveraging jq/CEL #1978

Open

mrueg reviewed Jun 14, 2023

View reviewed changes

rexagod reviewed Jun 15, 2023

View reviewed changes

benhodgkiss mentioned this pull request Jun 17, 2023

Missing replacement config for VPA collector in CRM #2041

Open

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 20, 2024

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 20, 2024

chrischdi reviewed Jan 30, 2024

View reviewed changes

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 29, 2024

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simplify custom resource state metrics API using CEL #2059

Simplify custom resource state metrics API using CEL #2059

CatherineF-dev commented May 7, 2023

k8s-ci-robot commented May 7, 2023

k8s-ci-robot commented May 7, 2023

mrueg Jun 14, 2023

rexagod Jun 14, 2023

rexagod Jun 15, 2023 •

edited

mrueg Jun 14, 2023

rexagod Jun 15, 2023

rexagod left a comment

rexagod Jun 15, 2023

rexagod Jun 15, 2023

rexagod Jun 15, 2023

rexagod Jun 15, 2023 •

edited

k8s-triage-robot commented Jan 20, 2024

CatherineF-dev commented Jan 20, 2024

rexagod commented Jan 21, 2024

chrischdi Jan 30, 2024

k8s-triage-robot commented Apr 29, 2024

CatherineF-dev commented Apr 29, 2024

	kind: CustomResourceStateMetricsV2
	kind: CustomResourceStateMetrics
	apiVersion: v2

Simplify custom resource state metrics API using CEL #2059

Are you sure you want to change the base?

Simplify custom resource state metrics API using CEL #2059

Conversation

CatherineF-dev commented May 7, 2023

k8s-ci-robot commented May 7, 2023

k8s-ci-robot commented May 7, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rexagod Jun 15, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rexagod left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rexagod Jun 15, 2023 • edited

Choose a reason for hiding this comment

k8s-triage-robot commented Jan 20, 2024

CatherineF-dev commented Jan 20, 2024

rexagod commented Jan 21, 2024

Choose a reason for hiding this comment

k8s-triage-robot commented Apr 29, 2024

CatherineF-dev commented Apr 29, 2024

rexagod Jun 15, 2023 •

edited

rexagod Jun 15, 2023 •

edited