Proposal: Tekton Opinionated CI project

[vdemeester]

Summary

Create a project that is an opinionated CI project used and maintained by the Tekton community. The goal is to replace completely prow to be able to dog-food Tekton entirely with Tekton.

tektoncd/plumbing would become either obsolute, or the configuration repository for that project.

Goals

• Very little plumbing (from the repository maintainer perspective)
  • Everything comes built-in, opiniated (PR oriented)
  • A new tektoncd project should go from creation to "able to release" in a matter of minutes or hours
  • "As code" configuration, aka CI definition lives in the repository the CI is on (e.g. GitHub workflows)
• Following SLSA guidelines and Secure supply chain on tektoncd projects
  • Enable image signing (with chains, …) for all tektoncd project
• Approve and LGTM flow
• CI oriented dashboard
  • Think of prow
• optional can serve as a blueprint for a CI system based on top of tektoncd component
• optional Simple syntax
  • Close to tektoncd/pipeline if even wrapped

Use cases

• Pipeline on push event
  • branch, tag, rev, …
• Pipeline on PR event
  • On-demand pipeline
• Pipeline on a schedule
  • Automated releases (push a tag or release branch, the rest is automatic — and configurable)
  • Nightly releases
• Chatops & notifications
  • PR comments
  • Other integrations (slack, …)
• Shared set of Tasks and Pipelines…
  • … while still allow customization per projects
  • … and "as code"
    aka changing the CI setup happens in the repository instead of an external one (e.g. GitHub workflows vs Prow)

Benefits

• Move entrypoint for features in that project instead of "nowhere" today
  • Links multiple project together in one "feature" drive
• Opinionated
  • Opinion helps taking decisions
• Implement / satisfy all SLSA
  • Removing the need to do this in tektoncd/pipeline (primitives can still be in tektoncd/pipeline)
• Control Step.image in order to give some "guarantees" (SLSA). Features like the following would be "moved" on that project instead of tektoncd/pipeline.
  • Enforce Step.image is under control of Task Author #7085
  • Allow Task Authors specify a Param Allowed List #7084
• Identify missing features required for us to adopt our own tools
• Identify missing task that the community needs…
  • … and would support

Related work

• Dogfooding Roadmap - Tekton Based CI/CD for Tekton #912
• Tekton Workflows #464
• Pipeline as code TEP #341

Who will own it

This is yet to be decided, but it would most likely be the current tektondc/plumbing maintainers, and any existing tektoncd project maintainers interested into working on it.

cc @dibyom @bobcatfish @jerop @chmouel @afrittoli

[chmouel]

I will be happy if you want to consider pipelines as code for this.

Last time we discussed about getting pac upstream the discussion stalled because PAC is using PipelineRun + annotations instead of a new CR on top of it to specify configuration...

Most of the requirements as you listed should work with PaC, except:

Following SLSA guidelines and Secure supply chain on tektoncd projects
Enable image signing (with chains, …) for all tektoncd project

I am not sure what that mean because from what I understand chains is doing this but I will be happy to talk how we can improve this.

CI oriented dashboard

We don't have any! we have a tkn pac (the CLI) command that give you an overview of all the Repository Statuses but no UI that looks like prow (and no integration with results at the moment for historical datas)

reading the Benefits section if you proposal it seems that you are targeting making a release of tektoncd/pipeline as the target?

I am not quite sure i understand this listed benefits :

Move entrypoint for features in that project instead of "nowhere" today
Links multiple project together in one "feature" drive

it seems to me this goes beyond of a Opinionated CI and more about and it's more about an organizational challenge?

I can help set-up pipelinesascode on a wiling project to have a 'feel' of how it works for maintainers and maybe help taking a decision...

[vdemeester]

goes beyond of a Opinionated CI and more about and it's first more about an organization challenge?

Yes, but having this opinionated CI would help. A CI feature for Tekton, would most likely go into that project first. And from there we could define where it fits (which components needs some updates/features to support that use case, …).

[bendory]

👍🏾 Love this idea! Tekton-on-Tekton will be quite beneficial:

• We'll eat our own dogfood which will uplevel our user empathy and make us more likely to spot both issues and opportunities
• Our implementation will provide templates to ease Tekton adoption for new users and well-lit golden paths for existing users

[adambkaplan]

The goal is to replace completely prow to be able to dog-food Tekton entirely with Tekton.

Must everything be handled by a Tekton-maintained component? For instance - it may be better for "eventing" things to be contributed to cdEvents or implementing Knative Eventing producers/consumers.

[vdemeester]

Must everything be handled by a Tekton-maintained component? For instance - it may be better for "eventing" things to be contributed to cdEvents or implementing Knative Eventing producers/consumers.

Not everything. cdEvents is a good example, and there is probably others. The main idea is to get as much as we can using tekton-maintainer components and use whatever make the most sense for things we don't want to reinvent the wheel.