New settings and resources API

[eliobischof]

⚠️ This PR should not be merged as long as we don't want to have the new APIs published in our docs.

Design an improved UX for settings and resources APIs

Based on the outcomes of this PR we implement future APIs and change existing APIs. The main goal is to have a uniform API structure and behavior.

• Work out an initial API style guide (see /API_STYLE_GUIDE.md in this PR).
• Define the most important endpoints specifically in proto (see additions to the /proto folder in this PR)
• Internally challenge the API style guide and the protos
• Invite the community to share their opinions

Internal Discussion

@zitadel/staff the proposal for a new experience with the ZITADEL resources and settings APIs tracked in #5875 is open for the internal discussion. Everybody who has opinions is invited to express themselves before we invite the community to the discussion (see the external discussion draft below). The external GitHub discussion is planned to be published on May 3rd. I will invite some of you to personal meetings next week.

Please review the /API_STYLE_GUIDE.md and all changes in the /proto and /docs folders. Everything else is for testing purposes only and will be cleaned up before this PR is merged.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
docs	❌ Failed (Inspect)			May 29, 2024 0:00am

[codecov]

Codecov Report

Attention: Patch coverage is 39.28571% with 34 lines in your changes are missing coverage. Please review.

Project coverage is 62.82%. Comparing base (cc56eea) to head (f160e3d).
Report is 467 commits behind head on main.

❗ Current head f160e3d differs from pull request most recent head 4e5d06b

Please upload reports for the commit 4e5d06b to get more accurate results.

Files	Patch %	Lines
internal/api/grpc/resources/actions/actions.go	0.00%	22 Missing ⚠️
internal/api/grpc/settings/template/template.go	0.00%	8 Missing ⚠️
cmd/start/start.go	0.00%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #7821       +/-   ##
===========================================
+ Coverage   50.10%   62.82%   +12.72%     
===========================================
  Files        1244     1316       +72     
  Lines      117337   108778     -8559     
===========================================
+ Hits        58788    68341     +9553     
+ Misses      55594    36500    -19094     
- Partials     2955     3937      +982     

Flag	Coverage Δ
core-integration-tests-postgres	62.82% <39.28%> (+38.39%)	⬆️
core-unit-tests	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[eliobischof]

External Discussion Draft

Feedback on new Resources and Settings APIs

Dear Community, we need your help

We are working on a new API version for resources and settings:

To achieve a uniform design and behavior, we propose the API Style Guide below. If you want to have more plastic view on the new design, have a look at the protos we defined for all settings as well as some resource blueprints for users, user schemas, action exections and action targets in the PR #7821. The end goal is to have all resources defined according to these examples.

Any feedback is appreciated, preferrably as early in the process as possible. What do you think about the API Style Guide? What is unclear or not helpful? Is something overengineered? Can you cover your use cases? Are you missing any rules? Is the API and the terminology intuitive? Anything else you would like to mention?

Please comment in this discussion or directly in #7821.

Thank you very much!

API Style Guide

The ZITADEL API adheres to the following style guide.

General

All APIs share the following behavior:

• APIs, that can be scoped with an optional RequestContext (system, instance, org), use the instance discovered by the requests Host header as default scope.

All Resources

All resource APIs share the following behavior:

• Search request results can be scoped to a RequestContext.
• Search request results only contain results for which the requesting user has the necessary read permissions.
• Search requests are limited to 100 by default. The limit can be increased up to 1000.
• Resource configurations are partially updatable. With HTTP, this is done via PATCH requests. If no changes were made, the response is successful.
• Status changes or other actions on resources with side effects are done via POST requests. Their HTTP path ends with the underscore prefixed action name. For example POST /resources/idps/{id}/_activate.

Reusable Resources

• Reusable resources can be created in a given context level (system, instance, org).
• For requests, that require a request ID, no request context is needed.
• Reusable resources are available in child contexts and by default have the same state (active or inactive) as in their immediate parent context.
• In child contexts, the state of a reused resource is inherited by default and can be changed to active, inactive or inherit.
• In child contexts, a reused resources configuration is read-only.
• Child contexts can always read the following properties of reused resources:
  • ID
  • name
  • description
  • state
  • sequence
  • last changed date
  • parent context
  • effective state in the immediate parent context.
• Managers of reusable resources in a parent context can restrict their readability in child contexts to the properties listed above.

Settings

• Setting and retrieving settings is always context-aware. By default, the context is the instance discovered by the requests Host header.
• All settings properties can be partially overwritten in child-contexts.
• All settings properties can be partially reset in child-contexts, so their values default to the parent contexts property values.
• All settings properties returned by queries contain the value and if it is inherited, the context where it is inherited from.

API Overview

Resources APIs

The following resource APIs adhere to the behavior described in the All Resources section.
Some of them are reusable resources and also adhere to the behavior described in the Reusable Resources section.

• Instances
  • Instances
  • InstanceDomains
• Organizations
  • Organizations
  • OrganizationDomains
• Projects
  • Projects
  • Apps
  • Roles
• Sessions
• Authorizations
  • Memberships
  • ProjectGrants
  • UserGrants
• Users
  • Users
  • Schemas ¹
• Actions
  • Executions ¹
  • Targets ¹
• IDPs ¹
• Notifiers
  • SMTPProviders (only target types system and instance) ¹
  • SMSProviders ¹

Settings APIs

The following settings APIs adhere to the behavior described in the Settings section:

• DefaultLogin (default login settings only take effect for the built-in login UI. Users of the session API who build their own login UI can use them too, but they won't have any effect in the ZITADEL core)
  • Texts (key-value pairs for localized login texts, previously known as login texts)
  • Branding (predefined branding settings and custom key-value pairs, previously known as label policy or branding settings)
  • Login (previously known as login policy)
  • Lockout (previously known as lockout policy)
  • Password (previously known as password complexity policy)
  • Help (previously known as legal and support settings or privacy policy)
  • Domain (previously known as domain policy)
• Features (feature toggles)
• Languages (default language, restricted languages)
• Instances (instance-wide settings like disallow_public_org_registration. These settings are not overwritable for orgs)
• Templates (html and text templates for fully customizable emails and sms)

gRPC-Gateway

• For accessing a resource by IDs, the IDs are mapped to path parameters.
• For searching resources, as many search parameters as possible are mapped to query parameters. Body: "*" should be avoided in the google.api.http proto option. Complex or sensitive queries are passed in the POST body.
• Request contexts are always mapped to query parameters.

Documentation

• The API defines the ubiquitous language used in docs and other references.
• Each endpoint is documented in a way that is understandable to a developer who is not familiar with the API.
• The API docs contain references to concepts and other endpoints if it helps to understand the documented endpoint.
• The API docs contain at least one example with realistic data for each endpoint.
• Each API docs page links this style guide.

Footnotes

1. Reusable Resources ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

[eliobischof]

flatten defaultlogin folder

[eliobischof]

release strategy internal

[hifabienne]

Please have a look at this (grpc error docs): #5797