Improve modules quality by examples

[marcosnils]

Problem #

In many cases it's not clear for module consumers how to use them once they find them in the Daggerverse. Even though https://daggerverse.dev has detailed docs about the module functions, their arguments and return types, sometimes modules are designed to be consumed in a particular way which is currently hard to express given the current module documentation features.

For example, looking at this module: https://daggerverse.dev/mod/github.com/jcsirot/daggerverse/java@c591e9e0b99def2fc8a67bb090fca5cd06cf6a1d

Daggerverse.dev doesn't show anything about how this module could be used but if you navigate to the module README file here https://github.com/jcsirot/daggerverse/tree/c591e9e0b99def2fc8a67bb090fca5cd06cf6a1d/java#usage, you can find more approachable example(s) on how this module can be utilized.

The purpose of this proposal is to elevate our modules quality by enabling module authors to define examples of their existing modules and surface them in the Daggerverse.

Solution #

Similar to other package managers and indexers like npm, pkg.go.dev, docs.rs, pulumi registry, etc. the goal is to provide Dagger module authors a way to define examples for their modules and later render them in Daggerverse.dev so consumers have a better UX while using them.

The challenge is that we would like Dagger module examples to follow the same UX that other aspects of the Dagger platform have. This translates to these examples ideally being defined through working code and the ability to be consumed in any of the officially supported SDKs.

After brainstorming with Dagger team members @vito @helderco and @jedevc, we came to the conclusion that there seems to be 2 possible way of providing such experience:

1. Leverage the dagql.ID object from Dagger core types and perform GraphQL code generation to multiple SDK's.

Pros:

• Leverage existing tooling
• Generate examples in multiple SDK's

Cons:

• All examples need to return Dagger core types ( to be confirmed )
• Examples can only show Dagger SDK code (no loops, if statements, etc)

example:

We want to write an example for the java module mentioned above

func (e *JavaExample) ExampleJava() *Container {
    c:= dag.Java().WithJdk("17").
        WithMaven("3.9.5").
        Container().
        WithExec([]string{"mvn", "--version"})
        if "foo" == "bar" { fmt.Println("test") } // this line won't get translated to the examples.
        return c
}

note: the if statement above won't be present in the final example snippet for this approach given that's not part of the GraphQL query sent to the engine

by manipulating the resulting dagql.ID of the returned container, we can then translate the above function call into the following GraphQL expression:

{
    java {
        withJdk(version: "17") {
            withMaven(version: "3.9.5") {
                container {
                    withExec(args: ["mvn", "-version"])  { }
                }
            }
        }
    }
}

Once we have the corresponding GraphQL query, we could use some code generation to translate the query into other SDKs.

2. Allow users to write examples similarly to option #1 and copy them verbatim in the Daggerverse docs without any SDK specific example generation

Pros:

• Simple and easier to implement
• Examples are displayed exactly as they're written.

Cons:

• Examples won't be shown for each SDK and will leak module SDK implementation details

A follow-up idea around this approach to make examples multi-language, is to rely on LLMs for automatic code translation across multiple SDKs. This presents its own challenges by itself, but some preliminary tests are already showing some promising results.

UX proposal #

Some ideas came up internally at Dagger about how we could approach this. The one that's getting the most traction is following a similar pattern how Go handles examples in their docs (https://go.dev/blog/examples) by establishing a convention on how to define them to then render them accordingly in the package docs.

The proposed UX is as follows:

Given a module M, with a type T and function F, any files under a ./examples folder in the module source directory with the conventions described below will be counted as module examples.

func Example() { ... } //defines example for module M
func ExampleT() { ... } //defines example for type T
func ExampleT_F() { ... } //defines example for T function F 

Multiple example functions for a module/type/function may be provided by appending a distinct suffix to the name:

func Example_suffix() { ... }
func ExampleT_suffix() { ... }
func ExampleT_F_suffix() { ... }

note: using Go as an example here. This applies for all the other SDK's with their standard code conventions

Once these examples are defined, they'd become part of a new core.Example type in the core GraphQL core.Module type here (https://github.com/marcosnils/dagger/blob/9baaa467f14bd4abf5399da1608b9d17d07e8fd7/core/module.go?plain=1#L17) and the core.Function type (https://github.com/marcosnils/dagger/blob/9baaa467f14bd4abf5399da1608b9d17d07e8fd7/core/typedef.go?plain=1#L18) which can be later queried.

In order to not penalize the module initialization time with example parsing, we're proposing to add a new WithExamples flag and to the core.ModuleSource type here (https://github.com/marcosnils/dagger/blob/9baaa467f14bd4abf5399da1608b9d17d07e8fd7/core/modulesource.go?plain=1#L48) so we can have better control when to trigger this example parsing logic. The resulting graphql query would be something like:

{
  moduleSource(refString: "github.com/marcosnils/[email protected]") {
    withExamples { // enables example parsing
      asModule{
        examples { //module level examples
          name
          content
        }
      }
    }
  }
}

Next steps

We're planning to spend a few cycles on solution #1 to get a feel about how approachable it is and hopefully come up with a working PoC to share with the community. Any other ideas or suggestions are always welcome.

[shykes]

In "solution 1", what steps does the user follow, to add an example to a module?

[shykes]

In "solution 1", can you generate a CLI example too?

[marcosnils]

In "solution 1", what steps does the user follow, to add an example to a module?

Both solution 1 and 2 steps to create examples has been added here #7217 (comment)

In "solution 1", can you generate a CLI example too?

Yes, both solutions would allow it 🙏