Improve run list doc with available --json fields

[babakks]

This PR adds the name of available JSON fields to the command doc.

Fixes #8588

[williammartin]

Thanks for this PR @babakks. I'm hesitant to accept this as is because as you call out in #8588 (comment) the same problem exists for every list command, but not only that, it also exists for other commands like issue view.

My concern with documenting them like this is that there is a good chance we either forget to document this, or they get out of date as fields are added. It makes me think there are two options:

1. We indicate that this list of fields can be produced by providing the --json flag and no argument:

➜  cli git:(merge-json) gh issue view --json
Specify one or more comma-separated fields for `--json`:
  assignees
  author
  body
  closed
  closedAt
  comments
  createdAt
  id
  labels
  milestone
  number
  projectCards
  projectItems
  reactionGroups
  state
  title
  updatedAt
  url

2. We build something so that at the time of execution we execute the same code path and include the output in the generated docs.

The first option is clearly much easier to achieve but the second might be more friendly in the long run.

[babakks]

@williammartin Thanks for the comment. I agree with the second approach being more useful. I came up with this simple string formatting function, named FormatListDoc. Please check my latest changes. And also, I couldn't find a better place than cmdutil package to put the new function.

Now the doc looks like this:

Available JSON fields to be used with the `--json` option are:
  `conclusion`, `createdAt`, `databaseId`, `displayTitle`, `event`, `headBranch`,
  `headSha`, `name`, `number`, `startedAt`, `status`, `updatedAt`, `url`,
  `workflowDatabaseId`, `workflowName`

[babakks]

@williammartin I checked all references of AddJSONFlags function and these are the commands that should get the same update to list available JSON fields in their docs:

• cache list
• codespace list
• codespace ports
• codespace view
• extension search
• issue list
• issue status
• issue view
• label list
• pr list
• pr status
• pr view
• release list
• release view
• repo deploy-key list
• repo list
• repo view
• run list (done in this PR)
• run view
• search code
• search commits
• search issues
• search prs
• search repos
• secret list
• variable list
• workflow list

If you're okay with the changes in this PR, I can apply the same change for the above commands as well.

[williammartin]

Thanks a lot for trying this out. I'm in two minds about this change. Here's what it looks like to me:

It seems like a lot of help real estate for what the user can discover by providing `--json` with no args, as is documented in [`gh help formatting`](https://cli.github.com/manual/gh_help_formatting):

The --json flag requires a comma separated list of fields to fetch. To view the possible JSON field names for a command omit the string argument to the --json flag when you run the command

This is linked to from the --template flag usage:

-t, --template string Format JSON output using a Go template; see "gh help formatting"

Maybe we should just include "; see "gh help formatting on the usage of --jq and --json?

On the other hand, being able to see the available fields at a glance in the manual or help is also quite nice for ctrl+F and if you don't have the CLI to hand right at that moment.

I think I'd like @andyfeller to chime in here to choose a direction.

---

Finally, if we did proceed with this, I also had a slightly different approach in mind. In my idea, when we pass the cmd to AddJSONFlags, it would update the Long field with this text concatenated. This way there's no way for us to either forget to add this content to a command that has the JSON flags, or to provide different slices by accident (e.g. when copying and pasting commands).

It would also mean that every command you listed above and in future get the behaviour with no further changes needed.

[babakks]

@williammartin Thanks. I, too, agree that it should only be seen in the long doc. And also I liked the idea of doing it in the AddJSONFields function so that all cases get updated.

One more argument in favor of listing available fields in the docs is that the online CLI manual would also benefit from it.

I'll wait for @andyfeller's opinion on this and will then update the requested changes.

[andyfeller]

Finally, if we did proceed with this, I also had a slightly different approach in mind. In my idea, when we pass the cmd to AddJSONFlags, it would update the Long field with this text concatenated. This way there's no way for us to either forget to add this content to a command that has the JSON flags, or to provide different slices by accident (e.g. when copying and pasting commands).

I love how we both came to the same conclusion as I really prefer minimizing additional changes needed by commands that can fall out of sync. However, I'd prefer a separate section lower in the help usage for JSON fields over tacking it onto Long description as it can be a lot and getting information on usage is generally more used than listing out the json fields which you can get by running with empty --json flag. So instead of modifying command metadata, I'd enhance the places that output help depending on the medium.

Mock up of gh repo view --help with JSON FIELDS section

Display the description and the README of a GitHub repository.

With no argument, the repository for the current directory is displayed.

With '--web', open the repository in a web browser instead.

With '--branch', view a specific branch of the repository.

For more information about output formatting flags, see `gh help formatting`.

USAGE
  gh repo view [<repository>] [flags]

FLAGS
  -b, --branch string     View a specific branch of the repository
  -q, --jq expression     Filter JSON output using a jq expression
      --json fields       Output JSON with the specified fields
  -t, --template string   Format JSON output using a Go template; see "gh help formatting"
  -w, --web               Open a repository in the browser

INHERITED FLAGS
  --help   Show help for command

JSON FIELDS
  assignableUsers
  codeOfConduct
  contactLinks
  createdAt
  defaultBranchRef
  deleteBranchOnMerge
  description
  diskUsage
  forkCount
  fundingLinks
  hasDiscussionsEnabled
  hasIssuesEnabled
  hasProjectsEnabled
  hasWikiEnabled
  homepageUrl
  id
  isArchived
  isBlankIssuesEnabled
  isEmpty
  isFork
  isInOrganization
  isMirror
  isPrivate
  isSecurityPolicyEnabled
  isTemplate
  isUserConfigurationRepository
  issueTemplates
  issues
  labels
  languages
  latestRelease
  licenseInfo
  mentionableUsers
  mergeCommitAllowed
  milestones
  mirrorUrl
  name
  nameWithOwner
  openGraphImageUrl
  owner
  parent
  primaryLanguage
  projects
  projectsV2
  pullRequestTemplates
  pullRequests
  pushedAt
  rebaseMergeAllowed
  repositoryTopics
  securityPolicyUrl
  squashMergeAllowed
  sshUrl
  stargazerCount
  templateRepository
  updatedAt
  url
  usesCustomOpenGraphImage
  viewerCanAdminister
  viewerDefaultCommitEmail
  viewerDefaultMergeMethod
  viewerHasStarred
  viewerPermission
  viewerPossibleCommitEmails
  viewerSubscription
  visibility
  watchers

LEARN MORE
  Use `gh <command> <subcommand> --help` for more information about a command.
  Read the manual at https://cli.github.com/manual

In scope

... --help

cli/pkg/cmd/root/help.go

Lines 90 to 195 in 6f35082

	func rootHelpFunc(f *cmdutil.Factory, command *cobra.Command, args []string) {
	flags := command.Flags()
	if isRootCmd(command) {
	if versionVal, err := flags.GetBool("version"); err == nil && versionVal {
	fmt.Fprint(f.IOStreams.Out, command.Annotations["versionInfo"])
	return
	} else if err != nil {
	fmt.Fprintln(f.IOStreams.ErrOut, err)
	hasFailed = true
	return
	}
	}
	cs := f.IOStreams.ColorScheme()
	if help, _ := flags.GetBool("help"); !help && !command.Runnable() && len(flags.Args()) > 0 {
	nestedSuggestFunc(f.IOStreams.ErrOut, command, flags.Args()[0])
	hasFailed = true
	return
	}
	namePadding := 12
	type helpEntry struct {
	Title string
	Body string
	}
	longText := command.Long
	if longText == "" {
	longText = command.Short
	}
	if longText != "" && command.LocalFlags().Lookup("jq") != nil {
	longText = strings.TrimRight(longText, "\n") +
	"\n\nFor more information about output formatting flags, see `gh help formatting`."
	}
	helpEntries := []helpEntry{}
	if longText != "" {
	helpEntries = append(helpEntries, helpEntry{"", longText})
	}
	helpEntries = append(helpEntries, helpEntry{"USAGE", command.UseLine()})
	if len(command.Aliases) > 0 {
	helpEntries = append(helpEntries, helpEntry{"ALIASES", strings.Join(command.Aliases, "\n")})
	}
	for _, g := range GroupedCommands(command) {
	var names []string
	for _, c := range g.Commands {
	names = append(names, rpad(c.Name()+":", namePadding)+c.Short)
	}
	helpEntries = append(helpEntries, helpEntry{
	Title: strings.ToUpper(g.Title),
	Body: strings.Join(names, "\n"),
	})
	}
	if isRootCmd(command) {
	var helpTopics []string
	if c := findCommand(command, "actions"); c != nil {
	helpTopics = append(helpTopics, rpad(c.Name()+":", namePadding)+c.Short)
	}
	for _, helpTopic := range HelpTopics {
	helpTopics = append(helpTopics, rpad(helpTopic.name+":", namePadding)+helpTopic.short)
	}
	sort.Strings(helpTopics)
	helpEntries = append(helpEntries, helpEntry{"HELP TOPICS", strings.Join(helpTopics, "\n")})
	}
	flagUsages := command.LocalFlags().FlagUsages()
	if flagUsages != "" {
	helpEntries = append(helpEntries, helpEntry{"FLAGS", dedent(flagUsages)})
	}
	inheritedFlagUsages := command.InheritedFlags().FlagUsages()
	if inheritedFlagUsages != "" {
	helpEntries = append(helpEntries, helpEntry{"INHERITED FLAGS", dedent(inheritedFlagUsages)})
	}
	if _, ok := command.Annotations["help:arguments"]; ok {
	helpEntries = append(helpEntries, helpEntry{"ARGUMENTS", command.Annotations["help:arguments"]})
	}
	if command.Example != "" {
	helpEntries = append(helpEntries, helpEntry{"EXAMPLES", command.Example})
	}
	if _, ok := command.Annotations["help:environment"]; ok {
	helpEntries = append(helpEntries, helpEntry{"ENVIRONMENT VARIABLES", command.Annotations["help:environment"]})
	}
	helpEntries = append(helpEntries, helpEntry{"LEARN MORE", heredoc.Docf(`
	Use %[1]sgh <command> <subcommand> --help%[1]s for more information about a command.
	Read the manual at https://cli.github.com/manual
	`, "`")})
	out := f.IOStreams.Out
	for _, e := range helpEntries {
	if e.Title != "" {
	// If there is a title, add indentation to each line in the body
	fmt.Fprintln(out, cs.Bold(e.Title))
	fmt.Fprintln(out, text.Indent(strings.Trim(e.Body, "\r\n"), " "))
	} else {
	// If there is no title print the body as is
	fmt.Fprintln(out, e.Body)
	}
	fmt.Fprintln(out)
	}
	}

man ...

cli/internal/docs/man.go

Lines 181 to 207 in 6f35082

	func genMan(cmd *cobra.Command, header *GenManHeader) []byte {
	cmd.InitDefaultHelpCmd()
	cmd.InitDefaultHelpFlag()
	// something like `rootcmd-subcmd1-subcmd2`
	dashCommandName := strings.Replace(cmd.CommandPath(), " ", "-", -1)
	buf := new(bytes.Buffer)
	manPreamble(buf, header, cmd, dashCommandName)
	for _, g := range root.GroupedCommands(cmd) {
	fmt.Fprintf(buf, "# %s\n", strings.ToUpper(g.Title))
	for _, subcmd := range g.Commands {
	fmt.Fprintf(buf, "`%s`\n: %s\n\n", manLink(subcmd), subcmd.Short)
	}
	}
	manPrintOptions(buf, cmd)
	if len(cmd.Example) > 0 {
	buf.WriteString("# EXAMPLE\n")
	buf.WriteString(fmt.Sprintf("```\n%s\n```\n", cmd.Example))
	}
	if cmd.HasParent() {
	buf.WriteString("# SEE ALSO\n")
	buf.WriteString(fmt.Sprintf("`%s`\n", manLink(cmd.Parent())))
	}
	return buf.Bytes()
	}

Website

cli/internal/docs/markdown.go

Lines 111 to 153 in 6f35082

	// genMarkdownCustom creates custom markdown output.
	func genMarkdownCustom(cmd *cobra.Command, w io.Writer, linkHandler func(string) string) error {
	fmt.Fprint(w, "{% raw %}")
	fmt.Fprintf(w, "## %s\n\n", cmd.CommandPath())
	hasLong := cmd.Long != ""
	if !hasLong {
	fmt.Fprintf(w, "%s\n\n", cmd.Short)
	}
	if cmd.Runnable() {
	fmt.Fprintf(w, "```\n%s\n```\n\n", cmd.UseLine())
	}
	if hasLong {
	fmt.Fprintf(w, "%s\n\n", cmd.Long)
	}
	for _, g := range root.GroupedCommands(cmd) {
	fmt.Fprintf(w, "### %s\n\n", g.Title)
	for _, subcmd := range g.Commands {
	fmt.Fprintf(w, "* [%s](%s)\n", subcmd.CommandPath(), linkHandler(cmdManualPath(subcmd)))
	}
	fmt.Fprint(w, "\n\n")
	}
	if err := printOptions(w, cmd); err != nil {
	return err
	}
	fmt.Fprint(w, "{% endraw %}\n")
	if len(cmd.Example) > 0 {
	fmt.Fprint(w, "### Examples\n\n{% highlight bash %}{% raw %}\n")
	fmt.Fprint(w, cmd.Example)
	fmt.Fprint(w, "{% endraw %}{% endhighlight %}\n\n")
	}
	if cmd.HasParent() {
	p := cmd.Parent()
	fmt.Fprint(w, "### See also\n\n")
	fmt.Fprintf(w, "* [%s](%s)\n", p.CommandPath(), linkHandler(cmdManualPath(p)))
	}
	return nil
	}

Out of scope

gh reference

cli/pkg/cmd/root/help_reference.go

Lines 54 to 72 in 6f35082

	func cmdRef(w io.Writer, cmd *cobra.Command, depth int) {
	// Name + Description
	fmt.Fprintf(w, "%s `%s`\n\n", strings.Repeat("#", depth), cmd.UseLine())
	fmt.Fprintf(w, "%s\n\n", cmd.Short)
	// Flags
	// TODO: fold in InheritedFlags/PersistentFlags, but omit `--help` due to repetitiveness
	if flagUsages := cmd.Flags().FlagUsages(); flagUsages != "" {
	fmt.Fprintf(w, "```\n%s````\n\n", dedent(flagUsages))
	}
	// Subcommands
	for _, c := range cmd.Commands() {
	if c.Hidden {
	continue
	}
	cmdRef(w, c, depth+1)
	}
	}

[williammartin]

@andyfeller and I discussed this on Slack and we agree that including it in a separate section of the help is the best option here. However, we think it would be better to provide a compressed form as you have done to avoid annoying users by scrolling past the most important sections when there are a lot of fields.

[babakks]

@williammartin @andyfeller Thanks for clarifying this. Since this PR is no longer just about the gh run list command, I've force-pushed a fresh series of commits. I wanted to also change the PR title, but wanted to make sure it's okay with you.

The "JSON Fields" section is now included in Markdown (website format), man pages, and --help output. I've left the gh reference command untouched since that should be kept as terse as possible.

For this to work, within the AddJSONFlags function, I used the command's Annotations map and assigned the comma-separated list of JSON fields to the new "help:json-fields" key. When generating the docs, this specific annotation is checked and if it's there we'll then split it (over commas), sort it, and print it to the output. Depending on the medium, backticks will be added to surround the individual field names.

I've also moved the newly created FormatSlice function (to apply word wrapping and enclosing with backticks) to the internal/text package so that it'll be accessible to the different parts of the code.

Different docs (for gh run list) now look like the following.

--help

List recent workflow runs

For more information about output formatting flags, see `gh help formatting`.

USAGE
  gh run list [flags]

ALIASES
  ls

FLAGS
  -b, --branch string     Filter runs by branch
  -c, --commit SHA        Filter runs by the SHA of the commit
      --created date      Filter runs by the date it was created
  -e, --event event       Filter runs by which event triggered the run
  -q, --jq expression     Filter JSON output using a jq expression
      --json fields       Output JSON with the specified fields
  -L, --limit int         Maximum number of runs to fetch (default 20)
  -s, --status string     Filter runs by status: {queued|completed|in_progress|requested|waiting|action_required|cancelled|failure|neutral|skipped|stale|startup_failure|success|timed_out}
  -t, --template string   Format JSON output using a Go template; see "gh help formatting"
  -u, --user string       Filter runs by user who triggered the run
  -w, --workflow string   Filter runs by workflow

INHERITED FLAGS
      --help                     Show help for command
  -R, --repo [HOST/]OWNER/REPO   Select another repository using the [HOST/]OWNER/REPO format

JSON FIELDS
  conclusion, createdAt, databaseId, displayTitle, event, headBranch, headSha,
  name, number, startedAt, status, updatedAt, url, workflowDatabaseId,
  workflowName

LEARN MORE
  Use `gh <command> <subcommand> --help` for more information about a command.
  Read the manual at https://cli.github.com/manual

Man pages

Markdown (website)

[babakks]

@andyfeller I couldn't ask for your review from the sidebar. Could you please have a look?

[andyfeller]

@andyfeller I couldn't ask for your review from the sidebar. Could you please have a look?

Thank you for your patience, @babakks, going to get this done now 🙇

[andyfeller]

@babakks : once again, thank you for improving the GitHub CLI user experiences ❤️

Capturing examples of the updated output looks good:

... --help

$ ./bin/gh repo list --help
List repositories owned by a user or organization.

Note that the list will only include repositories owned by the provided argument,
and the `--fork` or `--source` flags will not traverse ownership boundaries. For example,
when listing the forks in an organization, the output would not include those owned by individual users.

For more information about output formatting flags, see `gh help formatting`.

USAGE
  gh repo list [<owner>] [flags]

ALIASES
  ls

FLAGS
      --archived            Show only archived repositories
      --fork                Show only forks
  -q, --jq expression       Filter JSON output using a jq expression
      --json fields         Output JSON with the specified fields
  -l, --language string     Filter by primary coding language
  -L, --limit int           Maximum number of repositories to list (default 30)
      --no-archived         Omit archived repositories
      --source              Show only non-forks
  -t, --template string     Format JSON output using a Go template; see "gh help formatting"
      --topic strings       Filter by topic
      --visibility string   Filter by repository visibility: {public|private|internal}

INHERITED FLAGS
  --help   Show help for command

JSON FIELDS
  assignableUsers, codeOfConduct, contactLinks, createdAt, defaultBranchRef,
  deleteBranchOnMerge, description, diskUsage, forkCount, fundingLinks,
  hasDiscussionsEnabled, hasIssuesEnabled, hasProjectsEnabled, hasWikiEnabled,
  homepageUrl, id, isArchived, isBlankIssuesEnabled, isEmpty, isFork,
  isInOrganization, isMirror, isPrivate, isSecurityPolicyEnabled, isTemplate,
  isUserConfigurationRepository, issueTemplates, issues, labels, languages,
  latestRelease, licenseInfo, mentionableUsers, mergeCommitAllowed, milestones,
  mirrorUrl, name, nameWithOwner, openGraphImageUrl, owner, parent,
  primaryLanguage, projects, projectsV2, pullRequestTemplates, pullRequests,
  pushedAt, rebaseMergeAllowed, repositoryTopics, securityPolicyUrl,
  squashMergeAllowed, sshUrl, stargazerCount, templateRepository, updatedAt, url,
  usesCustomOpenGraphImage, viewerCanAdminister, viewerDefaultCommitEmail,
  viewerDefaultMergeMethod, viewerHasStarred, viewerPermission,
  viewerPossibleCommitEmails, viewerSubscription, visibility, watchers

LEARN MORE
  Use `gh <command> <subcommand> --help` for more information about a command.
  Read the manual at https://cli.github.com/manual

$ ./bin/gh issue view --help
Display the title, body, and other information about an issue.

With `--web` flag, open the issue in a web browser instead.

For more information about output formatting flags, see `gh help formatting`.

USAGE
  gh issue view {<number> | <url>} [flags]

FLAGS
  -c, --comments          View issue comments
  -q, --jq expression     Filter JSON output using a jq expression
      --json fields       Output JSON with the specified fields
  -t, --template string   Format JSON output using a Go template; see "gh help formatting"
  -w, --web               Open an issue in the browser

INHERITED FLAGS
      --help                     Show help for command
  -R, --repo [HOST/]OWNER/REPO   Select another repository using the [HOST/]OWNER/REPO format

JSON FIELDS
  assignees, author, body, closed, closedAt, comments, createdAt, id, labels,
  milestone, number, projectCards, projectItems, reactionGroups, state, title,
  updatedAt, url

LEARN MORE
  Use `gh <command> <subcommand> --help` for more information about a command.
  Read the manual at https://cli.github.com/manual

[andyfeller]

Realizing I don't know many commands where ARGUMENTS section would show up to say definitively if JSON fields should come before or after 🤔 ... my gut says JSON fields after arguments but will keep looking for an example as not the end of the world

[babakks]

I agree that arguments should be printed before the JSON fields, since they're more like an appendix section. Since this is already merged, I can submit another PR for that if you say so.