Adds wandb example

[thomasjpfan]

This is an wandb example associated with flyteorg/flytekit#2405

You can replace the container_image with:

@task(
    container_image="ghcr.io/thomasjpfan/wandb:0.0.4",
    secret_requests=[wandb_secret],
)
@wandb_init(project=WANDB_PROJECT, entity=WANDB_ENTITY, secret=wandb_secret)

If you want to use the image builder:

REGISTRY = "localhost:30000"

wandb_package = "git+https://github.com/thomasjpfan/flytekit.git@925f2a3403f605f4ef018893b99ae68ec81fdae4#egg=flytekitplugins-wandb&subdirectory=plugins/flytekit-wandb"

image = ImageSpec(
    name="wandb_example",
    apt_packages=["git"],
    python_version="3.11",
    packages=["xgboost", "scikit-learn", wandb_package],
    registry=REGISTRY,
)

Note, that we need to merge flyteorg/flytekit#2405 first before the example can run as is.

[thomasjpfan]

@cosmicBboy From the CI issue is there another place I need to add auto_examples/wandb_plugin/index to the toc?

[cosmicBboy]

ah, so this index.md page actually isn't used any more (we still haven't sunsetted the old docs @ppiegaze @neverett, we need to decide when we want to do this). it's fine if it's here, but the entry needs to be added somewhere on this page: https://github.com/flyteorg/flytesnacks/blob/master/docs/integrations.md in the appropriate list-table directive.

For context:
The sphinx dark magic that's happening here is, these list tables are being picked up in the flyte docs here and converted into {list-table-toc}s that automatically add them to the flyte monodocs toc: https://github.com/flyteorg/flyte/blob/master/docs/_ext/import_projects.py#L59-L81

[neverett]

Left a few comments, otherwise LGTM. Mainly I changed "Weights & Biases" to "Weights and Biases" for consistency.

[neverett]

Suggested change

	The Weights & Biases MLOps platform helps AI developers streamline their ML workflow from end-to-end. This plugin
	The Weights and Biases MLOps platform helps AI developers streamline their ML workflows from end to end. This plugin

[neverett]

Suggested change

	# execution id as Weight and Biases run id. The body of the task is XGBoost training
	# execution id as the Weights and Biases run id. The body of the task is XGBoost training

[neverett]

Suggested change

	# Weights & Biases
	# Weights and Biases

[neverett]

Suggested change

	* - {doc}`Weights & Biases <auto_examples/wandb_plugin/index>`
	* - {doc}`Weights and Biases <auto_examples/wandb_plugin/index>`

[neverett]

Suggested change

	# To enable dynamic log links add plugin to Flyte's configuration file:
	# To enable dynamic log links, add the plugin to Flyte's configuration file:

[cosmicBboy]

Creating virtualenv at: .venv
  × No solution found when resolving dependencies:
  ╰─▶ Because flytekitplugins-wandb was not found in the package registry and
      you require flytekitplugins-wandb, we can conclude that the requirements
      are unsatisfiable.

looks like we need to publish this on pypi?

[thomasjpfan]

looks like we need to publish this on pypi?

Yea, flyteorg/flytekit#2405 needs to be merged first and we need to do a release.

[thomasjpfan]

To test you can replace the imagespec with:

@task(
    container_image="ghcr.io/thomasjpfan/wandb:0.0.4",
    secret_requests=[wandb_secret],
)
@wandb_init(project=WANDB_PROJECT, entity=WANDB_ENTITY, secret=wandb_secret)

If you want to use the image builder:

REGISTRY = "localhost:30000"

wandb_package = "git+https://github.com/thomasjpfan/flytekit.git@925f2a3403f605f4ef018893b99ae68ec81fdae4#egg=flytekitplugins-wandb&subdirectory=plugins/flytekit-wandb"

image = ImageSpec(
    name="wandb_example",
    apt_packages=["git"],
    python_version="3.11",
    packages=["xgboost", "scikit-learn", wandb_package],
    registry=REGISTRY,
)