Weights & Biases Google Module

This is a Terraform module for provisioning a Weights & Biases Cluster on Google Cloud. Weights & Biases Local is our self-hosted distribution of wandb.ai. It offers enterprises a private instance of the Weights & Biases application, with no resource limits and with additional enterprise-grade architectural features like audit logging and single sign-on.

About This Module

Pre-requisites

This module is intended to run in an Google Cloud account with minimal preparation, however it does have the following pre-requisites:

Terrafom version >= 1

Credentials / Permissions

Google Services Used

• Google SQL Cloud (MySQL)
• Google Kubernetes Engine
• Google Storage Bucket
• Google PubSub
• Google Managed Certificates
• Google Cloud DNS

How to Use This Module

• Ensure account meets module pre-requisites from above.
• Create a Terraform configuration that pulls in this module and specifies values of the required variables:

provider "google" {
  project = "<desired google project>"
  region = "<desired google region>"
  zone = "<desired google zone>"
}

module "wandb" {
  source    = "<filepath to cloned module directory>"
  namespace = "<prefix for naming google resources>"
}

• Run terraform init and terraform apply

Examples

We have included documentation and reference examples for common installation scenarios, as well as examples for supporting resources that lack official modules.

• Public Instance with HTTPS using Cloud DNS

Requirements

Name	Version
terraform	~> 1.0
google	~> 4.82
helm	~> 2.10
kubernetes	~> 2.23

Providers

No providers.

Modules

Name	Source	Version
app_gke	./modules/app_gke	n/a
app_lb	./modules/app_lb	n/a
database	./modules/database	n/a
gke_app	wandb/wandb/kubernetes	1.14.1
kms	./modules/kms	n/a
networking	./modules/networking	n/a
project_factory_project_services	terraform-google-modules/project-factory/google//modules/project_services	~> 13.0
redis	./modules/redis	n/a
service_accounts	./modules/service_accounts	n/a
storage	./modules/storage	n/a
wandb	wandb/wandb/helm	1.2.0

Resources

No resources.

Inputs

Name	Description	Type	Default	Required
allowed_inbound_cidrs	Which IPv4 addresses/ranges to allow access. This must be explicitly provided, and by default is set to ["*"]	list(string)	[ "*" ]	no
app_wandb_env	Extra environment variables for W&B	map(string)	{}	no
bucket_name	Use an existing bucket.	string	""	no
create_redis	Boolean indicating whether to provision an redis instance (true) or not (false).	bool	false	no
database_machine_type	Specifies the machine type to be allocated for the database	string	"db-n1-standard-2"	no
database_sort_buffer_size	Specifies the sort_buffer_size value to set for the database	number	67108864	no
database_version	Version for MySQL	string	"MYSQL_8_0_31"	no
deletion_protection	If the instance should have deletion protection enabled. The database / Bucket can't be deleted when this value is set to true.	bool	true	no
disable_code_saving	Boolean indicating if code saving is disabled	bool	false	no
domain_name	Domain for accessing the Weights & Biases UI.	string	null	no
force_ssl	Enforce SSL through the usage of the Cloud SQL Proxy (cloudsql://) in the DB connection string	bool	false	no
gke_machine_type	Specifies the machine type to be allocated for the database	string	"n1-standard-4"	no
gke_node_count	n/a	number	2	no
labels	Labels to apply to resources	map(string)	{}	no
license	Your wandb/local license	string	n/a	yes
local_restore	Restores W&B to a stable state if needed	bool	false	no
namespace	String used for prefix resources.	string	n/a	yes
network	Pre-existing network self link	string	null	no
oidc_auth_method	OIDC auth method	string	"implicit"	no
oidc_client_id	The Client ID of application in your identity provider	string	""	no
oidc_issuer	A url to your Open ID Connect identity provider, i.e. https://cognito-idp.us-east-1.amazonaws.com/us-east-1_uiIFNdacd	string	""	no
oidc_secret	The Client secret of application in your identity provider	string	""	no
other_wandb_env	Extra environment variables for W&B	map(string)	{}	no
parquet_wandb_env	Extra environment variables for W&B	map(string)	{}	no
redis_reserved_ip_range	Reserved IP range for REDIS peering connection	string	"10.30.0.0/16"	no
redis_tier	Specifies the tier for this Redis instance	string	"STANDARD_HA"	no
resource_limits	Specifies the resource limits for the wandb deployment	map(string)	{ "cpu": null, "memory": null }	no
resource_requests	Specifies the resource requests for the wandb deployment	map(string)	{ "cpu": "2000m", "memory": "2G" }	no
size	Deployment size for the instance	string	null	no
ssl	Enable SSL certificate	bool	true	no
subdomain	Subdomain for accessing the Weights & Biases UI. Default creates record at Route53 Route.	string	null	no
subnetwork	Pre-existing subnetwork self link	string	null	no
use_internal_queue	Uses an internal redis queue instead of using google pubsub.	bool	false	no
wandb_image	Docker repository of to pull the wandb image from.	string	"wandb/local"	no
wandb_version	The version of Weights & Biases local to deploy.	string	"latest"	no
weave_wandb_env	Extra environment variables for W&B	map(string)	{}	no

Outputs

Name	Description
address	n/a
bucket_name	Name of google bucket.
bucket_queue_name	Pubsub queue created for google bucket file upload events.
cluster_ca_certificate	Certificate of the kubernetes (GKE) cluster.
cluster_client_certificate	n/a
cluster_client_key	n/a
cluster_endpoint	Endpoint of the kubernetes (GKE) cluster.
cluster_id	ID of the kubernetes (GKE) cluster.
cluster_name	n/a
cluster_node_pool	Default node pool where Weights & Biases should be deployed into.
cluster_self_link	Self link of the kubernetes (GKE) cluster.
database_connection_string	Full database connection string. You must be in the VPC to access the database.
database_instance_type	n/a
fqdn	The FQDN to the W&B application
gke_node_count	n/a
gke_node_instance_type	n/a
service_account	Weights & Biases service account used to manage resources.
standardized_size	n/a
url	The URL to the W&B application