Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor StoreHub to fix bugs around blueprint persistence #5849

Draft
wants to merge 49 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
63afebd
Introduce blueprint APIs for connect/spawn (#5814)
jleibs Apr 5, 2024
fb7e9e8
Remove aspect ratio check from `upload_image.py` (#5806)
abey79 Apr 5, 2024
f0661d8
nuScenes: add blueprint info + minor cleaning (#5813)
abey79 Apr 5, 2024
7295e77
Objectron: add blueprint info + minor cleaning (#5810)
abey79 Apr 5, 2024
ceb48a3
Tune copy on welcome page (#5811)
nikolausWest Apr 5, 2024
b5ca805
Improve documentation previews (#5805)
jprochazk Apr 5, 2024
5ab86d9
Bump versions to 0.15.0-rc.1
rerun-bot Apr 2, 2024
3f4d2f9
Stop using `<h1>` in example README.md (#5817)
abey79 Apr 5, 2024
04f3c94
Add changelog for the unreleased 0.15 (#5809)
emilk Apr 5, 2024
6ffda0a
Add dep for docs-path-filter job (#5818)
jleibs Apr 5, 2024
0be4d3f
Bump versions to 0.15.0-rc.2 (#5819)
rerun-bot Apr 5, 2024
773f5f6
More documentation tweaks as requested (#5821)
jleibs Apr 7, 2024
e1126db
Misc example readme fixes (#5824)
abey79 Apr 7, 2024
e3686c3
Update VRS thumbnail
emilk Apr 7, 2024
63641b4
Update roadmap for 0.15 (#5829)
nikolausWest Apr 7, 2024
53015da
Iteration on What is Rerun? page (#5828)
nikolausWest Apr 7, 2024
660bf56
Remove `description` from example frontmatter (#5832)
emilk Apr 7, 2024
d0ef14b
Updated READMEs for the examples - Batch 3 (#5784)
andreasnaoum Apr 7, 2024
63c7943
Request attention when Rerun Viewer is sent new recording in backgrou…
emilk Apr 7, 2024
2f62bfd
Fix video elements in docs (#5826)
jprochazk Apr 7, 2024
27623e3
Update blueprint shortcut
emilk Apr 8, 2024
9fc4e76
Fix/improve UI when viewing all components of an entity (#5831)
emilk Apr 8, 2024
76c64a8
Add preambles to the Paper Walkthroughs' READMEs (#5833)
nikolausWest Apr 8, 2024
686c6b4
Update concept section (#5834)
abey79 Apr 8, 2024
8071eed
0.15 migration guide for removed instance keys (#5830)
emilk Apr 8, 2024
32da395
Mention `log_tick` in timeline docs
emilk Apr 8, 2024
9f51f1b
Title casing -> Human casing
emilk Apr 8, 2024
d321387
Do not show the remove button when the origin is empty (#5840)
abey79 Apr 8, 2024
b9da1b1
Release 0.15.0 (#5843)
rerun-bot Apr 8, 2024
d7a672e
Support displaying 1D tensors (#5837)
Wumpf Apr 8, 2024
68e8d73
Add inline markdown example description to public examples (#5855)
emilk Apr 8, 2024
808bc27
Pin `mediapipe` to 0.10.9 to fix Human Pose Tracking example (#5861)
abey79 Apr 8, 2024
141d353
Add schematics to the blueprint section (#5851)
abey79 Apr 9, 2024
381ec1c
Refactor: move blueprint->message generation to the StoreHub
emilk Apr 8, 2024
f7e3454
Refactor: clump active and default blueprint into one struct
emilk Apr 8, 2024
67d1a6b
Refactor: split blueprint gc and blueprint persist into two calls
emilk Apr 8, 2024
8ff7088
Correctly persist cleared state
emilk Apr 8, 2024
da1a6dd
Add profile scopes
emilk Apr 8, 2024
5d7b0fc
Fix wrong logging
emilk Apr 8, 2024
2a4e371
GPU allocations are not worth noting
emilk Apr 8, 2024
a4ee293
Refactor: make the active app id non-default
emilk Apr 8, 2024
f622c36
Fix typo
emilk Apr 8, 2024
49da95b
Fix bug in `remove_recording_by_uri`
emilk Apr 8, 2024
662db21
Turn on debug logging when debugging via VSCode
emilk Apr 9, 2024
462e9c1
Save blueprints before removing them
emilk Apr 9, 2024
77a5d5c
Keep blueprints around
emilk Apr 9, 2024
b362fc6
Cleanup
emilk Apr 9, 2024
00a6efb
Wasm/clippy fixes
emilk Apr 9, 2024
80b540c
Lint against tabs
emilk Apr 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 11 additions & 3 deletions .github/actions/deploy-vercel/action.yml
@@ -1,6 +1,10 @@
# This action handles updating the target commit env variable (`RELEASE_COMMIT`)
# which is used as the pointer for `rerun.io/docs` and `rerun.io/examples`
# and triggering a redeploy of `rerun.io`.
# If `target` is set to `production`, this action handles updating the
# target commit env variable (`RELEASE_COMMIT`) which is used as the
# pointer for `rerun.io/docs` and `rerun.io/examples` and triggering
# a redeploy of `rerun.io`.

# If `target` is set to `preview`, then this instead deploys a fresh preview
# with an override for `release_commit`, and sets the `vercel_preview_url` output.

name: "Deploy rerun.io"

Expand All @@ -25,6 +29,10 @@ inputs:
release_version:
description: "Which release version to update the deployment to"
type: string
required: false
target:
description: "Which Vercel environment to deploy to"
type: string
required: true

runs:
Expand Down
77 changes: 23 additions & 54 deletions .github/actions/deploy-vercel/index.mjs
@@ -1,63 +1,32 @@
// @ts-check

import { Client } from "./vercel.mjs";
import { assert, getRequiredInput, info } from "./util.mjs";
import { assert, getInput, getRequiredInput } from "./util.mjs";
import { deployToProduction } from "./production.mjs";
import { deployToPreview } from "./preview.mjs";

// These inputs are defined in `action.yml`, and should be kept in sync
const token = getRequiredInput("vercel_token");
const teamName = getRequiredInput("vercel_team_name");
const projectName = getRequiredInput("vercel_project_name");
const releaseCommit = getRequiredInput("release_commit");
const releaseVersion = getRequiredInput("release_version");
const team = getRequiredInput("vercel_team_name");
const project = getRequiredInput("vercel_project_name");
const commit = getRequiredInput("release_commit");
const target = getRequiredInput("target");

const client = new Client(token);

info`Fetching team "${teamName}"`;
const availableTeams = await client.teams();
assert(availableTeams, `failed to get team "${teamName}"`);
const team = availableTeams.find((team) => team.name === teamName);
assert(team, `failed to get team "${teamName}"`);

info`Fetching project "${projectName}"`;
const projectsInTeam = await client.projects(team.id);
const project = projectsInTeam.find((project) => project.name === projectName);
assert(project, `failed to get project "${projectName}"`);

info`Fetching latest production deployment`;
const productionDeployments = await client.deployments(team.id, project.id);
const latestProductionDeployment = productionDeployments[0];
assert(
latestProductionDeployment,
`failed to get latest production deployment`,
);

const environment = await client.envs(team.id, project.id);
const RELEASE_COMMIT_KEY = "RELEASE_COMMIT";
const RELEASE_VERSION_KEY = "RELEASE_VERSION";

info`Fetching "${RELEASE_COMMIT_KEY}" env var`;
const releaseCommitEnv = environment.find(
(env) => env.key === RELEASE_COMMIT_KEY,
);
assert(releaseCommitEnv, `failed to get "${RELEASE_COMMIT_KEY}" env var`);

info`Fetching "${RELEASE_VERSION_KEY}" env var`;
const releaseVersionEnv = environment.find(
(env) => env.key === RELEASE_VERSION_KEY,
);
assert(releaseVersionEnv, `failed to get "${RELEASE_VERSION_KEY}" env var`);

info`Setting "${RELEASE_COMMIT_KEY}" env to "${releaseCommit}"`;
await client.setEnv(team.id, project.id, releaseCommitEnv.id, {
key: RELEASE_COMMIT_KEY,
value: releaseCommit,
});

info`Setting "${RELEASE_VERSION_KEY}" env to "${releaseVersion}"`;
await client.setEnv(team.id, project.id, releaseVersionEnv.id, {
key: RELEASE_VERSION_KEY,
value: releaseVersion,
});

info`Triggering redeploy`;
await client.redeploy(team.id, latestProductionDeployment.uid, "landing");
switch (target) {
case "production": {
const version = getRequiredInput("release_version");
await deployToProduction(client, { team, project, commit, version });
break;
}

case "preview": {
await deployToPreview(client, { team, project, commit });
break;
}

default: {
throw new Error(`"target" must be one of: production, preview`);
}
}
49 changes: 49 additions & 0 deletions .github/actions/deploy-vercel/preview.mjs
@@ -0,0 +1,49 @@
// @ts-check

import { assert, info, setOutput } from "./util.mjs";
import { Client } from "./vercel.mjs";

/**
*
* @param {Client} client
* @param {{
* team: string;
* project: string;
* commit: string;
* }} options
*/
export async function deployToPreview(client, options) {
info`Fetching team "${options.team}"`;
const availableTeams = await client.teams();
assert(availableTeams, `failed to get team "${options.team}"`);
const team = availableTeams.find((team) => team.name === options.team);
assert(team, `failed to get team "${options.team}"`);

info`Fetching project "${options.project}"`;
const projectsInTeam = await client.projects(team.id);
const project = projectsInTeam.find(
(project) => project.name === options.project,
);
assert(project, `failed to get project "${options.project}"`);

info`Fetching latest production deployment`;
const productionDeployments = await client.deployments(team.id, project.id);
const latestProductionDeployment = productionDeployments[0];
assert(
latestProductionDeployment,
`failed to get latest production deployment`,
);

info`Deploying preview with RELEASE_COMMIT=${options.commit}`;
const { url } = await client.deployPreviewFrom(
team.id,
latestProductionDeployment.uid,
"landing-preview",
{
RELEASE_COMMIT: options.commit,
IS_PR_PREVIEW: "true",
},
);

setOutput("vercel_preview_url", url);
}
68 changes: 68 additions & 0 deletions .github/actions/deploy-vercel/production.mjs
@@ -0,0 +1,68 @@
// @ts-check

import { assert, info } from "./util.mjs";
import { Client } from "./vercel.mjs";

/**
*
* @param {Client} client
* @param {{
* team: string;
* project: string;
* commit: string;
* version: string;
* }} options
*/
export async function deployToProduction(client, options) {
info`Fetching team "${options.team}"`;
const availableTeams = await client.teams();
assert(availableTeams, `failed to get team "${options.team}"`);
const team = availableTeams.find((team) => team.name === options.team);
assert(team, `failed to get team "${options.team}"`);

info`Fetching project "${options.project}"`;
const projectsInTeam = await client.projects(team.id);
const project = projectsInTeam.find(
(project) => project.name === options.project,
);
assert(project, `failed to get project "${options.project}"`);

info`Fetching latest production deployment`;
const productionDeployments = await client.deployments(team.id, project.id);
const latestProductionDeployment = productionDeployments[0];
assert(
latestProductionDeployment,
`failed to get latest production deployment`,
);

const environment = await client.envs(team.id, project.id);
const RELEASE_COMMIT_KEY = "RELEASE_COMMIT";
const RELEASE_VERSION_KEY = "RELEASE_VERSION";

info`Fetching "${RELEASE_COMMIT_KEY}" env var`;
const releaseCommitEnv = environment.find(
(env) => env.key === RELEASE_COMMIT_KEY,
);
assert(releaseCommitEnv, `failed to get "${RELEASE_COMMIT_KEY}" env var`);

info`Fetching "${RELEASE_VERSION_KEY}" env var`;
const releaseVersionEnv = environment.find(
(env) => env.key === RELEASE_VERSION_KEY,
);
assert(releaseVersionEnv, `failed to get "${RELEASE_VERSION_KEY}" env var`);

info`Setting "${RELEASE_COMMIT_KEY}" env to "${options.commit}"`;
await client.setEnv(team.id, project.id, releaseCommitEnv.id, {
key: RELEASE_COMMIT_KEY,
value: options.commit,
});

info`Setting "${RELEASE_VERSION_KEY}" env to "${options.version}"`;
await client.setEnv(team.id, project.id, releaseVersionEnv.id, {
key: RELEASE_VERSION_KEY,
value: options.version,
});

info`Triggering redeploy`;
await client.redeploy(team.id, latestProductionDeployment.uid, "landing");
}
38 changes: 38 additions & 0 deletions .github/actions/deploy-vercel/test.mjs
@@ -0,0 +1,38 @@
// @ts-check

import { Client } from "./vercel.mjs";
import { assert, info } from "./util.mjs";

const client = new Client("NzuZ9WBTnfUGiwHrhd7mit2E");

const teamName = "rerun";
const projectName = "landing";

info`Fetching team "${teamName}"`;
const availableTeams = await client.teams();
assert(availableTeams, `failed to get team "${teamName}"`);
const team = availableTeams.find((team) => team.name === teamName);
assert(team, `failed to get team "${teamName}"`);

info`Fetching project "${projectName}"`;
const projectsInTeam = await client.projects(team.id);
const project = projectsInTeam.find((project) => project.name === projectName);
assert(project, `failed to get project "${projectName}"`);

info`Fetching latest production deployment`;
const productionDeployments = await client.deployments(team.id, project.id);
const latestProductionDeployment = productionDeployments[0];
assert(
latestProductionDeployment,
`failed to get latest production deployment`,
);

const response = await client.deployPreviewFrom(
team.id,
latestProductionDeployment.uid,
"rerun-custom-preview-test",
{
RELEASE_COMMIT: "main",
},
);
console.log(response);
14 changes: 13 additions & 1 deletion .github/actions/deploy-vercel/util.mjs
@@ -1,5 +1,8 @@
// @ts-check

import { appendFileSync } from "fs";
import os from "os";

/**
* Log a message with level `INFO`
*
Expand Down Expand Up @@ -41,6 +44,16 @@ export function getRequiredInput(name) {
return input;
}

/**
* Set a GitHub Actions output for other workflows steps to read.
* @param {string} key
* @param {string} value
*/
export function setOutput(key, value) {
const outputFile = /** @type {string} */ (process.env["GITHUB_OUTPUT"]);
appendFileSync(outputFile, `${key}=${value}${os.EOL}`);
}

/**
* Assert that `value` is truthy, throwing an error if it is not.
*
Expand All @@ -61,4 +74,3 @@ export function assert(value, message) {
throw new Error(error);
}
}