Proposal: New API + set of types for working with blocks in a type-safe way

[adroitwhiz]

There are a lot of issues with the current block/opcode API that we seem to keep running up against:

• There's a large amount of duplicate data stored about each block. For instance, we define a BlockBase type signature for each individual block, but also store that data at runtime in KnownBlockInputMap.
• We're missing out on a lot of type information, and need to add lots of type assertions to the code based on the block's opcode.
  • In particular, there are cases where the compiler gets upset at us because we can't statically "prove" that a certain opcode maps to a given set of block inputs.
• getDefaultInput is very loosely typed, as it cannot make use of said type information.
• It's hard to extend things like getDefaultInput because they depend on a built-in list of known blocks.
• We cannot validate blocks' fields at runtime; we just have to type-assert them to the correct types and hope they weren't serialized with any fields missing or of the incorrect types.

I propose to fix this by moving blocks' opcode + input data into runtime-accessible "type objects". Each defined block would have an immutable "block prototype" instance, which can be queried both at runtime and compile-time (since it's immutable, the TypeScript compiler can read its fields). This solves our issues nicely:

• Block data is defined in one place only, solely by defining the block prototype. We can perform introspection on them using TypeScript's typeof operator (not to be confused with JavaScript's runtime typeof), which allows us to provide static-type guarantees.
• We can test whether a block is an instance of a (statically-defined) block prototype using a type predicate, and since we know the block's intended fields and their types at runtime, this lets us safely interact with unknown blocks.
• Users could potentially define their own block prototypes.

Here's some proof-of-concept code I wrote which demonstrates this approach:

Type definitions w/ demo code

/**
 * Maps a block input interface (e.g. {type: "number", value: string | number}) to the corresponding block-prototype
 * input's interface (e.g. {type: "number", initial: string | number}), which defines the input's initial value.
 */
type ProtoInput<Input extends BlockInput.Any> = Input extends BlockInput.Any ?
	// Needed to distribute over the union, so that `type` must belong to the same union branch as `value`.
	// See https://www.typescriptlang.org/docs/handbook/release-notes/typescript-2-8.html#distributive-conditional-types
	{type: Input['type'], initial: Input['value']} :
	never;

/**
 * Runtime type information for a block with a certain opcode, which tells us what its inputs are and what their
 * default values should be.
 */
type BlockPrototype<
	OpCode extends string = string,
	Inputs extends {[x: string]: ProtoInput<BlockInput.Any>} = {[x: string]: ProtoInput<BlockInput.Any>}
> = Readonly<{
	opcode: OpCode;
	inputs: Inputs;
}>;

/**
 * Instance of a block, with a given opcode and inputs.
 */
type Block<
	OpCode extends string = string,
	Inputs extends {[x: string]: BlockInput.Any} = {[x: string]: BlockInput.Any}
> = {
	opcode: OpCode;
	inputs: Inputs;
};

/**
 * Maps a type BlockPrototype<opcode, default inputs> to the corresponding Block<opcode, inputs>.
 */
type BlockForPrototype<P extends BlockPrototype> =
	// Infer the prototype's opcode and default inputs
	P extends BlockPrototype<infer OpCode, infer Defaults> ?
		Block<OpCode, {
			// The type of the input is whichever types in the BlockInput.Any union overlap with the default field's
			// "type" (found using the "&" operator).
			[K in keyof Defaults]: BlockInput.Any & {type: Defaults[K]['type']}
		}> :
		never;

/**
 * Check whether a block instance is assignable to a given block prototype.
 * @param proto The block prototype to check against.
 * @param block The given block instance.
 * @returns true if the block has the same opcode and inputs as the block prototype
 */
function blockMatchesProto<V extends BlockPrototype> (proto: V, block: Block): block is BlockForPrototype<V> {
	if (block.opcode !== proto.opcode) return false;
	for (const [inputName, inputTypeAndInitial] of Object.entries(proto.inputs)) {
		if (!(inputName in block.inputs)) return false;
		const input = block.inputs[inputName];
		if (input.type !== inputTypeAndInitial.type) return false;
	}
	return true;
}

// We can define a block prototype, and the compiler will infer a type for it and ensure it satisfies the properties of
// a BlockPrototype!
const MotionMoveSteps = {
	opcode: 'motion_movesteps',
	inputs: {
		STEPS: {
			type: 'number',
			initial: 10
		}
	}
} as const satisfies BlockPrototype;

const InvalidExample1 = {
	opcode: 'invalid_block',
	inputs: {
		STEPS: {
			type: 'number',
			initial: 10,
			// The compiler will reject this (field doesn't exist on ProtoInput<T>)
			oops: 999
		}
	}
} as const satisfies BlockPrototype;

const InvalidExample2 = {
	opcode: 'invalid_block',
	inputs: {
		// The compiler will reject this (missing "type" and "initial")
		STOMPS: {}
	}
} as const satisfies BlockPrototype;

const InvalidExample3 = {
	opcode: 'invalid_block',
	inputs: {
		STOMPS: {
			type: 'number',
			// The compiler will reject this (type `boolean` not assignable to `string | number`)
			initial: true
		}
	}
} as const satisfies BlockPrototype;

// We can guarantee that this function returns either a motion_movesteps block or nothing at all!
function foo (block: Block): BlockForPrototype<typeof MotionMoveSteps> | null {
	if (blockMatchesProto(MotionMoveSteps, block)) {
		return block;
	}
	return null;
}

// We can safely access the block's inputs!
function bar (block: Block): void {
	if (blockMatchesProto(MotionMoveSteps, block)) {
		// We can even tell the type and value of the inputs!
		const steps: string | number = block.inputs.STEPS.value;
		const inputType: 'number' = block.inputs.STEPS.type;
		console.log(steps, inputType);
	}
}

[towerofnix]

Here's your notes from #94, for reference:

(When deserializing [sb3], null block inputs are skipped completely for now) I'll revisit this later. If there's, for instance, an empty C-block inside a .sb3 file, its SUBSTACK will be either null or completely absent in the .sb3's block inputs, depending on whether a stack was ever dragged into the C-block. We previously used to treat the two cases differently, treating the input as {type: "string", value: null} in the former case (flat-out wrong), and leaving the input out in the latter case. Now, we always leave the input out. It'll be easier to fix this properly once we have a better way to enumerate a block's intended inputs-- see #93.

[towerofnix]

@adroitwhiz I'm trying to understand how this issue and #100 are separate a bit better. It seems like they largely address the same areas, reworking overall API structure to improve type safeness, expand library capabilities, and tidy code style and structure legibility. I'm interested if you feel there are separate things/goals they address, or if #100 was more of a follow-up detailing a specific example of why the overall rework discussed here is necessary?

[adroitwhiz]

#100 was something I discovered after writing this issue. I think this issue better describes my proposed solution (in retrospect)

[towerofnix]

Yeah, it felt more like a diagnosis of an issue via description of symptoms rather than a detailed way forward, which is mostly covered here.