Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement full batch and transaction support #4358

Open
wants to merge 43 commits into
base: main
Choose a base branch
from

Conversation

mattwiller
Copy link
Member

No description provided.

Copy link

vercel bot commented Apr 11, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
medplum-provider ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 30, 2024 11:34pm
medplum-storybook ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 30, 2024 11:34pm
2 Ignored Deployments
Name Status Preview Comments Updated (UTC)
medplum-app ⬜️ Ignored (Inspect) Visit Preview May 30, 2024 11:34pm
medplum-www ⬜️ Ignored (Inspect) Visit Preview May 30, 2024 11:34pm

Copy link
Member

@codyebberson codyebberson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Structurally, this looks great to me.

@@ -143,31 +183,43 @@ async function patchResource(req: FhirRequest, repo: FhirRepository): Promise<Fh
return [allOk, resource];
}

export type RestInteraction =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low pri mental note: we may want to try to merge this with packages/server/src/util/auditevent.ts, either here in fhir-router or in core.

@@ -143,31 +183,43 @@ async function patchResource(req: FhirRequest, repo: FhirRepository): Promise<Fh
return [allOk, resource];
}

export type RestInteraction =
| CapabilityStatementRestInteraction['code']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low pri mental note: This is another case where having a proper type for a ValueSet-derived string union would have been helpful, so you could use CapabilityStatementRestInteractionCode rather than CapabilityStatementRestInteraction['code'] -- I'm pretty sure the former would be faster for tsc, and probably more discoverable.

this.router.add('DELETE', ':resourceType/:id', deleteResource);
this.router.add('PATCH', ':resourceType/:id', patchResource);
this.router.add('POST', '$graphql', graphqlHandler);
this.router.add('GET', '', searchMultipleTypes, { interaction: 'search-system' });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely starting to paint a clearer picture for the future.

Something like:

  • New outer type such as FhirProcessor which wraps FhirRouter and FhirRepository
  • registerHandler(handler: OperationHandler) which adds routes and whatnot

I'm not sure if FHIR has a good term that encompasses both interaction and operation ?

@dillonstreator
Copy link
Contributor

dillonstreator commented Apr 12, 2024

This is exciting to see taking shape!

I have a few questions/concerns to better understand and expectation set.

@mattwiller
Copy link
Member Author

@dillonstreator Great questions! At this point, strict uniqueness guarantees for conditional create aren't part of the implementation: it's a hard problem to solve perfectly, as you've pointed out with the concerns around table locking. The FHIR spec is somewhat vague about exactly what the guarantees for this functionality should be, but it's something I'm planning to explore as a fast follow after the initial implementation is complete. In any case, I don't think that we're likely to go with a heavyweight solution like whole table locking — for exactly the scale/performance concerns you mentioned. I'll need to spend some time thinking through other options and the correctness/performance tradeoffs around them

@dillonstreator
Copy link
Contributor

dillonstreator commented Apr 15, 2024

Thank you for the transparency, @mattwiller. I appreciate the thoughtfulness being put into this and the commitment to addressing the uniqueness needs.
As a stop-gap, to enable the uniqueness guarantees that I need, I'm looking at standing up a Redis instance to employ arbitrary locking.
On the Medplum side, this might be a interesting pattern to emulate table locking and could be namespaced on project id to prevent cross-project disruptions.

Copy link
Member

@codyebberson codyebberson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks great. Thanks for slogging through 🙏

Added initial feedback. I want to review more in-depth when I get some time.

One immediate thing we need to add is disabling "read from Redis" when in the middle of a transaction. Otherwise things could get weird (I think?)

packages/fhir-router/src/batch.ts Outdated Show resolved Hide resolved
try {
resolved = await this.resolveIdentity(entry, `Bundle.entry[${index}]`);
} catch (err: any) {
if ((err as OperationOutcomeError).outcome && this.bundle.type !== 'transaction') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, this as OperationOutcomeError seems risky, although I'm not sure why. I've been trying to always use normalizeOperationOutcome or whatever to avoid casts in catch.

Suggested change
if ((err as OperationOutcomeError).outcome && this.bundle.type !== 'transaction') {
if (err instanceof OperationOutcomeError && this.bundle.type !== 'transaction') {

return this.processModification(entry, path);
case 'GET':
case 'HEAD':
// Ignore read-only requests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, I'm surprised this is is valid, and that these are even real values in the Bundle.entry.request.method value set.

const route = this.router.find(entry.request?.method as HttpMethod, requestPath);
const params = route?.params;

switch (route?.data?.interaction) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I need to review this more, but I would have assumed that most of this switch statement would have been handled by handleRequest - if nothing else, a lot of these checks are duplicated, which would be nice to clean up if possible.

    const result = await this.router.handleRequest(
      {
        method: request.method as HttpMethod,
        pathname: url.pathname,
        params: Object.create(null),
        query: Object.fromEntries(url.searchParams),
        body,
      },
      this.repo
    );

packages/server/src/config.ts Outdated Show resolved Hide resolved
Copy link

sonarcloud bot commented May 23, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fhir-datastore Related to the FHIR datastore, includes API and FHIR operations
Projects
Status: 👀 In review
Development

Successfully merging this pull request may close these issues.

None yet

4 participants