Embedding in web applications

[arthuro555]

Hello 👋

I was thinking that it could be cool if a "full SurrealDB node" could be embedded with wasm inside a web app, as a layer between the application logic and localStorage/indexedDB/whatever browser storage solution. That would allow to access all of the SurrealDB goodness, like the Query language, relationships, etc. from Web Applications meant to run fully client side. It would also be cool in development, as it would allow to get developing instantly with a persistent database running in the browser without needing to go spin up a development instance externally.
I get that this is something that is easier said than done, but I think that would be a pretty cool thing to have nonetheless 🤩

[arthuro555]

I only now realized that this is already a thing 😅 Just need to put indxdb: as the protocol in the WASM library, apparently, although I have not tried it out yet: https://github.com/surrealdb/surrealdb.wasm/blob/main/src/app/mod.rs#L46

[mathe42]

Did not work 100% reliable... But probably my fault at building it...

[james-innes]

Nice extension 👍 One less click for anyone wanting to see the code for this. Would be nice to have a tiny demo site or Gist come up in search results or some documentation to show it can be done. More info from Tobie here on Discord.

[arthuro555]

Note that this is an in-memory database, it is not persisted - you should be able to make one persisted to indexed db by replacing memory by indxdb:storagekey

[maxbaluev]

But how do I make it work in a browser and not in a node, i need it in service worker of chrome extension.

[ibilux]

I only now realized that this is already a thing sweat_smile Just need to put indxdb: as the protocol in the WASM library, apparently, although I have not tried it out yet: https://github.com/surrealdb/surrealdb.wasm/blob/main/src/app/mod.rs#L46

Can you provide a complete example ?

[johannesmutter]

Would love this too for building offline/local-first apps (e.g. using Tauri).

So instead of writing queries for a local state management solution like xState, Valtio or Redux or something like PouchDB/ IndexedDB, I’d prefer to have a WebAssembly powered instance of SurrealDB running in the browser, so I have to write all queries only once. It would be extra cool if the local instance would synchronize to the cloud version.

Quote from @tobiemh (source - 9 months ago)

We are close to releasing our WebAssembly release, which will enable SurrealDB to run in the browser, using IndexedDB as a persistent backing store. Eventually, this in-browser version will sync to the cloud allowing for offline and online data syncing for applications.

Related:
Discussion: Offline Capabilities? #107

Discussion: Offline Database #1499

Discussion: Support for optimistic UI? #1449

Github: SurrealDB driver for WebAssembly

Docs: Embedding SurrealDB in JavaScript

[justin0mcateer]

It would be really nice to have a client that allows running Surreal in a worker with the clients sitting on the main thread.

[arthuro555]

Nothing stops you from doing that easily? That's nothing you need surreal to do for you, you can just do it yourself with something like this:

// worker.js
import { Surreal } from 'surrealdb.wasm';

const db = new Surreal();
await db.connect("indxdb:main.db");
await db.signin({ username: "root", password: "root" });
await db.use({ namespace: "test", database: "test" });

self.onmessage = async (e) => {
  const { id, query, params } = event.data;
  const results = await db.query(query, params);
  postMessage({ id, results });
}

// surreal.js
const worker = new Worker("./worker.js");

// UID generation that should be safe to use on multiple windows for if you were to use a SharedWorker instead
const time = Date.now();
const constant = time.toString().slice(6, -1) + Math.random().toString().slice(15);
const generateId = () => constant + "" + (Date.now()-time) + Math.random().toString().slice(15);

function waitForWorkerResponse(id) {
  return new Promise((resolve => {
    worker.addEventsListener("message", function listener({ data }) {
      if (data.id !== id) return;
      worker.removeEventsListener("message", listener);
      resolve(data.results)
    })
  })
}

export async function query(query, params) {
  const id = generateId();
  worker.postMessage({ id, query, params });
  return await waitForWorkerResponse(id);
}

// index.js
import { query } from "./surreal.js";

export async function doSomething() {
  return await query(
    "SELECT marketing, count() FROM type::table($table) GROUP BY marketing", 
    { table: "person" }
  );
}