Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replacing indices with references during parsing. #383

Open
vext01 opened this issue Dec 15, 2023 · 0 comments
Open

Replacing indices with references during parsing. #383

vext01 opened this issue Dec 15, 2023 · 0 comments

Comments

@vext01
Copy link
Contributor

vext01 commented Dec 15, 2023

Hi,

We have a parser for a compiler IR where we use indices for the (immutable) on-disk serialisation format, and we read it in with Deku.

We'd like to be able to convert the indices to immutable references for ease of use once parsing is complete.

I mocked this up:

#![allow(dead_code)]

use deku::prelude::*;

#[deku_derive(DekuRead)]
#[derive(Debug)]
struct TopLevel {
    num_foos: usize,
    #[deku(count = "num_foos")]
    foos: Vec<Foo>,

    num_bars: usize,
    #[deku(count = "num_bars")]
    bars: Vec<Bar>,
}

#[deku_derive(DekuRead)]
#[derive(Debug)]
struct Foo {
    val: u8,
}

#[deku_derive(DekuRead)]
#[derive(Debug)]
struct Bar {
    #[deku(temp)]
    foo_idx: usize,
    // XXX: How to we convert the `foo_idx` to a reference to foos[0] in the super-struct?
    //foo_ref: &'a Foo,
}

fn main() {
    // Assume little-endian.
    let data: Vec<u8> = vec![
        // num_foos=1
        1, 0, 0, 0, 0, 0, 0, 0,
        // foos[0]
        // val
        0xaa,
        // num_bars=1
        1, 0, 0, 0, 0, 0, 0, 0,
        // bars[0]
        // foo_idx
        0, 0, 0, 0 ,0, 0, 0, 0
    ];
    let (_rest, mut val) = TopLevel::from_bytes((data.as_ref(), 0)).unwrap();

    dbg!(&val);
}

Is there a way to convert the foo_idx into a reference to foos[0] during parsing?

I can imagine an API that perhaps looks like this (or similar):

#[deku_derive(DekuRead)]
#[deku(endian = "big")]
#[derive(Debug)]
struct Bar {
    #[deku(temp)]
    foo_idx: usize,
    #[deku(skip, map = "|_| -> &deku_parent.foos[foo_idx]")]
    foo_ref: &Foo,
}

Supposing deku had the deku_parent thing (which can probably be done already, albeit verbosely, with ctx), something tells me that this wouldn't sit well with the borrow checker. The parent struct is still being parsed and therefore needs to be mutable, but we are making immutable references to it.

skipping foo_ref also requires &Foo to have a Default, which makes no sense. Maybe you'd have to use raw pointers to allow a NULL pointer...

Another option is to not to the conversion during parsing, but to post-process it, or generate some kind of "index to reference" map after the fact.

But anyway, I'm wondering, Is there an idiom for doing this kind of thing with deku?

Thanks

@vext01 vext01 changed the title Replaceing indices with references during parsing. Replacing indices with references during parsing. Dec 19, 2023
vext01 added a commit to vext01/yk_pv that referenced this issue Dec 19, 2023
The JIT IR is designed to be small. There are two kinds of instruction:
 - short instructions, with inlined operands.
 - long ones (unimplemented as of yet)

Ideally, function, block and instruction IDs (which are indices read
from the on-disk AOT IR) would be either converted to references as we
decode, but this would require changes to the decoder. We may have to
ditch deku:

sharksforarms/deku#383

Co-authored-by: Lukas Diekmann <[email protected]>
vext01 added a commit to vext01/yk_pv that referenced this issue Dec 20, 2023
The JIT IR is designed to be small. There are two kinds of instruction:
 - short instructions, with inlined operands.
 - long ones (unimplemented as of yet)

Ideally, function, block and instruction IDs (which are indices read
from the on-disk AOT IR) would be either converted to references as we
decode, but this would require changes to the decoder. We may have to
ditch deku:

sharksforarms/deku#383

Co-authored-by: Lukas Diekmann <[email protected]>
nmdis1999 pushed a commit to nmdis1999/yk_fork that referenced this issue Jan 29, 2024
The JIT IR is designed to be small. There are two kinds of instruction:
 - short instructions, with inlined operands.
 - long ones (unimplemented as of yet)

Ideally, function, block and instruction IDs (which are indices read
from the on-disk AOT IR) would be either converted to references as we
decode, but this would require changes to the decoder. We may have to
ditch deku:

sharksforarms/deku#383

Co-authored-by: Lukas Diekmann <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant