Strict Flag: NBTData Option + Auto-Parsing #31
Assignees
Labels
enhancement
New feature or request
Comments
Offroaders123
added a commit
that referenced
this issue
Sep 13, 2023
```ts
const gg = await read(new Uint8Array(),{ name: "Level", endian: "big", compression: null, bedrockLevel: null });
```
This is an experimental demo to add 'generic propagation' to the `read()` function. It doesn't quite work fully because the parameters/options don't quite line up with `Format`, so I'm gonna decide how I want to handle that, since using a boolean for value-based types still seems easiest to use and understand from the user's perspective. I also have to handle the strict flag in the options too. Speaking of that, it might be related to whether the 'strict' flag should propagate to the resulting `NBTData` object & type, which I currently haven't added/fully looked into yet. It may make sense at the type level too though, would definitely be helpful for debugging between types of things! One thing about it is that it would show a type error if you tried opening a non-strict NBT file as a strict one, which thinking of it now, is a great think to also keep track of! Ok, I think I probably will end up adding that with time. Building on that, I may have to add a min file output size/length too, which would ensure that the output file can symmetrically parse to and from it's original content size. If there's any raw unnmanaged bytes in the deadspace for that original file though, those will have to be lost with time.
So together, possible `NBTData` strict and file length flags, which would help for symmetrical reads/writes, and would also be helpful for distinguishing parsed files at the type level too :)
#29
#31
Offroaders123
added a commit
that referenced
this issue
Sep 15, 2023
As with how I've been doing the last few updates, I make a some of the changes from the few previous updates since the last version release, so you can go to each commit about these topics specifically to get more info about what changed, and to see discussion about the process along the way too. I think trying to summarize it again just when I publish it isn't worth it compared to just referencing you towards the original write up about the changes I did the first time around. You'll get more explanations and such, I think it's more worth it :) - ...and optional ListTag values - RootTagLike (& RootTag default return types) - Run-Time List Item Type Check (SNBT) - Format Options Generic Reverting #14 #19 #25 #28 #29 #31 #32
Offroaders123
added a commit
to Offroaders123/Bedrock-LevelDB
that referenced
this issue
Dec 15, 2023
This is really nice now! Here's an explanation I wrote about how the function goes through the buffer: The control flow is that it tried parsing the file outright, asserting that it read to the end of the file If that fails, then it gets the error byte offset from the error message (I am going to make better DX for this from NBTify, in relation to strict mode, this should be accessible as a simple property when parsing a file), then it reads from the current position until it completes, not asserting the end length of the file (non-strict mode), then it trims the view to exclude this read portion of the buffer So I think the takeaway from this is that I can better expose NBTify's use of strict mode through the NBTData object. Offroaders123/NBTify#31
|
Referenced above in my Bedrock-LevelDB project, I need to provide a more-public way of determining where a |
Offroaders123
added a commit
that referenced
this issue
Feb 3, 2024
Working on figuring out the concepts behind making your own error handlers, in that I can provide the missing metadata I need to be able to get without needing to parse the error message itself. This is a big of a weird setup at the moment, since I don't quite know the correct way to structure these kinds of extensions. https://developer.mozilla.org/en-US/docs/Web/API/DOMException https://stackoverflow.com/questions/5136727/manually-artificially-throwing-a-domexception-with-javascript https://javascript.info/custom-errors https://stackoverflow.com/questions/67558074/how-to-throw-custom-errors https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Errors https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error/cause #31
Offroaders123
added a commit
that referenced
this issue
Feb 3, 2024
Working on figuring out the concepts behind making your own error handlers, in that I can provide the missing metadata I need to be able to get without needing to parse the error message itself. This is a big of a weird setup at the moment, since I don't quite know the correct way to structure these kinds of extensions. https://developer.mozilla.org/en-US/docs/Web/API/DOMException https://stackoverflow.com/questions/5136727/manually-artificially-throwing-a-domexception-with-javascript https://javascript.info/custom-errors https://stackoverflow.com/questions/67558074/how-to-throw-custom-errors https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Errors https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error/cause #31
Offroaders123
added a commit
to Offroaders123/Bedrock-LevelDB
that referenced
this issue
Feb 3, 2024
Tried out bringing over the experimental new error object I made over in NBTify, which allows you to find out where the parsing stage failed, as well as access the data that was read thus far, and to be able to easily know where the reading process left off, allowing you to continue reading afterwards later on. This helps a lot with the `readNBTList()` implementation, because chunks of NBT data are placed adjacent to each other, and I had to do some error message-reading magic logic to get that info. This simplifies this concept, by plainly providing this error to the user. I'm not sure how I want this to look yet, I think it is a great concept to have though! It's like how `DOMException` works in conjunction with when `showOpenFilePicker()` throws, and you can actually get the info about why it threw using that error type. I didn't know that was something you could do, and that was a big find for me! Git Submodules are extra cool, I wish I had used them for STE with NumText and MenuDrop way back, before I used a bundler or npm. Not sure how submodules work in conjunction with GitHub Pages though. My hope would be that they are cloned when the page is built as well, so definitely something I'll have to look into. I would have liked to just install the source from GitHub using npm, but it doesn't work with my TS code, because of the `.npmignore` file I have, it excludes the `src` directory. Otherwise, I would like to use that instead, if it did work for that. Too bad there's not a distinction to ignore the `.npmignore` when cloning a specific project. Maybe that is already some flag I don't know about? Offroaders123/NBTify@388ac02 Offroaders123/NBTify#31 https://git-scm.com/book/en/v2/Git-Tools-Submodules https://stackoverflow.com/questions/10914022/how-do-i-check-out-a-specific-version-of-a-submodule-using-git-submodule
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Since some NBT file data terminates before the end of the full file itself, this has brought up the use for adding the
strictflag to the reading process. I'm curious if this should also be symmetrical, in that the re-written file currently won't be the same as the input file. Should NBTify provide a symmetrical API which will give you a file with the same file length as the write output? This would require storing thestrictflag somewhere in theNBTDataprocess, and possibly the minimum file length. Should it instead be a fixed file length? I'm not sure why the nature of some files are longer than the data they actually hold, which is what would lead me to implement this in a certain way.So if you were to open a file with the flag
NBT.read(data,{ strict: false }), should this be reflected as part of the metadata for the resultingNBTDataobject?While writing this, I just thought of another thing. I think this goes in hand with this feature idea (only if I do implement this symmetrical handling). If the
strict-ness of the file length can be represented in theNBTDataobject, I think it would also be safe to start automatically reading files withoutstrictenabled, only after parsing all of the other file types first, like it does now. So this would run after the newly-addeddeflate-rawhandling, and it would simply pass all format options asundefined, only withstrictset to false. I think that might sound a bit risky though, so I'm not sure about that one. I think a lot could go wrong there, having written that idea out. I think it might pick up on some weird data first, say if it was a non-strictfile, and it was also compressed or something. The blocks might have NBT-looking data somewhere in there, and it might exit early with malformed, unexpected data.So, having discussed that, I think auto-parsing for
strictmay not be viable because you can't ensure that what's coming out is indeed what it should be, unless there's another way to do this that I'm not thinking of. Oooh! This sounds horribly messy, and not worth it, but maybe if it opens the file successfully with a lot of bytes left, it will keep trying, and check against the (multiple?) read results to see which one has read the most of the file. I think that's fairly unlikely to happen, unless the data is magically synonymous with the different format options somehow, but maybe that can help ensure that it's the correct one? I feel like that could technically be incorrect in some circumstances too, though. Especially if the trailing data at the end of the file has random byte data there, it could incorrectly terminate the NBT in a different format, later in the file than where the real, shorter data is stored.So, I will look into having a symmetrical API, see if that's viable (less likely), and secondly, go through with just letting the user know whether the resulting data was loaded non-
strict-ly, through theNBTDataobject result. Thinking about it too, I think this could be necessary for the TypeScript-level of things too, as having two files that have the same structure, but one is loadedstrict-ly, and the other non-strict-ly, they would appear the same on the type level, while they weren't actually loaded with the same format options. Saying it like that, it might not be an issue though, since they are same at the nominal-type-level anyways. I think this distinction should also probably happen at the reading level anyways, since that's where the types would differ when you pass them into the function. But then again, this would break up being able to useNBTDataobjects asFormatOptionsobjects directly, so I think this is probably necessary either way. Not having this currently limits you from being able to use them interchangeably like that.Ok, nice rant. Also, I don't quite like the new GitHub UI layout yet. Why change it?? This time, I'm going to help myself learn to like it, because GitHub is just too darn useful to be annoyed by it. Why they gotta change it though, eeeeh. Hey, at least they're trying something? Wouldn't want the other option, GitHub to go under, and fall behind and not get 'any' updates. Ok, rant over hehe.
The text was updated successfully, but these errors were encountered: