Global Shell Script #25

Offroaders123 · 2023-02-20T21:29:33Z

Running NBTify directly on your machine from the shell could be a really neat use case! It would be similar to the FFmpeg's ffmpeg command, where you work with your files right from the terminal.

The text was updated successfully, but these errors were encountered:

Added a build script to run the TypeScript compiler, rather than running it myself. Looking into some other setups to help automate the library setup a bit more. Looking into adding a `rm -rf ./dist` (`npx rimraf ./dist`) call before `npx tsc`, since sometimes some ghost files are there after removing old files that are no longer in the codebase (they are still in `./dist` though). But, thinking about it, I also don't want to accidentally remove files that are meant to be there, so I probably won't add it. I will have to remember to reset the `./dist` folder myself, if I removed a file from `./src`. https://stackoverflow.com/questions/45082648/npm-package-json-os-specific-script https://github.com/isaacs/rimraf While looking into that, I also found an article about making your npm package into a shell script. I realized that could be a great thing for NBTify to have too! Hadn't even thought about that before. Being able to simply install NBTify globally to your machine, then running it as a command, `nbtify`, and it can manipulate NBT files directly on your file system. Then you wouldn't even need to write a JavaScript script to work with NBTify! I'm gonna go make an issue over on GitHub for that! A really cool thing to look into next. #25 https://blog.deepgram.com/npx-script/ https://2ality.com/2022/07/nodejs-esm-shell-scripts.html#node.js-esm-modules-as-standalone-shell-scripts-on-unix https://github.com/nodejs/modules/issues/152 Not too familiar with all of the different ways to add scripts to your npm package, so I'm looking to find out what most libraries do, since it doesn't always seem to be the same thing (`npm run dev` vs `npm start`). https://docs.npmjs.com/cli/v9/using-npm/scripts https://stackoverflow.com/questions/69400243/whats-the-difference-between-npm-run-dev-and-npm-run-start-in-next-js https://stackoverflow.com/questions/51358235/difference-between-npm-start-and-npm-run-start Relating to the previous thing, it sounds like some TypeScript libraries will run their dev server using `tsc` for the type checking, then instead use `esbuild` for the build process, since it tends to be much quicker, and it doesn't perform any type checking. I think it's because you don't need to worry about type errors during the build process, but instead during the dev process, when you're actively working on the project. I do like that crossover to make things a bit lighter to build for production. evanw/esbuild#1519

Offroaders123 · 2023-07-17T17:50:05Z

Just looked into nbtlib a bit, and their CLI is like what I'd want to have for NBTify. Making CLIs are a bit over my head at the moment still, so I'm going to look into it some more to figure out the best ways and things to implement.

https://github.com/vberlier/nbtlib#command-line-interface

Offroaders123 · 2023-09-11T21:49:47Z

While working on my fork of NBT-NPC-text-fix, I realized again how it could be helpful to have a CLI-based tool to use on top of NBTify's API. It would make things easier to use quickly, because you wouldn't have to set up a JS file for every action you want to make, you can just run the action in one go.

Similar to how UI frameworks work, I just thought: Is a CLI a declarative way (rather than imperative) of accessing a library's API? It actually kind of seems like that's the case! @HoldYourWaffle, what do you think?

A few new references:
nbted
Any good command line NBT editors out there? - Reddit

(I always forget to do this step when trying this out, hahe lol) https://stackoverflow.com/questions/40561591/node-package-json-bin-value-command-not-working "Hot Cakes, or Abbatoir?", "A Plethora of Piñatas", "Assembly Language" (aptly named :) ) #25

Some links I forgot to add over the last few commits: https://stackoverflow.com/questions/40561591/node-package-json-bin-value-command-not-working https://stackoverflow.com/questions/30652243/how-to-pipe-output-of-a-node-cli-program-to-shell https://stackoverflow.com/questions/9191235/whats-the-right-way-to-exit-node-js-script-with-a-log-message https://www.geeksforgeeks.org/node-js-exit-codes/ https://stackoverflow.com/questions/40867345/catch-all-uncaughtexception-for-node-js-app (for this commit) https://askubuntu.com/questions/420981/how-do-i-save-terminal-output-to-a-file (I want to get these working with both NBT and SNBT files) https://stackoverflow.com/questions/70765090/buffered-writes-to-stdout-in-node-js #25

Thinking about a new release, so I'm going to tidy things up a bit so the new features are all nice and ready to use elsewhere :D #25

As with how I've been doing the last few updates, I make a some of the changes from the few previous updates since the last version release, so you can go to each commit about these topics specifically to get more info about what changed, and to see discussion about the process along the way too. I think trying to summarize it again just when I publish it isn't worth it compared to just referencing you towards the original write up about the changes I did the first time around. You'll get more explanations and such, I think it's more worth it :) - ...and optional ListTag values - RootTagLike (& RootTag default return types) - Run-Time List Item Type Check (SNBT) - Format Options Generic Reverting #14 #19 #25 #28 #29 #31 #32

Offroaders123 · 2023-09-15T07:52:39Z

Initial version released to npm! Gonna have more work to do on it, but at least it's going somewhere 😃

Here's some demo commands on how to install and use it on the CLI:

# install it globally, allowing it to run as a CLI command in your terminal
npm install -g nbtify

# log the contents of the NBT file at the given path
nbtify ./level.dat
# show the data contents of the file as SNBT
nbtify ./level.dat --snbt

# pipe the contents into another file (useful for the next steps which follow)
# `--pipe` makes sure to send the raw binary data to stdout rather than the `console.log()` of the `NBTData` object itself
nbtify ./level.dat --pipe > ./level-copy.dat
# change the format of the file just with some flags! this was pretty much inspired by ffmpeg, hehe
# `--bedrock-level` isn't really supported yet, nor is a `null` `--name` value
nbtify ./level.dat --name="rootNameIsNowACustomOne" --endian="little" --compression="gzip" --pipe > ./level-copy.dat

# turn the file into SNBT and pipe it into a new file!
nbtify ./level.dat --snbt > ./level.snbt
# turn the file into SNBT, and pipe it directly into Vim!
# next I want to figure out if I can pipe the contents *out* of Vim, and back into NBTify, that would be sick!
nbtify ./level.dat --snbt | vim -

# parse the data from the SNBT file, then rebuild it into a binary NBT file using the format you specify
# these flags default to that of what the `NBTData` class defaults to, which is how this is all handled internally by NBTify
nbtify ./level.snbt --endian="big" --pipe > ./level.nbt

I need to better account for compression differences with my tests, as right now it depends on which zlib compression implementation you write your NBT files with, will depend on whether the tests will pass or not. This should be excluded I think, I thought it would be perfect to ensure that the compressed versions are the exact same as well, but I don't think that's worthwhile if you can check the raw bytes themselves anyways. Say, to make this file recompiled with the matching zlib layouts, and with the newly updated empty ListTag types ( #40 ), simply opening it with Dovetail and re-saving it didn't work, because it looks like Chrome's zlib backing is different than Node's, even though they both use V8, and they are both on the same macOS ARM machine. So I can rule those things out. I already knew this was the case between Linux and macOS, but I thought this wasn't different between Chrome and Node on the same machine for some reason. I guess I assumed that they would *happen* to be using the same compression implementations, even if they may be two separate installations of that. That would make sense, I didn't expect them to use the same install of it. Just the same or nearly same versions. Another weird thing I encountered. Since I discovered the zlib differences between Node and the browser, I thought I'd use my local global install of NBTify in the CLI to resave the file, using Node as well. That actually encountered an error, and it was only happening with the `--pipe` flag to write it back to the original file. I'm not sure what it was about, that was frustrating though. Going to look into debugging that one. #25

Now the format of the output file is combined into a full object, which is then used by the main file, rather than importing all of the arguments as individual flags. Added a flag to specify the SNBT indentation spacing! It was only set to 2 before, now you can have fully minified SNBT, or use any indentation you'd like. Like the programming NBTify API, it accepts a number or a string. Removed the `--pipe` flag, in favor of simply specifying whether you want to use an NBT or SNBT output to stdout. If neither `--nbt` or `--snbt` are passed as flags, it will simply log out the structure of the file, with nice pretty-printing, colors, and such, as it currently has been doing thus far. The pretty-print logging now fully logs out the NBT file's tree, other than for TypedArray-based tags. For everything else though, you see all of it's content, rather than being abbreviated (the normal behavior for `console.log()` in Node.js). Learned about the difference between `isNan()` and `Number.isNaN()`. The first one coerces the input value, checks if it's `NaN`, while the other simply checks what it is currently (doesn't convert to `number` first). #25 microsoft/TypeScript#31025 https://stackoverflow.com/questions/56143158/how-to-use-util-promisify-and-bind-functions-in-nodejs (I think I ran into this issue previously, but I didn't know that function binding would work nicely here!) https://stackoverflow.com/questions/43362222/nodejs-short-alias-for-process-stdout-write https://askubuntu.com/questions/510890/how-do-i-redirect-command-output-to-vim-in-bash (Looked into this again, was curious in whether it's viable to edit SNBT with Vim again) Oh yeah, that reminds me, I also made it so SNBT is added to stdout with a trailing `\n`, so it nicer looking in the terminal, as well as in the editor, as most editors utilize trailing new lines. I think those are finally catching on with me! They really bugged me for some reason, since the start of my programming lol. Now I'm getting Linux'ed haha. nodejs/node#20366 https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/isNaN https://www.google.com/search?q=comments+in+powershell

Now you can pipe files into NBTify! This works nicely for chaining CLI calls together to make multiple manipulations over a single file, without having to save it to the file system every time. ```sh cat ./test/nbt/house3.mcstructure | tsx ./src/bin/index.ts --snbt --space=0 ``` https://stackoverflow.com/questions/30441025/read-all-text-from-stdin-to-a-string https://stackoverflow.com/questions/39801643/detect-if-node-receives-stdin https://nodejs.org/api/tty.html#readstreamistty https://stackoverflow.com/questions/9231847/node-js-how-to-detect-an-empty-stdin-stream https://stackoverflow.com/questions/20585729/how-to-detect-if-nodes-process-stdout-is-being-piped Not completely sure about `node:fs/promises` not being able to support file descriptors, I'm going to continue to look into that. Also, wasn't sure how to deduce that a file is an SNBT or NBT file, without knowing the extension beforehand. I counteracted this with a simple try-catch block, like the main `read()` function does to detect the file format, as I wasn't sure of a nicer way to figure it out. I was thinking maybe checking if the SNBT is valid UTF-8, but now we know that Bedrock for example doesn't use plain UTF-8 strings in some Entity data, so that wouldn't work with those. #25

#25 Figured out how to simply diff binary files/outputs with the CLI ```sh diff <(xxd ./square.mclevel) <(nbtify ./square.mclevel --nbt | xxd) ``` https://superuser.com/questions/125376/how-do-i-compare-binary-files-in-linux https://unix.stackexchange.com/questions/155806/how-do-i-diff-the-outputs-of-two-commands https://askubuntu.com/questions/418771/uncompress-and-pipe-the-output-to-script https://stackoverflow.com/questions/3219479/how-to-write-buffer-content-to-stdout (Curious about using Vim to edit NBT, by way of piping) https://stackoverflow.com/questions/2600783/how-does-the-vim-write-with-sudo-trick-work https://stackoverflow.com/questions/71771515/how-to-wait-until-a-program-finishes-to-pipe-the-output (Didn't work in combination with Vim) ```sh nbtify ./bigtest.snbt --snbt | vim - nbtify ./bigtest.snbt --snbt | vim - | tsx ./index.ts tsx ./index.ts <<< "$(nbtify ./bigtest.snbt --snbt | vim - ) ```

The CLI flag wasn't being maintained properly, and it took me a good while to figure out that it was because I was omitting the `undefined` state, which is key for this case, in that it's how you can maintain the original setting of the file you opened. #25

Now you can simply convert between NBT, SNBT, and JSON, just with NBTify in the CLI! This is inherently not meant to be a lossless conversion step, but rather one that is just a simple helper for maybe easier data visualizations, as well as just wanting to view a given format as another kind. #25 Wow, cannot believe I've been working on the CLI for a year now! Or at least exploring it. That means I've about had my Mac for a year now coming up soon as well, that's crazy. This last year has been my most productive one yet. I only want to keep trying to see what I can further accomplish and improve on from here. It's a combination of both getting more focused on accomplishing things, as well as letting go when it's not the right time to work on a given thing, learning to give myself a break when I need one.

Offroaders123 added the enhancement New feature or request label Feb 20, 2023

Offroaders123 self-assigned this Feb 20, 2023

Offroaders123 added a commit that referenced this issue Sep 15, 2023

Shell Script Tidying

e7b00ed

Thinking about a new release, so I'm going to tidy things up a bit so the new features are all nice and ready to use elsewhere :D #25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Global Shell Script #25

Global Shell Script #25

Offroaders123 commented Feb 20, 2023

Offroaders123 commented Jul 17, 2023

Offroaders123 commented Sep 11, 2023

Offroaders123 commented Sep 15, 2023

Global Shell Script #25

Global Shell Script #25

Comments

Offroaders123 commented Feb 20, 2023

Offroaders123 commented Jul 17, 2023

Offroaders123 commented Sep 11, 2023

Offroaders123 commented Sep 15, 2023