How to add custom syntax/operators/keywords to JavaScript with Babel?

[abalter]

I asked this question on SO, and although it received positive response in terms of upvotes and stars, there are no comments or answers. So perhaps it's outside the scope of expertise of users who track the babeljs tag.

I would like to add custom operators, keywords, and syntax to Javascript.

Someone asked for help with this many years ago and got some great instructions:
https://stackoverflow.com/questions/20762338/how-would-i-extend-the-javascript-language-to-support-a-new-operator/20764137#20764137

Implemented here:
https://github.com/IonicaBizau/js-custom-operators

This method does not work with the most current version of esprima.js.

Is this possible with babel.js?

[hzoo]

Great question @abalter! Short answer with Babel at the moment is you need to fork the parser itself + use the parserOverride option in a plugin to parse it differently. (we made it rather difficult to extend on purpose, although I would like people to experiment at least to learn about parsers/ASTs/languages). And you can definitely do this with other parsers (esprima/acorn) as well.

(I would like to experiment with making it easier to at least try some things, like make a codesandbox to test (I made one recently that helps you write a babel plugin online), without necessarily making an npm package for it)

There was a good, older article about this, where that day we removed the ability to do so unfortunately (by bundling the parser, we disallowed modifying any internals). @tanhauhau has a good article on a current way to do this: https://lihautan.com/creating-custom-javascript-syntax-with-babel.

If the new syntax is a TC39 proposal, then someone (our the proposal champion) could implement it on their own and send a PR to Babel. Example (record and tuple syntax): #10865. You can also look through old PRs that were modifying the parser and were a new feature

import customParser from './custom-parser';

export default function ourBabelPlugin() {
  return {
    parserOverride(code, opts) {
      return customParser.parse(code, opts);
    },
  };
}

[abalter]

Thanks @hzoo. I've gotten clearer that my goal is to create a language that is basically javascript with some added syntax closer to data-centric languages (Python, R, Matlab, ...) such as

• Keyword arguments randomNormal(mean=1, sd=2.3)
• Slicing syntax: a = 3:2:12 --> 3,5,7,9,11
• Array slicing a[1:3] --> 3,5,7
• Indexing starting at 1 rather than 0 (not essential, but SOOO much more intuitive IMO)
• Sugary element-wise array operations: function var(x){ return sum(x*x)/(x.length-1); }

And a hopefully only a small number of other things. It would compile to vanilla Javascript.

Then you could easily run it in a browser. I guess the module to load could be a fork of babel, so not entirely out of the question that it is the way to go.

As it stands, I know virtually nothing about ASTs, parsers, and compilers. So I would have a lot to learn...

[abalter]

Thanks @hzoo.

I'm getting clearer that what I'm really after is a language that is very close to Javascript, and easily compiles to Javascript, but has some syntactic elements more in line with data-centric languages such as Python, R, Matlab, ...

Examples:

• Keyword arguments: randomNormal(mean=1, sd=2.3)
• Slicing syntax: a = 3:2:12 --> 3, 5, 7, 9, 11
• Array Slicing: a[2:3] --> 5, 7, a[-1] --> 11
• Sugary element-wise operations: function var(x){ return sum(x*x)/(x.length-1); }
• Indexing from 1 rather than 0 (Much more intuitive for data, IMO).

Hopefully only a small number of other things.

Maybe the module you load could be a forked version of babel. Right now I know almost nothing about ASTs, parsers, and compilers. So much to learn...

[hzoo]

Maybe I'll update the original answer with a few more links that could be helpful in a bit!

FYI, there is actually a proposal for slice notation: babel/proposals#41, and @JLHwung made a PR for both the parser/transform #10357 so that could potentially come to JS at some point (or not).