Replies: 3 comments 25 replies
-
|
babel-plugin-transform-typescript: |
Beta Was this translation helpful? Give feedback.
-
|
These are the tests we still need to solve oxc/tasks/transform_conformance/babel.snap.md Lines 835 to 896 in 0faa410 Progress (97/158)
Dependency plugins
In addition to this, access to scopes and access to parent nodes are also relied upon in some complex cases. |
Beta Was this translation helpful? Give feedback.
-
List of APIs needed for implementing transformers(WIP)
Since I want to list high-level concepts, I didn't link to their corresponding implementations in Babel or ESbuild. We might want to base them on either one of them or even write something unique to oxc, The thing is that we just need a way to do these operations. Feel free to edit this comment and help make it more organized and complete. |
Beta Was this translation helpful? Give feedback.
-
You wouldn't. You would need to do the 2nd or 3rd proposal in my post (I like the 2nd one). But basically you would mutate both the parent and child separately, but set state between the 2 to determine when and what to do. |
Beta Was this translation helpful? Give feedback.
-
|
Well, I think that abstracting changes to be done using the ctx can be a good thing, There is not much difference in their complexity when it comes to the implementation but the API with ctx can be more extensive and do a lot more if needed. For mutation, I like the third option a bit more than the second one and I'm not that fond of the first approach. The second approach is similar to the third with extra steps, Maybe I'm not getting it completely? |
Beta Was this translation helpful? Give feedback.
-
|
@rzvxa Yeah 2 and 3 are very similar, but the only difference I'd say is this. For 3, if your parent has a deep child tree, then you're doing a ton of traversal to figure out if a certain child exists within that tree. However, there's no re-traversal after mutating the parent node. For 2, it's the opposite. Since you're already in the child, you just need to loop through the handful of parent nodes to see if it's within a certain parent. However, if it is in a parent, then once we go back up the tree to the parent node, then we need to re-traverse down again if we mutated it. Honestly, we can probably support both patterns since they are so similar, and depending on how complex a rule is, they can choose either approach. |
Beta Was this translation helpful? Give feedback.
-
|
I'll start spiking this if you want to focus on the ast node IDs. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, It would be great if we start working in parallel on these 2 subjects. I suspect that the ast node ID won't take that long since it's mostly grunt work and everything is already figured(at least as far as we know at the moment). So I get done with it and join you on the mutability train. |
Beta Was this translation helpful? Give feedback.
-
|
Here's my notes and thoughts on this matter. Requirements
Nice to haves
Different patterns for mutatingSome ideas I had while trying to think of a new transformer implementation. Immutable AST builds a mutable AST on-demandWith this pattern, we would have 2 ASTs, the 1st is the original immutable AST, and the 2nd is a new mutable AST (starts at Pros
Cons
impl Transformer for X {
fn transform(&mut self, old_node: &AstNode) -> Result<AstNode> {
let mut new_node = old_node.clone();
// Mutate node
Ok(new_node)
}
}Enter/Leave specific transform methodsInstead of having a single Pros
Cons
impl Transformer for X {
fn transform(&mut self, node: &AstNode) -> Result<Option<AstNode>> {
match node.kind() {
SomeNode => {
if self.ctx.is_within_function(...) {
// Flag parent changes within child
self.in_function_body = true;
}
}
};
Ok(None)
}
fn transform_on_leave(&mut self, node: &AstNode) -> Result<Option<AstNode>> {
if self.in_function_body && node.kind() == Function {
let mut node = node.clone();
// Mutate parent and return a replacement node
return Ok(Some(node));
}
Ok(None)
}
}No child -> parent traversal, instead parent -> child visit pre-checks
This pattern goes against our requirement of "traversing up to the parent from the child", but instead offers an alternative. If we have situations where a child needs to detect if it's within a certain parent, or that parent needs to be mutated base on the status of a child, we can solve these by implementing a custom Pros
Cons
struct IsChildInParent {
pub yes: bool,
}
impl Visit for IsChildInParent {
fn enter_node(&mut self, kind: &AstKind) {
match kind {
Child1 | Child2 => {
self.yes = true;
}
_ => {}
};
}
}
impl Transformer for X {
fn transform(&mut self, node: &AstNode) -> Result<Option<AstNode>> {
match node.kind() {
ParentNode => {
let visitor = IsChildInParent { yes: false };
visitor.visit_parent_node(&ast.node);
if visitor.yes {
// Child exists within this parent,
// so mutate the parent from this context
}
}
ChildNode => {
// Mutate child if need be
}
};
Ok(None)
}
}Open questionsBased on all of this, the big problem I keep coming back to, is how do we keep the semantic tables in sync? Especially when we need to add/remove/rename/move scopes/bindings/symbols/etc. This is further made complicated by the fact that |
Beta Was this translation helpful? Give feedback.
-
Just to be sure, You are talking about custom semantic data? Or the idea of having ast_node_id in general? If it is about custom semantics, It would be just a fortunate side effect and not the goal. |
Beta Was this translation helpful? Give feedback.
-
|
The |
Beta Was this translation helpful? Give feedback.
-
I agree, We may want to shift our approach as we implement it. We can start simply by only extracting needed functions from SemanticBuilder into a semantic helper utility since it should get things working and if we need better abstraction we can always build on top of it. |
Beta Was this translation helpful? Give feedback.
-
|
I was thinking you might not need to sync Thus, I was thinking maybe we could have a low-level However, it's just a thought of my head and don't have something specific. |
Beta Was this translation helpful? Give feedback.
-
|
@hyf0 You are right about semantics not being that important in the transformation section we for sure can do without most of the semantic information, But as a library, it would be nice to have the updated semantics after the transformation in case some oxc users want to have some operations based on semantic after transforming their code to the target platform. However, we are currently trying to reinvent the transformers, so we might go either way depending on the current requirements and how long it will take to do it in the "right way". We most likely are going to separate SemanticBuilder from utilities needed for mutation of the semantics this way we can have manual mutations using these utilities when transforming the code, So we don't need to worry about the SemanticBuilder and we just add or remove a few identifiers, symbols and scopes in the semantics we got. However, We can also go without updating the semantics and let the end user generate a new semantic if needed after the transformations. Requires more work if semantics is needed since now we have to walk the whole tree to create the new semantics of the transformed result but might simplify the transformations. All these bring me to my last point, Having semantics helps us validate the transformations since we need to keep track of old variables for not shadowing them, if we do not mutate the semantics will require us to keep track of applied changes in some other manner. |
Beta Was this translation helpful? Give feedback.
-
The current top-to-bottom porting strategy is not working right now. I get depressed whenever I want to implement a plugin due to so many obstacles.
We are going to try another strategy, by doing research and layout all the dependencies of the plugins. After all the dependencies of each plugins are collected, we are going to implement all the utilities from the bottom-up.
Dependencies are anything that we haven't implemented yet.
Beta Was this translation helpful? Give feedback.
All reactions