feat: improved reformat edits #87

mtoohey31 · 2022-06-12T19:28:44Z

Summary & Motivation

The current method of reformatting documents consists of replacing the whole document with a single edit containing the reformatted version. This is sub-optimal for two main reasons:

It is inefficient, which could potentially be an issue with extremely large documents.
It can result in loosing your cursor position in the document in some editors, as @fufexan and I have experienced with helix.

This pull request updates the reformatting so that it sends multiple small edits, instead of one huge edit.

Further context

Since accomplishing this requires exposing the edits that nixpkgs-fmt uses internally, this depends on nixpkgs-fmt#296.

This pull request also adds one additional dependency: textedit-merge, which I wrote in the proccess of creating this PR. nixpkgs-fmt reformats the document with two sets of edits (spacing edits, then indentation edits), however, from my understanding, the language server protocol requires that the response to a reformatting request be a single set of textedits. Since the ranges in the indentation edits used by nixpkgs-fmt refer to the state of the document after the spacing edits have been applied, a somewhat complex algorithm is required to merge the two sets of edits into a single set of edits that can be applied in one step.

One implementation detail that I would like a second opinion on are the data types:

returned by the new nixpkgs_fmt::reformat_edits function
accepted by textedit_merge::merge
returned by textedit_merge::merge

The current choices for these data types ((Vec<AtomEdit>, Vec<AtomEdit>), &[(Range<usize>, AsRef<str>)], and Vec<(Range<usize>, String)> respectively) were chosen with the following goals in mind:

minimal extra work being done in nixpkgs_fmt::reformat_edits (so that other users of this function can use it efficiently)
no dependencies in textedit-merge (so that other users, if there ever are any, aren't tied to additional dependencies)

The main shortcoming of the current types is that the edits must be transformed twice, once from Vec<AtomEdit> to Vec<(Range<usize>, AsRef<str>)> for merging, then a second time from Vec<(Range<usize>, String)> to Vec<TextEdit> so they can be returned in the response to the reformatting request. If there's a way to do things that still achieves both of those goals while avoiding the double transformation, please let me know!

Finally, it would be dishonest of me if I didn't mention that I'm not 100% confident that textedit-merge is bug-free. I've tested reformatting on nixpkgs-fmt's test cases and it works correctly on all of them now, but while I was working on it, I missed quite a few edge cases that those tests brought to light. If anyone can think of edge cases that might break edit merging, please give them a try! I'd much rather fix bugs with textedit-merge before merging this than find out about them later, as many of the bugs that I encountered during development resulted in panics since they involved indexing strings by invalid ranges.

Closes #81.

Ma27

Nice work, mostly some questions regarding the implementation here and in the corresponding nixpkgs-fmt PR.

Also, do you think you'd be able to write a small integration test in src/tests.rs? :)

Ma27 · 2022-06-27T09:52:04Z

src/utils.rs

@@ -323,6 +325,24 @@ pub fn selection_ranges(root: &SyntaxNode, content: &str, pos: Position) -> Opti
 root.map(|b| *b)
 }

+pub fn atom_edit_to_tuple(atom_edit: AtomEdit) -> (ops::Range<usize>, String) {
+ let r: std::ops::Range<usize> = atom_edit.delete.start().into()..atom_edit.delete.end().into();
+ (r, atom_edit.insert.to_string())


Would be nice to document what this is doing and also to add a test.

Good idea, done in 4cc254a.

Ma27 · 2022-06-27T09:52:17Z

src/utils.rs

@@ -323,6 +325,24 @@ pub fn selection_ranges(root: &SyntaxNode, content: &str, pos: Position) -> Opti
 root.map(|b| *b)
 }

+pub fn atom_edit_to_tuple(atom_edit: AtomEdit) -> (ops::Range<usize>, String) {
+ let r: std::ops::Range<usize> = atom_edit.delete.start().into()..atom_edit.delete.end().into();


Are the two dots really needed here or is that a Rust feature I'm not aware of? :)

The .. operator creates a Range from two numbers. Since AtomEdit's delete property is of type TextRange, which belongs to the text-size crate, I'm converting that range type to the builtin std::ops::Range type to avoid adding dependencies to textedit-merge.

Ma27 · 2022-06-27T09:54:29Z

src/main.rs

- new_text: fmt.text().to_string(),
- }]
+ let (spacing_edits, indent_edits) = nixpkgs_fmt::reformat_edits(&ast.node());
+ let merged_edits = textedit_merge::merge(


I'm not sure why this is needed, but I guess this can be answered with nix-community/nixpkgs-fmt#296 (comment)

Yeah, like I mentioned in nix-community/nixpkgs-fmt#296 (comment), the delete ranges in the spacing edits apply to the original document, and the indent edits apply to the document after the spacing edits have been applied. Since the language server protocol expects a single set of edits in response to a reformat request, I believe merging the two sequential sets of edits into a single set of edits is necessary.

mtoohey31 · 2022-06-28T02:47:15Z

Nice work, mostly some questions regarding the implementation here and in the corresponding nixpkgs-fmt PR.

Also, do you think you'd be able to write a small integration test in src/tests.rs? :)

Thanks! I've responded to the questions here and on nix-community/nixpkgs-fmt#296; let me know if there's anything that's still unclear. I'd be happy to add the two tests you mentioned, though I might not get to it until later in the week.

mtoohey31 · 2022-07-06T02:08:41Z

I've added an integration test for reformatting in 5ffd970. The one thing I noticed while doing that is that editor indentation preferences are not respected. I don't think there's anything we can do about that though as long as we're using nixpkgs-fmt, since it doesn't seem to support configurable indentation.

mtoohey31 · 2022-07-14T23:24:46Z

Ok, I've rebased things against the latest master and switched the remote for nixpkgs-fmt now that nix-community/nixpkgs-fmt#296 has been merged. I assume we'd like to wait for a compatible release of that crate before merging this though?

mtoohey31 · 2022-09-18T15:57:30Z

@Ma27 I'd like to get this merged at some point; I've been using these changes for about two months and things seem good. Would you be able to create a new release of https://github.com/nix-community/nixpkgs-fmt on crates.io, or should I open an issue on that repository?

Ma27 · 2022-09-18T16:02:36Z

I resigned as maintainer here, cc @aaronjanse

mtoohey31 · 2023-06-02T12:26:34Z

I'm closing this because I've switched to using https://github.com/oxalica/nil, so I haven't been using or maintaining these changes for a while. Also, at one point while I was still using these changes, I did run into one situation where applying the formatting broke the code. So I think there must be a bug somewhere in either nixpkgs-fmt or in the textedit-merge package that I wrote for this merge request, but I don't have an example to reproduce the issue unfortunately...

mtoohey31 mentioned this pull request Jun 12, 2022

feat: add reformat_edits function nix-community/nixpkgs-fmt#296

Merged

mtoohey31 mentioned this pull request Jun 21, 2022

format with rnix-lsp leaves selection helix-editor/helix#2655

Closed

Ma27 reviewed Jun 27, 2022

View reviewed changes

Ma27 mentioned this pull request Jun 27, 2022

feat: format via alejandra when feature is set #89

Open

mtoohey31 force-pushed the feat/improved-format-edits branch from 5ffd970 to 6beffc1 Compare July 14, 2022 23:20

mtoohey31 marked this pull request as ready for review July 14, 2022 23:24

mtoohey31 force-pushed the feat/improved-format-edits branch from 6beffc1 to a284cf4 Compare July 22, 2022 22:16

mtoohey31 added 6 commits July 27, 2022 19:22

feat: improved reformat edits

fbbdfc1

refactor: bump nixpkgs-fmt version and clean up edit conversion

9f515db

style: revert unnecessary reformatting

78a0800

docs, test: document and add tests for atom_edit_to_tuple

67d4840

test: add test_reformat_integration

51a8128

chore: switch nixpkgs-fmt to nix-community/main

d79a372

mtoohey31 force-pushed the feat/improved-format-edits branch from a284cf4 to d79a372 Compare July 27, 2022 23:22

mtoohey31 closed this Jun 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: improved reformat edits #87

feat: improved reformat edits #87

mtoohey31 commented Jun 12, 2022 •

edited

Ma27 left a comment

Ma27 Jun 27, 2022

mtoohey31 Jul 6, 2022

Ma27 Jun 27, 2022

mtoohey31 Jun 28, 2022

Ma27 Jun 27, 2022

mtoohey31 Jun 28, 2022

mtoohey31 commented Jun 28, 2022

mtoohey31 commented Jul 6, 2022

mtoohey31 commented Jul 14, 2022 •

edited

mtoohey31 commented Sep 18, 2022

Ma27 commented Sep 18, 2022

mtoohey31 commented Jun 2, 2023

feat: improved reformat edits #87

feat: improved reformat edits #87

Conversation

mtoohey31 commented Jun 12, 2022 • edited

Summary & Motivation

Further context

Ma27 left a comment

Choose a reason for hiding this comment

Ma27 Jun 27, 2022

Choose a reason for hiding this comment

mtoohey31 Jul 6, 2022

Choose a reason for hiding this comment

Ma27 Jun 27, 2022

Choose a reason for hiding this comment

mtoohey31 Jun 28, 2022

Choose a reason for hiding this comment

Ma27 Jun 27, 2022

Choose a reason for hiding this comment

mtoohey31 Jun 28, 2022

Choose a reason for hiding this comment

mtoohey31 commented Jun 28, 2022

mtoohey31 commented Jul 6, 2022

mtoohey31 commented Jul 14, 2022 • edited

mtoohey31 commented Sep 18, 2022

Ma27 commented Sep 18, 2022

mtoohey31 commented Jun 2, 2023

mtoohey31 commented Jun 12, 2022 •

edited

mtoohey31 commented Jul 14, 2022 •

edited