Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: improve gf gen service using AST #3488

Open
wants to merge 37 commits into
base: master
Choose a base branch
from

Conversation

oldme-git
Copy link
Member

@oldme-git oldme-git commented Apr 14, 2024

Review Dear:
本次 PR 对 gf gen service 进行了部分重构,即使用 AST 替换 Regex 来解析 logic 文件。因为gf gen是一个比较重要功能,反馈的问题也比较多,所以此次提交比较重要

性能方面

相较于多次正则不同,AST一次循环解析出所有需要的语法,例如 import, func, param, comment,所以性能得到了略微的提升。
以下是对1000个文件,若干方法的测试结果,单位秒,测试环境:go1.22.1 windows11/AMD Ryzen 5500U/16G memory
AST 三次:

4.0956484
3.862473
3.8385975

Regex 三次:

4.1110877
4.0835742
4.2559844

测试文件和方法click me

改动简述

fixed #3067
fixed #3492
fixed #2310
fixed #3104
fixed #3434
fixed #3474

@oldme-git oldme-git added the wip label Apr 14, 2024
@oldme-git oldme-git changed the title enhance: improve gf gen ctrl using AST enhance: improve gf gen service using AST Apr 15, 2024
@oldme-git oldme-git removed the wip label Apr 17, 2024
@gqcn
Copy link
Member

gqcn commented Apr 17, 2024

这块逻辑比较复杂,需要花点时间review

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


This logic is relatively complex and will take some time to review.

@gqcn
Copy link
Member

gqcn commented Apr 29, 2024

@oldme-git 我在项目中执行了一下,遇到生成方法重复的问题。我把文件作为附件上传,你可以尝试一下,建议加上一个类似的单例在提交代码中。
image

打包的logic最小文件:logic.zip

@oldme-git
Copy link
Member Author

oldme-git commented May 2, 2024

@oldme-git 我在项目中执行了一下,遇到生成方法重复的问题。我把文件作为附件上传,你可以尝试一下,建议加上一个类似的单例在提交代码中。 image

打包的logic最小文件:logic.zip

@gqcn

单测已经加上了。生成文件的那一块也整理了一下,去掉了因为正则不准确而又多添加的矫正代码。另外,在生成文件的地方进行了两个优化:

  • 使用协程
  • 使用 bytes.Buffer 替代字符串拼接,以提高性能。

最终性能得到了约 30% 的提升,以下是对 3000 个文件,若干方法测试的结果,单位秒,测试环境:go1.22.1 windows11/i7-13700k/64G memory

正则
38.7244941
33.9636121
45.8428419

AST
26.774159
21.3870076
31.7287746

在测试中,大部分的时间都用来 gofmt 生成好的文件。如果想获得更好的性能,可以对 gofmt 这一步进行优化,不过当前,也足够使用了

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


@oldme-git I executed it in the project and encountered the problem of duplicate generation methods. I uploaded the file as an attachment. You can try it. It is recommended to add a similar singleton in the submission code. image

Packaged logic minimum file: logic.zip

Single test has been added. The generated file has also been tidied up, and the correction code added due to inaccurate regular expressions has been removed. In addition, two optimizations have been made where the file is generated:

  • Use coroutines
  • Use bytes.Buffer instead of string concatenation to improve performance.

The final performance was improved by about 30%. The following are the results of testing 3000 files with several methods, unit seconds, test environment: go1.22.1 windows11/i7-13700k/64G memory

regular
38.7244941
33.9636121
45.8428419

AST
26.774159
21.3870076
31.7287746

In testing, most of the time is spent generating good files with gofmt. If you want to get better performance, you can optimize the gofmt step, but currently, it is enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants