Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data mapping to struct is slow when the number of database queries is large. #3406

Closed
marunrun opened this issue Mar 20, 2024 · 3 comments · Fixed by #3412 · May be fixed by #3557
Closed

Data mapping to struct is slow when the number of database queries is large. #3406

marunrun opened this issue Mar 20, 2024 · 3 comments · Fixed by #3412 · May be fixed by #3557

Comments

@marunrun
Copy link
Contributor

What do you want to ask

CleanShot 2024-03-20 at 16 14 05@2x

如上图所示,我使用clickhouse , 然后查询了大概8万条数据, 总共耗时6.4s, 其中sql执行花费2.2s , 剩下的应该是数据映射到struct 所花费的时间,可以看到这个时间接近sql执行的两倍, 这个时间在数据量更大的时候,耗时只会更夸张。

请问这种情况该如何优化?

是否只能将数据映射成map, 然后自己再手动转化成具体的map?

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


What do you want to ask

CleanShot 2024-03-20 at 16 14 05@2x

As shown in the picture above, I used clickhouse and then queried about 80,000 pieces of data. It took a total of 6.4s, of which sql execution took 2.2s. The rest should be the time spent mapping the data to the struct. You can see this time Nearly twice as long as SQL execution. This time will only become more exaggerated when the amount of data is larger.

How can I optimize this situation?

Can I only map the data into a map, and then manually convert it into a specific map?

@wln32
Copy link
Member

wln32 commented Apr 12, 2024

@marunrun 可以升级到2.7.0版本,这个版本使用gf gen dao时会把entity目录的所有结构体加上orm tag,加快匹配速度

@github-actions github-actions bot removed the inactive label Apr 12, 2024
@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


@marunrun You can upgrade to version 2.7.0. When using gf gen dao, this version will add orm tag to all structures in the entity directory to speed up matching.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants