Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Poor performance #1568

Open
jas88 opened this issue Jun 15, 2023 · 2 comments
Open

Poor performance #1568

jas88 opened this issue Jun 15, 2023 · 2 comments

Comments

@jas88
Copy link
Member

jas88 commented Jun 15, 2023

Very slow reading data from SQL compared to SSMS - possibly DataTable/pipelining bottleneck?

Investigate batching DataTable writes, clone instead of repeating chunk metadata, refactor as IEnumerable instead of repeatedly calling DataTable GetChunk.
@tznind
Copy link
Contributor

tznind commented Jun 15, 2023

Also check which database you are connecting to. If the query has any collation in it and RDMP is connecting to master or another db with a different default collation this can be a problem.

@jas88
Copy link
Member Author

jas88 commented Jul 21, 2023

Same SQL executed via SSMS runs in a tenth of the time, so not an SQL level issue :(
Synchronous logging looks like at least part of the culprit - single-row inserts on ProgressLog are sometimes timing out since it's over 100m rows. Will investigate further and see what can be done there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jas88 @tznind