Significant performance issues when exporting messages

[musa11971]

Hi, thanks for creating this tool. I've been trying to use it to export my chats, but keep running into significant performance issues. The amount of time it would take me to export makes it simply not feasible to finish, see the following:

⠉ [21m] [>-------------------] 9999/429340 (0.1784/s, ETA: 4w)

or a bit later...

⠁ [23m] [>-------------------] 10395/429340 (0.0198/s, ETA: 35w)                     

Do note the huge chat size of 33 GB. I did not expect the export to be fast at all, but this seems extreme.
(Estimated export size: 33.68 GB)

Running on MacBook Pro M1 Max (32 GB) 2021, full disk access enabled.

Any ideas on what I could do to speed this up? It's a bummer I can't do it in batches. E.g. run the export for a bit today, and then continue tomorrow etc. I found that if I close my Mac and come back to it later, the export process will freeze completely and not make any progress.

[ReagentX]

This doesn't make a lot of sense to me, it takes about 30 seconds on my machine to export more than 500k messages from my personal db. On my M1 Max MacBook Pro, imessage-exporter reaches 18k messages per second, nowhere near your 0.0198/s.

Did you recently delete a large conversation? I noticed that while iCloud syncs that deletion, the table gets very slow to query.

Can you provide details on your machine, (cpu, os, etc) as well as the imessage-exporter command you are using?