[bug]: OOM killer on stalwart-email when importing accounts

[EliRibble]

What happened?

I'm working on migrating from v0.5.3 to 0.7.3. I've exported my accounts individually. I'm working on importing one export of 10G of mailbox on a Virtual Private Server with 8G of RAM. I've freed up 4G of RAM to do the import. I'm still hitting OOM killer on the server process. I'm not sure if this means we need the full size of an import in available RAM in order to complete it, or if there's some kind of bug I've hit.

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:

1. Export my mailbox from Stalwart v0.5.3
2. Create a new installation of v0.7.3
3. Import the exported file
4. Get OOM killer stopping the Stalwart v0.7.3 mail server, while CLI process dumps many, many instances of

Failed to import emailId ee1aaabiyj: Transport error: error sending request for url (http://anasazi.theribbles.org:8080/jmap/upload/a/): error trying to connect: tcp connect error: Connection refused (os error 111)

Version

v0.7.x

What database are you using?

RocksDB

What blob storage are you using?

RocksDB

Where is your directory located?

Internal

What operating system are you using?

Linux

Relevant log output

Failed to import emailId ee1aaabiyj: Transport error: error sending request for url (http://anasazi.theribbles.org:8080/jmap/upload/a/): error trying to connect: tcp connect error: Connection refused (os error 111)

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct

[EliRibble]

I freed up a bit more RAM and it was able to complete. Seems like the sweet spot was pretty close to 4G.

Successfully processed 24 mailboxes (0 imported, 24 already exist).
Successfully processed 50203 emails (42171 imported, 8032 already exist).
Failed to import identity b: Set failed: invalidProperties: E-mail address not configured for this account. (properties: email)
Failed to import identity c: Set failed: invalidProperties: E-mail address not configured for this account. (properties: email)
Failed to import identity d: Set failed: invalidProperties: E-mail address not configured for this account. (properties: email)
Failed to import identity e: Set failed: invalidProperties: E-mail address not configured for this account. (properties: email)
Successfully processed 1 identities (1 imported, 0 already exist).

[mdecimus]

Hi,

Will investigate this. I have a few questions:

• What is the size of the largest email in the mailbox you're trying to import? And if possible, what is the average size of a message in the mailbox.
• How many concurrent threads are you using to import?

Stalwart only keeps messages in memory temporarily to parse them, so if there are multiple concurrent sessions uploading messages this might use all your available memory. Regular accounts have a concurrency limit but the import is done using a super user account where no limits are set.

[EliRibble]

I'm not intimately familiar with the export format. Looks to me like <exportroot>/<account>/blobs contains most of the attachments and things that would increase email size.

> ls -Slh | head -n10
total 10G
-rw-r--r-- 1 root root   47M May  8 10:07 cbe3hljvetqljlgflcbrhd7gu1qw22judeiuos2rur0jaak13qvmmaia9h3qe
-rw-r--r-- 1 root root   43M May  8 10:07 cbhhh2qbdg21oex2j3w7hx7izxg70tnset7zrmngetqlirzkqya2caiayg2ac
-rw-r--r-- 1 root root   43M May  8 10:07 cc17eoooldf0ocywiogo9mwaagcgeapypkelzyzh27urpj9am21oqaiay10ac
-rw-r--r-- 1 root root   42M May  8 10:15 cbobqzbkd2379o2acktshou3nyo97m3ir3xnwtince1mxym2qgtuaaia0tvqc
-rw-r--r-- 1 root root   37M May  8 10:07 cbr1jnssszg97qaycmwjzgfaakf3nlc9y1wq2pp7yv37sndanf7fkaia2k9qc
-rw-r--r-- 1 root root   34M May  8 10:08 cc9pj37idwyle0s7qh1xseybeazwphqifvioij3xhatuxodrsr2w7aiascyac
-rw-r--r-- 1 root root   33M May  8 10:09 cd7aerq9gzxsx72z0a0cjfu2ey2icza7y9wnuu0k2ypqvh1acgmloaia1cgqc
-rw-r--r-- 1 root root   32M May  8 10:13 ccectosdcy2h2orrq7reggyeu3umunb1v1iocelhybsvu1jhmdhjwaia3w1ae
-rw-r--r-- 1 root root   32M May  8 10:09 ccnkmncpb03yoa9vwbjko90h9c0ssunz0x3rqesuz0nbb9ftdh73saiaz7gqc

emails.json appears to contain a relatively small amount of message metadata, most likely not even the email headers. To answer your question on the average size I'm assuming you want the average of the files in blob:

> ls -l | gawk '{sum += $5; n++;} END {print sum/n;}'

211258

I don't know how many concurrent threads I was using on import. My command looked like this:

sudo bin/stalwart-cli -u http://127.0.0.1:8080 -c password import account eli /mnt/slab1/mail-exports/eli

I only ran one of those processes at a time, one for each account.