Asking help on message "cannot determinate replica position error"

[ymaillard]

I try to backup a database on a S3-compatible storage. When I start a docker container with litestream, I obtain this error:

hub-production-hub-production-litestream-1  | /data/audit-db/default.db(s3): snapshot written 9f533b0d1a73d1a4/00000018
hub-production-hub-production-litestream-1  | /data/audit-db/default.db(s3): monitor error: cannot determine replica position: no snapshot available: generation=9f533b0d1a73d1a4

On object storage, I find some backup files like

../audit-db/generations/705c5b4d1b438901/snapshots/00000018.snapshot.lz4

What "cannot determinate replica position error" message means and how can I solve it ?

Thanks for your help

[hifi]

If you run Litestream as a sidecar (as this seems to be compose) you must ensure the application SQLite doesn't roll in the WAL file which happens automatically when the last connection to the database is closed. To prevent that from happening in sidecar mode your application needs to open the database with SQLITE_FCNTL_PERSIST_WAL option. That might be hard to do as it's a very low level setting.

The easier approach is to use litestream replicate -exec to run Litestream as the parent process to your application. That ensures that Litestream can open the database first and close it the last preventing that from happening.

[ymaillard]

Thank for your answer, I will try.

In the meantime, I tried to backup on filesystem ( no type in config, only path ). And I didn't get any error and the restore process was correct. Shouldn't I have had the same in this case ?

[hifi]

It's replication that's affected by it, restore downloads a fresh copy every time anyway.

[ymaillard]

Sorry, I'm having trouble making myself understood.

I made a test with this following config (no S3 type) :

dbs:
  - path: /data/audit-db/default.db
    replicas:
      - path: /home/test/audit-db

I didn't have the error "monitor error: cannot determine replica position: no snapshot available ..." when I use replicate command. Shouldn't I have had the same in this case ?

Moreover, I notice that in this "filesystem mode", when I make a ls on /home/test/audit-db/generations/XXXXXX I have a "snapshots" directory AND a "wal" directory. In S3 mode, I have only on subdir "snapshots" in "generations/XXXXX/"