Correct options for sharing over NFS?

[jackwilsdon]

I've set up the following mergerfs mount (as per https://github.com/trapexit/mergerfs?tab=readme-ov-file#can-mergerfs-mounts-be-exported-over-nfs):

/mnt/disk1/data:/mnt/disk2/data /mnt/data fuse.mergerfs inodecalc=path-hash,noforget 0 2

An exported it over NFS:

/mnt/data 10.99.99.2(rw,all_squash,anonuid=500,anongid=500,fsid=1)

The share is mounted on the client like so:

10.99.99.1:/mnt/data /mnt/data nfs4 defaults 0 0

The issues I'm seeing are:

• fileid change errors:

  NFS: server 10.99.99.1 error: fileid changed
  fsid 0:52: expected fileid 0xc889138139138188, got 0x533f4aa07d917248
  

  I see 3 of these on startup, all triggered by Plex Media Server. The "got" inodes all exist inside Plex's data directory (it's also usually the same inodes each time, although I think that's just because it's the same path with the issue)
• Files and folders created on the server do not appear on the client without restarting the NFS server (or touching the parent folder)

Am I missing some options here? I'm starting to consider moving to SMB, but ideally it'd be better if I could get NFS working reliably.

The second issue seems fixable by setting the func.getattr=newest option on mergerfs, as it appears NFS is watching directory mtime for some form of caching, however this doesn't fix the first issue.

[trapexit]

The truth is NFS can not truly work reliability. Especially given you are changing things out of band. You must not change things out of band. This has nothing to do with mergerfs. NFS is a stateful protocol and all interactions must be done through it. If you use NFS you need to only use the storage via NFS.

[jackwilsdon]

Ah TIL - thanks!

I'm still seeing the fileid change errors even for files I'm not changing OOB. They consistenly happen every time I start Plex - is there a setting I can change to fix this? I thought the inode numbers would be consistent with inodecalc=path-hash.

[trapexit]

The inodes are consistent when using inodecalc=path-hash. But there are other things to consider. Without knowing exactly what is going on or the ability to reproduce I am limited in my ability to make suggestions.

You don't have any other NFS shares? Have you tried using a uuid for the fsid to ensure no collision?

NFS just isn't a good choice for a unionfs or FUSE in general. Too much out of band changes (which can be limited but still difficult to manage) and a fundamentally issues with how NFS and FUSE interact. I'm looking into improving things but it will always be somewhat problematic.

[jackwilsdon]

That's the only share I have - I didn't try a UUID. I've switched to virtiofs for my VM guest now, which seems to work fine on top of mergerfs with cache.files=partial,dropcacheonclose=true,func.getattr=newest. Thanks!