Library Scan Failing Repeatedly after 10.9 ('database is locked') #11589
Comments
|
same error here, after failure from update to 10.9.0 and the hotfix update 10.9.1 library scans did not run completly and i must restart library scan manually again to finish the scan |
|
Does setting parallel scanning to 1 help? You can change it in Dashboard->General, under performance section |
|
Attempting this now |
|
Well, the log error changed at least, but still wasn't able to complete the scan: |
|
Same error here. Docker. Upgraded from 10.8.13-1 to 10.9.1. Edit: |
|
I was having the same issue with Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 5: 'database is locked'. during library scans.
I am running it on a 24 core machine, reducing the thread count down to 2, seems to have resolved it for me. |
|
@mlclns I think you should open another bug report for this other error. It looks like the issue in this bug report and this other one has nothing in common.
|
|
Copy that, will do! |
I had a similar issue on my Synology DS224+. Can confirm that the above suggestion was a workaround for me. |
|
I started getting this issues on JF after the 10.9.1 upgrade. I didn't notice it happening on 10.9, but it may have happened when I didn't notice. Trickplay is currently still generating, not sure if that's potentially relevant. The parallel scanning workaround does appear to have helped though. Relevant logs below: |
|
I am also Running into this since i upgraded to 10.9. Running in Proxmox LXC Due to the Migration i have the jellyscrub migration, library scan and intro detection running resulting in frequent lockups. |
|
It's also happening for me, same issue as mentioned above with no pattern, seemingly random. |
|
Same issue here on 10.9.0/10.9.1, had to restore my backup to rollback to 10.8.13. Happening randomly and unable to fix it except restoring it to 10.8.X. Application deployed on docker with official image. |
|
@hardwareadictos it is enough to set parallel scan and encoding to 1 in General settings to workaround this bug (it could even be that it is already 1 for releases before 10.9). The database is not corrupted by this issue, only updates do not happen. |
Well rear the first "readonly database" error i wasn't unable to make the DB work again, even restarting the whole container or recreating it, only way for "reliving" the service was either deleting the db to start from 0 or just restore the whole 10.8.13 backup. Once the first read only database error the service was tottally down. Jellyfin unaccessible. |
|
Same error here. Docker. Upgraded from 10.8.13-1 to linuxserver/jellyfin:10.9.1 [2024-05-15 08:54:57.070 +02:00] [ERR] [303] MediaBrowser.Providers.Manager.ProviderManager: Invoking "RefreshCompleted" event handlers failed Edit:
|
|
Well, tried library scan and encoding limit both to one and i can confirm that it's stable for a couple of hours but then lockups start to appear: Wasn't that present on the betas? |
|
I encounter exactly the same thing despite several attempts to modify parameters.
|
|
@derrezzz you should post this other issue as another bug report. This seems not related to this one. I am getting the database lock error with library scan tasks en encoding tasks forced to "1", with the same error as OP: Setting these parallel tasks to "1" helped but did not prevent all parallel access to the DB. Seems it only avoided getting them all the time. |
|
Might be interesting to test if this is reproducible on Windows. Might be a dotnet for Linux bug. |
|
Try removing your music library. Skip this paragraph unless you are having off topic issues. I'm just leaving it here in case someone else is running through this and just plain lost in the hopes it may help. First off I am using docker on Synology and running jellyfin in a container. My upgrade failed miserably, and I did not mount /config or /cache and I did not figure out backing up the instance prior to trying to upgrade. Definitely my fault there. So blow it away and set it up, fine but media won't scan without failing quickly. I did not mount, /cache and /config. Since I did not have it before, but looks like it's key to troubleshooting. Darn, well blow it away and set jellyfin up again, this time mounting /config and /cache, but my still won't finish without failing quickly, but now I can troubleshoot and get logs easier. Found a notify watcher error, and fixed that by increasing the default to 60,000. Very weird because 32,000 was not enough, yet my old instance of jellyfin was fine without changing the low default of under 9,000. Moving on, I still can't get a good scan, just failing at like 81% over and over and quickly. Wish I found this to try before the last steps I took. My primary error was [ERR] [24] Emby.Server.Implementations.ScheduledTasks.TaskManager: Error executing Scheduled Task And of course my values of 4 did not exactly match on open web searches. Reviewing the logs I pinpointed the problem down to during the music library scan. I tried removing the music library, but nope, jellyfin now fails because it can't find the music library. Weird since I removed it from the jellyfin GUI logged in as admin. I spent today cleaning my music library as it had a bunch of junk from over the years (.nfo, .ini, .jog, .playlist files, some apple drum .m3p or w/e files, video files, etc) and was definitely not up to best naming standards. Keep in mind, with all that terrible junk the last jellyfin release worked fine. Anyway, I added the music library back and told it to rescan discarding the old data it just refused to give up. Got to 92% and failed still. I thought it failed at like 86%, so it was weird to see some progress with the same error, but I gave up after many scans hitting 92%. Frustrated because I can't pinpoint down the music files that are an issue, I took the last step and just rebuilt from scratch one more time but forget the music. Success. I will not be adding the music folder back for any testing. I can't find a way to actually remove the music library after it's added in the jellyfin GUI and later removed. I saw others giving it 12 hours and having success, but it did not work for me. When I would remove the music library I would not find the corresponding music folder anymore, but the scan demanded it to continue. I'm not sure what happened with this release, but it made me much better at troubleshooting jellyfin that's for sure. I really love jellyfin, but it's definitely frustrating after how spoiled the last release (my first one to set up) made me feel lol. |
Try 10.9.2 |
Wow, I did not realize that happened while I was still working on this lol, you are all quick :). Much appreciated for the quick response and all you do! Edit to change this part I should also say my install was the older 10.8.13. I did not realize there was a 10.9.0, we just got the windows client update notification a couple days ago, so I thought 10.9.1 was the next release from 10.8.13. |
|
Do you have a fix for the scan end with error ? |
it is just changing the journal mode. As long as memory or off mode are not used it should work fine |
|
Please. That PR is not even merged. Stop trying that. It will not have any effect because the code is not in current release. |
|
maybe not, but for me it has the effect. When I add it as env var I don't have the database locks and library scanning completes in under 1 min. When I put back wal mode the database locks are back and scanning fails. If I leave it as wal and put Parallel library scan tasks limit = 1 then scanning works but it takes several hours to finish. Using linuxserver docker btw |
This only means it probably does not scan your library at all. I am the person write that code, I know it would not have effect in the current release. |
|
Ok, I agree with you. But, it works for me for some reason and I can't explain why if there is nothing in the code that would cause this change to behave like this. |
|
@sasablazek Always follow the logs while you scan the library. My scan also completes without error on the web ui but the logs still show occasional database locks: |
|
Not solved for (I don't have music library) 1.9.4 always terminate scan with [failed] status. |
|
Logs... #11589 (comment) |
|
I wait tomorrow to switch on debug mode log (because jellyfin is used a lot on Sunday evening) |
|
I think they are some TV Show that make the [Failed] : I try the "Search for missing metadata" on entire TV Show library before to run another Scan. Same problem:( always terminate [failed] even with "Search for missing metadata" launched and finished before. |
|
Synology Container showed an image Update available. Clicked on it and now Jellyfin won't show log in. Won't show the server. Rebooted the Synology NAS. |
Is this your case? |
|
No. Not my case |
|
Why are you so sure? Do you have any logs? |
|
same issue here, nothing seems to resolve it |
|
At least to me, no more "database table is locked" errors since I updated yo 10.9.4 🙂 |
|
"database table is locked" still occurs for me on 10.9.6, although the lockup doesn't seem to last as long as it used to. |
|
@viown is it 5 or 6? A full log will be even more helpful |
|
This should be fixed for most users in 10.9.6. @MoonOverMiami Are you still having this issue? |
|
Eh, the upgrade path depends entirely on how you installed it in the first place. We expect some level of skill and knowledge about self-hosting from our users. We are not going to create videos and especially not for extremely basic stuff such as "how to upgrade". The fix is in 10.9.6. If you want the fix, upgrade. There may be some residual issues but the nature of SQLite means there will be the occasional db lock errors as it does not like concurrency. |
|
10.9.6 fix for me. Thanks for amazing work now all is perfect |
|
Hey guys! I have no more lockups since 10.9.4 (10.9.6 is for me the best 10.9.X release so far as it fixed most off the bugs outside those locks) but look what i found on another service which uses sqlite for media management: Someone upper here suggested to enable WAL also on Jellyfin which maybe could point to a solution. Maybe that's not the correct forum, but would be nice also that Jellyfin Devs start to consider let users to use 3rd party more robust DBs like Mysql or Pstgres like Nextcloud does. Have a nice day. |
We already enabled that. But that can only help us this much. SQLite concurrency is still limited and all the locking you are seeing is due to the concurrency. |
So it is enabled by default now? That could explain in my personal case why locks disappeared in my server.... |
It is enabled since the very first 10.9 version. It can only help that much and cannot avoid all lockings when concurrency is high and query is slow. |
10.9.6. I've attached the full logs below. |
This is not the library database being locked but the device one. The pending #11901 might fix your issue. As a current workaround, can you try set env variable |
|
I can confirm 10.9.6 fixed it for me. |
and others
Please describe your bug
Server/Web/Build Version: 10.9.1
Deployment: Docker (linuxserver/jellyfin)
Prelude: There are no other Jellyfin instances hooking into the database as far as I can tell. I did not set up any alternate instances with the same data directory mounts. I have
docker compose down'ed the container and redocker compose up'ed it numerous times thinking maybe there was some duplicate service running after upgrade, but to no avail.Upon updating to 10.9, I was unable to refresh and scan my library at all, getting the 'database table is locked' error for SQLite, but I diagnosed this as an incompatibility with a plugin, which I removed and fixed my issue. However, now after updating to 10.9.1, I'm getting stuck on a similar error, but cannot find any culprit (Interestingly my 10.9.1 upgrade also coincided with my Jellyfin restarting itself every 30 minutes or so, but I can't figure out that one right now). I cannot find anyone mentioning this issue regarding their 10.9 upgrade, other than a few mentions in the nightly stages, which seem to have been issues that have since been patched. Relevant logs attached.
Any help solving this would be greatly appreciated!
Reproduction Steps
Upgrade a 10.9.0 linuxserver/jellyfin container to 10.9.1 by
docker compose pull && docker compose down && docker compose up -dto recreate the containers. Upon doing so and waiting for the service to start up, completing a library scan will be impossible due to this error captured in the logs every time. My docker-compose.yml has the environment variableJELLYFIN_playlists:allowDuplicates=false, and the following mounts:Jellyfin Version
10.9.0
if other:
10.9.1
Environment
Jellyfin logs
The text was updated successfully, but these errors were encountered: