/$/compact/ with deleteOld=true does not remove the old folder (on Windows) #2127
Comments
|
I am quite sure there was a similar comment either in GitHub issue/discussions, or in the mailing list. And I think at that time the answer was that it was by design… but I couldn't find the issue or email. |
|
Hm. I've read #1252, but that was before 4.8.0.
|
|
Adapted |
|
Same root cause as #2092 (comment) - longstanding JDK on Windows bug. |
|
Hi @rvesse Thanks for putting this into context! That sounds a bit discouraging, though, meaning that the Not familiar with the code here, but I wonder if it wouldn't be possible to properly cleanup the mapped files then for the old "database"? I notice that the Windows file handles disappear after a while, and then it is possible to delete the folder. Is that connected to the GC settings as well? |
Could be - there have been proposed workarounds, some of which forced a GC. But they seemed to be GC-specific and also had reports they didn't always work. The best Jena can do is to detech it is running on Windows and log that deleteOld is being ignores(I don't know if that works if the host is linux, that is, a container, running on a Windows host with mounted host files) |
|
@afs possible solution idea: On Windows instead of attempting the delete write a marker file ( |
|
In the stacktrace, it appears there is another process active in some way. Deleting disk space is attempted for Windows; The situation around compaction is different for Jena 5.0.0 if a previous JVM had a problem in compaction. In 4.x partial, broken compaction could get left around causing a subsequent JVM to encounter problems. That might be what is occurring in the stacktrace (it isn't conclusive though). 5.0.0 has compaction robustness - issue #2254. PR #2304 added robustness by compacting to a temporary directory, moving it into place using filesystem atomic rename within the same directory. This is the commit step. Then it deletes the old directory before the compacted database is available to the application. (There is a period of vulnerability to external effects, not java factors, between rename and finishing the filesystem delete.) #2315 reported that the storage directory can't be renamed into place on MS Windows. The atomic rename above does not work if the directory has memory mapped files. #2321 changes MS Windows to write the new database storage into the correct place. It has different compaction robustness by a different clean-up. A control file is written which lists the directory to be cleared up the next time this database is opened/connected to. Compaction writes the name of the new database storage directory (the If compaction runs smoothly, the old directory, If the compaction does go wrong, there is highest The Windows compaction code deletes the control file when the compaction has finished and before the switchover to the new compacted storage and before it is available to the application. Cleaning up the control file is the commit step. There is an equivalent period of vulnerability to external effects. Code runs once per JVM per database before using a database for the first time in the JVM. That code understands both Windows-style and Linux-style compactions and so the database folder can still be moved between systems. Please try Jena 5.0.0. After running many Windows github actions and not seeing an occurrence of failure of one of the tests in Jena 4.x that would occasionally fail, I hope this is robust. |
Version
4.8.0
What happened?
We notice that on Windows (at least on Windows Server 2019), the compact call with the flag "deleteOld=true" active does not remove the old folder. There is not error message or exception in the log. The result is a new folder with a compacted database, but the old folder remains untouched.
Relevant output and stacktrace
Are you interested in making a pull request?
None
The text was updated successfully, but these errors were encountered: