dwarfs-0.9.10

Bugfixes

• When cloning LZMA compressor objects, the LZMA filter options of the cloned instance would still point to an options object in the original instance. This could lead to LZMA errors when initializing a new compressor. Fixes github #224.
• Fetch range-v3 if no suitable version is found. Fixes github #221.
• Filter rules did not work correctly when input is root dir
• duf reports odd sizes due to using bsize instead of frsize

dwarfs-0.9.9

Bugfixes

• A bug introduced by an optimization to skip hashing of large files if they already differ in the first 4 KiB could, under rare circumstances, lead to an unexpected "inode has no file" exception after the scanning phase. This bug did not cause any file system inconsistency issues; mkdwarfs either crashes with the exception, or its output will be correct. Fixes github #217 (see also for more details).

Features

• A sequential access detector was added to the block cache, which can trigger a prefetch of blocks assumed to be read in the future. This improves sequential read throughput roughly by a factor of two. Random access should typically be unaffected. Can be configured / disabled using -o seq_detector.

• Added tracing support in FUSE driver and dwarfsextract, which allows simple performance analysis using chrome://tracing. Traces can be enabled using -o perfmon_trace and --perfmon-trace.

• Added performance monitoring and tracing support for the block cache.

Performance

• Significantly improved the speed of dwarfsck --checksum.

dwarfs-0.9.8

Bugfixes

• Build custom version of libcrypto to link with the release binaries in order for them to run properly on FIPS-enabled setups. Fixes github #210.

• When mounting a DwarFS image on macOS and viewing the volume in Finder, only the directories were shown, but no files. The root cause was that a non-existent extended attribute is reported via a different error code in macOS (ENOATTR) compared to Linux (ENODATA) and the wrong error code was returned for certain Finder-related attributes. Fixes github #211.

• macOS builds using jemalloc were crashing when calling mallctl("version", ...). The root cause of the crash is still unclear, but as a workaround, the jemalloc version is compiled in from a preprocessor constant rather than using mallctl.

dwarfs-0.9.7

Bugfixes

• Handle root uid correctly in access() implementation. Fixes github #204.

Features

• Show and track library dependencies. Dependencies will be displayed in the command line help; they will also be tracked in the history metadata of a DwarFS image. See also github #207.

Documentation

• Describe nilsimsa ordering algorithm more accurately.

Performance

• Reorder branches to improve ricepp speed with real world data.
• Some tweaks to improve segmenter speed.

dwarfs-0.9.6

UPDATE: All prebuilt binaries available for download here are unaffected by the xz/liblzma backdoor issue. The binaries are statically linked against liblzma-5.4.1, which does not include the backdoor, so even on systems with an affected version of liblzma, the statically linked binaries are safe. #207 has more details. Thanks to @bionade24 for bringing this to my attention.

Bugfixes

• Add workaround for new glog release breaking the folly build. Fixes github #201.

Performance

• Improve ricepp decoding speed by about 25% on x86 and arm, and up to 100% on Windows. Also improve encoding speed on Windows by 25%.

dwarfs-0.9.5

Bugfixes

• Windows path handling was wrong and didn't work properly for e.g. network shares. This is hopefully fixed for all tools now.

dwarfs-0.9.4

Bugfixes

• (fix) Prevent installation of ricepp headers/libs. Fixes github #195.

• (fix) Don't fetch googletest in ricepp build if the targets are already available. Fixes github #194.

Features

• Added blocksize option to the FUSE driver, which allows the st_blksize value to be configured for the mounted file system. This can be used to optimize throughput.

• Added experimental readahead option to the FUSE driver. This can potentially increase throughput when performing sequential reads.

dwarfs-0.9.3

Bugfixes

• v0.8.0 removed the implementation of the null decompressor under the assumption that it was no longer used; it was, however, still used when recompressing an image with null-compressed blocks. The change to remove the implementation was reverted and a new test case was added. Fixes github #193.

Performance

• Some more ricepp compression speed improvements. Also, the universal binaries for x86_64 now automatically choose a ricepp version based on CPU capabilities.

• For Windows, there's an experimental -ricepp package/binary. This contains a "hybrid" build where the ricepp library was built using clang and everything else using cl. This binary offers significantly faster ricepp compression. Decompression speeds are similar to the regular package/binary. If you don't care about compressing large amounts of FITS files on Windows, just stick to the regular package/binary.

dwarfs-0.9.2

Bugfixes

• (fix) v0.9.0 introduced an optimization where large files of equal size were only fully hashed for deduplication if the first 4K of their contents also produced the same hash. This introduced a bug causing an exception to be thrown when processing large hard-linked files. The root cause was that the data structure intended to be used for exactly this case was just never populated, and the fix was adding a single line to fill the data structure. The test cases didn't cover large hard-linked files, so this slipped through into the release. A new test case has been added as well.

• (fix) On Windows, when using Power Shell, the error message dialog for a missing WinFsp DLL was not shown when running dwarfs.exe. The workaround is to use the same delayed loading mechanism that's already used for the universal binary and show the error in the terminal. See also the discussion on github #192.

Features

• Added a --list option to dwarfsck. This lists all files in the files system image. When used with --verbose, the list also shows permissions, size, uid/git and symbolic link information. Fixes github #192.

• Added a --checksum option to dwarfsck. This produces output similar to the *sum programs from coreutils and can be used to check the contents of a DwarFS image against local files.

dwarfs-0.9.1

Bugfixes

• Invalid UTF-8 characters in file paths would crash mkdwarfs if these paths were displayed in the progress output. A possible workaround was to disable progress output. This fix replaces any invalid characters before displaying them. Fixes github #191.

• The CMakeLists.txt would bail out as soon as it discovered --as-needed in the linker flags. However, --as-needed is only a problem when combined with BUILD_SHARED_LIBS=ON. The check has been changed to only trigger if both conditions are met.

Other

• Minor speed improvements in ricepp compression.