v1.11.0

[romange]

Dragonfly v1.11.0

🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱
Our beloved country is currently facing difficult times, but we will win, for hope and progress are on our side.
🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱 🇮🇱

Important features and changes:

• Starting from v1.11.0 we support the BullMQ framework. For more information see BullMQ post and our docs.
• Lots of ACL/Auth fixes. Specifically (IMPORTANT): we changed the default http user to be "default" instead of "user" to be consistent
  with Dragonfly ACL rules. We also removed the requirement for authenticated access to /metrics http page even if ACL are enabled.
• STREAMS and ACL APIs are complete! Please let us know if you encounter any issues.
• Lots of performance improvements for Sorted Sets. Now Dragonfly uses our own b+tree implementation for sorted sets by default instead
  of the old Redis skiplist implementation. We improved some memory and performance bottlenecks with listpack encoding as well.
• Fix logrotate rules for our debian package.

What's Changed

• fix: Add missing encodings to strEncoding by @royjacobson in fix: Add missing encodings to strEncoding #1798
• fix: add maxmemory_policy item to info memory section by @romange in fix: add maxmemory_policy item to info memory section #1802
• Update Dockerfile.alpine-dev by @dranikpg in Update Dockerfile.alpine-dev #1803
• chore: add macos daily build by @romange in chore: add macos daily build #1795
• feat(search): sized vectors by @dranikpg in feat(search): sized vectors #1788
• fix: alpine weekly pipeline by @romange in fix: alpine weekly pipeline #1811
• feat (hset): Support arguments (count, withvalues) in HRANDFIELD by @theyueli in feat (hset): Support arguments (count, withvalues) in HRANDFIELD #1804
• Remove ICU library by @dranikpg in Remove ICU library #1812
• bug(server): multi cleanup on error by @adiholden in bug(server): multi cleanup on error #1813
• chore(AclFamily): Move some of the regressions tests to unit and add by @kostasrim in chore(AclFamily): Move some of the regressions tests to unit and add #1816
• feat(stream): add support for xpending command by @Abhra303 in feat(stream): add support for xpending command #1806
• chore(AclFamily): remove unused using decl from acl_family_test by @kostasrim in chore(AclFamily): remove unused using decl from acl_family_test #1817
• feat(stream): add support for xclaim command by @Abhra303 in feat(stream): add support for xclaim command #1372
• docs: update README.zh-CN.md by @yancz2000 in docs: update README.zh-CN.md #1772
• chore: partially revert adjust batching behavior to reduce network by @kostasrim in chore: partially revert adjust batching behavior to reduce network #1821
• feat(search): HNSW by @dranikpg in feat(search): HNSW #1799
• fix: possible segfault bug by @romange in fix: possible segfault bug #1822
• fix(aws): initialise aws in a proactor by @andydunstall in fix(aws): initialise aws in a proactor #1818
• fix: Reply within async_dispatch block in squashed pipeline by @dranikpg in fix: Reply within async_dispatch block in squashed pipeline #1825
• Revert "chore: partially revert adjust batching behavior to reduce network" by @dranikpg in Revert "chore: partially revert adjust batching behavior to reduce network" #1824
• feat(AclFamily): load/store aclfile by @kostasrim in feat(AclFamily): load/store aclfile #1820
• feat(snapshot): add snapshot storage by @andydunstall in feat(snapshot): add snapshot storage #1827
• fix: regression tests for acl family test by @kostasrim in fix: regression tests for acl family test #1826
• fix(regression-tests): correct path for acl files by @kostasrim in fix(regression-tests): correct path for acl files #1829
• feat(CI): Configure sccache to work with GCS by @royjacobson in feat(CI): Configure sccache to work with GCS #1815
• Revert "feat(CI): Configure sccache to work with GCS" by @royjacobson in Revert "feat(CI): Configure sccache to work with GCS" #1833
• skip test_multidim_knn by @adiholden in skip test_multidim_knn #1834
• fix(search-parser): fix AND, OR, NOT parsing issue by @Equilibrium2702 in fix(search-parser): fix AND, OR, NOT parsing issue #1838
• feat(snapshot): move load path to snapshot storage by @andydunstall in feat(snapshot): move load path to snapshot storage #1830
• fix(AclFamily): move acl test files to pytest by @kostasrim in fix(AclFamily): move acl test files to pytest #1843
• Add build_rpm script and rpm spec by @romange in Add build_rpm script and rpm spec #1831
• Support 'databases' alias for CONFIG GET by @romange in Support 'databases' alias for CONFIG GET #1849
• chore(regression): send chat notificaiton only on main branch by @adiholden in chore(regression): send chat notificaiton only on main branch #1847
• Fix a potential crash in OnBreakCb by @romange in Fix a potential crash in OnBreakCb #1850
• fix(tls): User friendly logging of OpenSSL errors by @royjacobson in fix(tls): User friendly logging of OpenSSL errors #1851
• opt(lua): Coordinate single-shard Lua evals in remote thread by @chakaz in opt(lua): Coordinate single-shard Lua evals in remote thread #1845
• feat(debug): Adds a DEBUG STACKTRACE subcommand by @royjacobson in feat(debug): Adds a DEBUG STACKTRACE subcommand #1848
• feat(server): Getting flags from the environment variables by @aidarbek in feat(server): Getting flags from the environment variables #1810
• feat(snapshot): add support for snapshot recovery from S3 by @andydunstall in feat(snapshot): add support for snapshot recovery from S3 #1839
• fix: dfly_main small typo on unix_socket flag by @kostasrim in fix: dfly_main small typo on unix_socket flag #1853
• chore: enable search lib on Apple by @romange in chore: enable search lib on Apple #1854
• feat(tests): unset test DFLY env variables by @andydunstall in feat(tests): unset test DFLY env variables #1856
• feat(snapshot): fix build on macos by @andydunstall in feat(snapshot): fix build on macos #1861
• fix(ServerFamily): proper initialization of FileSnapshotStorage when replicaof flag is used by @kostasrim in fix(ServerFamily): proper initialization of FileSnapshotStorage when replicaof flag is used #1858
• feat(AclFamily): add acl commands by @kostasrim in feat(AclFamily): add acl commands #1844
• Run unit tests in macos build by @romange in Run unit tests in macos build #1859
• fix: fix json.arrappend not allowing passing JSON objects by @UkuLoskit in fix: fix json.arrappend not allowing passing JSON objects #1867
• chore: build rpm package for the release by @romange in chore: build rpm package for the release #1866
• fix(Auth): authenticate with any password for default user by @kostasrim in fix(Auth): authenticate with any password for default user #1868
• tests: choose open ports randomally by @royjacobson in tests: choose open ports randomally #1569
• chore: Fix regression test after tests: choose open ports randomally #1569 by @royjacobson in chore: Fix regression test after #1569 #1872
• chore: Add a context manager to DflyInstance so we don't forget to close them. by @royjacobson in chore: Add a context manager to DflyInstance so we don't forget to close them. #1873
• feat(replication): Use a ring buffer with messages to serve replication. by @royjacobson in feat(replication): Use a ring buffer with messages to serve replication. #1835
• fix: weekly build by @romange in fix: weekly build #1871
• feat(AclFamily): add acl log by @kostasrim in feat(AclFamily): add acl log #1865
• feat: Span-all no-key transactional commands by @dranikpg in feat: Span-all no-key transactional commands #1864
• fix: Disable Int<> calls in cmd_arg_parser_test for MacOS by @dranikpg in fix: Disable Int<> calls in cmd_arg_parser_test for MacOS #1880
• feat(AclFamily): add acl users command by @kostasrim in feat(AclFamily): add acl users command #1882
• feat(AclFamily): add acl cat command by @kostasrim in feat(AclFamily): add acl cat command #1883
• feat(AclFamily): add acl getuser command by @kostasrim in feat(AclFamily): add acl getuser command #1884
• feat(AclFamily): add acl dryrun command by @kostasrim in feat(AclFamily): add acl dryrun command #1894
• fix(regression-tests): increase waiting delay for df startup by @kostasrim in fix(regression-tests): increase waiting delay for df startup #1893
• fix: improve performance of listpack sorted sets by @romange in fix: improve performance of listpack sorted sets #1885
• chore: optimize BPTreeNode::BSearch by @romange in chore: optimize BPTreeNode::BSearch #1891
• chore: Introduce basic deadlock detector for tests by @romange in chore: Introduce basic deadlock detector for tests #1896
• fix: make blocking_controller_test being built on macos by @romange in fix: make blocking_controller_test being built on macos #1903
• fix: Better error handling in search algorithm by @dranikpg in fix: Better error handling in search algorithm #1881
• bug(server): zadd wrong insert when non unique members by @adiholden in bug(server): zadd wrong insert when non unique members #1906
• run regression tests on CI by @adiholden in run regression tests on CI #1874
• fix(server): Flip tcp_nodelay to on by default by @royjacobson in fix(server): Flip tcp_nodelay to on by default #1910
• bug(server): global command stalls on server load with pipeline mode by @adiholden in bug(server): global command stalls on server load with pipeline mode #1909
• fix: avoid calling 'sink_->Write()' with an empty buffer by @romange in fix: avoid calling 'sink_->Write()' with an empty buffer #1912
• doc(server): Copy important note from transactions doc to the code by @chakaz in doc(server): Copy important note from transactions doc to the code #1913
• fix: Correctly persist vector indices by @dranikpg in fix: Correctly persist vector indices #1905
• opt(server): Call reserve() with correct argument by @chakaz in opt(server): Call reserve() with correct argument #1914
• opt(lua): Avoid separate hops for lock & unlock on single shard execution by @chakaz in opt(lua): Avoid separate hops for lock & unlock on single shard execution #1900
• chore: remove redundant Find calls when inserting into DenseSet by @romange in chore: remove redundant Find calls when inserting into DenseSet #1917
• fix: Actually print fibers from all the threads by @romange in fix: Actually print fibers from all the threads #1918
• chore(regtest): Update redis dependency by @dranikpg in chore(regtest): Update redis dependency #1915
• chore: implement ParseDouble using fast_float by @romange in chore: implement ParseDouble using fast_float #1920
• fix(regtests): filter client setinfo from rewrite log by @dranikpg in fix(regtests): filter client setinfo from rewrite log #1921
• bug(streams): fixing xreadgroup's behavior to be compatible with Redis by @theyueli in bug(streams): fixing xreadgroup's behavior to be compatible with Redis #1908
• fix(regests): Fix port detection with gdb by @dranikpg in fix(regests): Fix port detection with gdb #1922
• feat(json): support JSON.GET pretty print options by @iko1 in feat(json): support JSON.GET pretty print options #1832
• feat(search): return scores by @dranikpg in feat(search): return scores #1870
• chore: Avoid allocating unique_members arrays by @romange in chore: Avoid allocating unique_members arrays  #1911
• fix(dragonfly_test): failing bug207 on build opt by @kostasrim in fix(dragonfly_test): failing bug207 on build opt #1919
• feat(build): fix lua on mac arm64 by @andydunstall in feat(build): fix lua on mac arm64 #1930
• chore: performance improvements around zset by @romange in chore: performance improvements around zset #1928
• fix: macos segfault loading snapshot by @andydunstall in fix: macos segfault loading snapshot #1931
• feat(replication): First iteration on partial sync. by @royjacobson in feat(replication): First iteration on partial sync. #1836
• fix: set the key index within the json.debug command by @iko1 in fix: set the key index within the json.debug command #1932
• chore: flip default sorted set implementation to bptree by @romange in chore: flip default sorted set implementation to bptree #1933
• chore: improve doubles encoding for listpacks by @romange in chore: improve doubles encoding for listpacks #1938
• feat(search): Early limit cutoff by @dranikpg in feat(search): Early limit cutoff #1943
• feat(streams): support XACK command by @theyueli in feat(streams): support XACK command #1923
• chore: Change the DNS log message to something that makes sense externally by @royjacobson in chore: Change the DNS log message to something that makes sense externally #1947
• fix(regtests): Bind to 127.0.0.1 and not localhost by @royjacobson in fix(regtests): Bind to 127.0.0.1 and not localhost #1937
• bug(streams): entries_read is not updated on xreadgroup command by @theyueli in bug(streams): entries_read is not updated on xreadgroup command #1946
• fix: test_cancel_replication_immediately by @kostasrim in fix: test_cancel_replication_immediately #1942
• fix: small typos on flags by @kostasrim in fix: small typos on flags #1953
• fix(AclFamily): do not allow to delete default user by @kostasrim in fix(AclFamily): do not allow to delete default user #1954
• chore: Small refactor of DflyInstance by @royjacobson in chore: Small refactor of DflyInstance #1951
• feat(streams): support entries_read and lag for XINFO GROUPS by @theyueli in feat(streams): support entries_read and lag for XINFO GROUPS #1952
• fix(regtests): Limit infinite loop in replication_test by @royjacobson in fix(regtests): Limit infinite loop in replication_test #1961
• Add a feature flag for partial_sync by @royjacobson in Add a feature flag for partial_sync #1957
• chore: preallocate lua arrays by @romange in chore: preallocate lua arrays #1964
• fix(regTests): adjust network disconnect small buffer to pass on by @kostasrim in fix(regTests): adjust network disconnect small buffer to pass on #1970
• feat(cmd): add restricted commands flag by @andydunstall in feat(cmd): add restricted commands flag #1967
• chore: rename owner to conn by @romange in chore: rename owner to conn #1973
• chore: adjust verbosity level 1 not be over-verbose in prod by @romange in chore: adjust verbosity level 1 not be over-verbose in prod #1974
• feat(search): Basic sorting by @dranikpg in feat(search): Basic sorting #1941
• fix(regtests): Colored per-instance log by @dranikpg in fix(regtests): Colored per-instance log #1971
• chore: Print useful info when printing DflyInstance objects by @romange in chore: Print useful info when printing DflyInstance objects #1977
• chore(regression): test bptree on regression pytests by @adiholden in chore(regression): test bptree on regression pytests #1963
• fix(RegTests): fix replication test_network_disconnect_active_stream by @kostasrim in fix(RegTests): fix replication test_network_disconnect_active_stream #1980
• chore(dfly_main): refactor to use RAII by @kostasrim in chore(dfly_main): refactor to use RAII #1981
• chore(Http): change authed username to default and skip auth on metrics by @kostasrim in chore(Http): change authed username to default and skip auth on metrics #1950
• fix(regtests): Reduce load on redis replication test by @dranikpg in fix(regtests): Reduce load on redis replication test #1984
• fix: typos by @omahs in fix: typos #1986
• fix(server): Fix async lua by @dranikpg in fix(server): Fix async lua #1982
• chore: DEBUG SHARDS by @dranikpg in chore: DEBUG SHARDS #1987
• fix(Listener): skip auth when requirepass is empty on http by @kostasrim in fix(Listener): skip auth when requirepass is empty on http #1985
• feat(streams): support LASTID option for XCLAIM command. by @theyueli in feat(streams): support LASTID option for XCLAIM command. #1968
• Docs: fix typos and grammar by @masterujjval in Docs: fix typos and grammar #1983
• fix(regTests): adjust small network buff test by @kostasrim in fix(regTests): adjust small network buff test #1991
• feat(search): Plug in polymorphic allocator by @dranikpg in feat(search): Plug in polymorphic allocator #1819
• fix(regTests): test_cancel_replication_immediately races by @kostasrim in fix(regTests): test_cancel_replication_immediately races #1990
• feat(aws): add s3 awsv2 by @andydunstall in feat(aws): add s3 awsv2 #1929
• fix(server): Add additional metrics by @dranikpg in fix(server): Add additional metrics #1975
• fix: fix eval in multi interpreter borrow by @dranikpg in fix: fix eval in multi interpreter borrow #1999
• fix: fix queue backpressure access by @dranikpg in fix: fix queue backpressure access #2000
• feat: add support for XINFO STREAM stream_name by @rounaknandanwar in feat: add support for XINFO STREAM stream_name #1533
• update(requirepass): behave as redis requirepass by @kostasrim in update(requirepass): behave as redis requirepass #1996
• feat(streams): support XAUTOCLAIM command by @theyueli in feat(streams): support XAUTOCLAIM command #1969
• fix: make --help flag work for development builds by @romange in fix: make --help flag work for development builds #2004
• fix: logrotate for dragonfly logs by @romange in fix: logrotate for dragonfly logs #1972
• fix(acl/admin_port): skip acl validation on admin port by @kostasrim in fix(acl/admin_port): skip acl validation on admin port #1997
• opt(server): If unspecified, use num_shards == proactor_threads by @chakaz in opt(server): If unspecified, use num_shards == proactor_threads #1998
• feat(aws): s3 snapshot error handling by @andydunstall in feat(aws): s3 snapshot error handling #2002
• chore: add balls and bins simulator by @romange in chore: add balls and bins simulator #2001

New Contributors

• @yancz2000 made their first contribution in docs: update README.zh-CN.md #1772
• @Equilibrium2702 made their first contribution in fix(search-parser): fix AND, OR, NOT parsing issue #1838
• @aidarbek made their first contribution in feat(server): Getting flags from the environment variables #1810
• @UkuLoskit made their first contribution in fix: fix json.arrappend not allowing passing JSON objects #1867
• @omahs made their first contribution in fix: typos #1986
• @masterujjval made their first contribution in Docs: fix typos and grammar #1983

Huge thanks to all the contributors! ❤️

Slava Ukraini ! 🇺🇦
Glory to Israel ! 🇮🇱

Full Changelog: v1.9.0...v1.11.0

---

This discussion was created from the release v1.11.0.

[misnaged]

עם ישראל חי

[snird]

Love it Roman!
Am Israel Chai!

[shamil]

🇮🇱 we will win

[rhummelmose]

<3 <3 <3

[masterujjval]

India is with Israel <3

[amrebada]

stand with Palestine 🇵🇸

[martadinata666]

I'm sorry, please don't add political issue on software release.