Skip to content

Releases: apache/kvrocks

2.9.0

09 Jul 10:08
@PragmaTwice PragmaTwice
v2.9.0
57b47a0
Compare
Choose a tag to compare
2.9.0 Latest
Latest

Important Notice

This new release introduces significant features, improvements, and fixes.

Numerous new commands are now supported, such as DUMP, SORT, LCS, COPY, ASKING, READONLY, and READWRITE. These additions enhance Kvrocks' capabilities by improving data migration and cluster features. A big thank you to all contributors for their valuable input to Kvrocks.

The default value of CMake option ENABLE_NEW_ENCODING is set to ON, which means the new data type encoding (specified here) is enabled by default while all old data is still compatible. (Note: After you upgrade to Kvrocks 2.9.0 and write new data, if you roll back Kvrocks to 2.3.0 or earlier versions, these newly-written data will not be available.)

Async IO in RocksDB is enabled by default since this release which might bring the performance benefit for the scan behavior if the io uring is supported.

In addition, the config option compaction-checker-range is still available but is marked as deprecated. We recommend using compaction-checker-cron instead. Also, the cron syntax in the config file has been extended. For example, you can now use cron expressions like * */2 1,3-6,8 * *.

Kvrocks Search (compatible with both SQL and RediSearch queries) and stream groups are two experimental new features that are already available in the unstable branch, but they are not enabled in this release.

RocksDB is still in version 8.11 (8.11.4) for this version. However, in the next feature release, we plan to bump RocksDB to version 9.

New Features

  1. Add the support of the LCS command by @JoverZhang in #2116
  2. Add compression level option by @Beihao-Zhou in #2171
  3. Add support of READONLY and READWRITE by @LiuYuHui in #2173
  4. Support value setting for nonexistent paths in JSON.SET by @PragmaTwice in #2178
  5. Add BIT support to BITPOS by @sheharyaar in #2170
  6. Add support of the command MOVEX by @Chiro11 in #2225
  7. Add support of the command DUMP by @AntiTopQuark in #2227
  8. Add support of the command JSON.MSET by @zjregee in #2228
  9. Add support of the SORT command by @PokIsemaine in #2262
  10. Support ASKING command by @jjz921024 in #2273
  11. Add support of the command JSON.DEBUG MEMORY by @jackyyyyyssss in #2323
  12. Enable the rocksdb async io by default by @xiaobiaozhao #2308

Bug Fixes

  1. Fix glog may buffer the stdout's output by @i18n-now in #2131
  2. Fix duplicate flag for building rocksdb by @aleksraiden in #2139
  3. Fix build warnings: local variable 's' will be copied despite being returned by name by @aleksraiden in #2148
  4. Fix missing fields in HELLO command response by @git-hulk in #2150
  5. Fix XREADGROUP command didn't fetch the latest metadata after creating a consumer by @Yangsx-1 in #2153
  6. Fix the pidfile and backup_dir will be rewritten even though they weren't modified by @git-hulk in #2186
  7. Fix wrongly append the ERR prefix in no script error by @lloydzhou in #2201
  8. Fix should forbid importing the slot which belongs to itself in cluster mode by @git-hulk in #2209
  9. Fix GetOptions in C++17 build by @mapleFU in #2213
  10. Fix wrongly try to rewrite the namespace in the cluster mode by @git-hulk in #2221
  11. Fix should use the minimum compatible RDB version when dumping the payload by @git-hulk in #2252
  12. Fix the replica should remove the master link when receiving the CLUSTER RESET command by @git-hulk in #2259
  13. Fix unused-variable error for lower gcc version due to structure bindings by @13015517713 in #2265
  14. Fix the error logging when the importer is not in a started state by @git-hulk in #2280
  15. Fix JSON.[STRLEN|OBJLEN] response inconsistently with Redis when the path doesn't exist by @jackyyyyyssss in #2336
  16. Fix minica version in CI workflow to prevent TLS test failure by @PragmaTwice in #2307
  17. Fix compiler warnings in KQIR and RDB by @raffertyyu in #2303
  18. Fix plan operator child iterator in KQIR by @PragmaTwice in #2282
  19. Fix JSON.ARRTRIM key no exists by @jackyyyyyssss in #2374
  20. Fix config: avoid rewriting the config file if it's unnecessary by @git-hulk in #2347
  21. Fix stream: make consumer decrement pending number when message is acknowledged by @LindaSummer in #2352

Code Improvements

  1. Extract common/port.h and optimize dbstats by @mapleFU in #2145
  2. Add macos arm64 runner to actions workflow by @PragmaTwice in #2155
  3. Optimize numeric comparison via interval analysis in KQIR by @PragmaTwice in #2257
  4. Refactor TTL processing in string type by @PragmaTwice in #2250
  5. Improve code style in cluster by @jihuayu in #2272
  6. Refactor Column Family Handling by @mapleFU in #2296
  7. Improve logging message for retryable background IO errors by @VasuDevrani in #2317
  8. Optimize the implementation of IntervalSet intersection by @13015517713 in #2300
  9. Improve RESP handling code in replication by @PragmaTwice in #2334
  10. Refactor cron: add CronPattern and implement more cron syntax by @PragmaTwice in #2377

Build & CI Changes

  1. Uniform image update/upgrade in Dockerfile by @aleksraiden in #2138
  2. Try to workaround the broken ubuntu image by @git-hulk in #2161
  3. Revert broken ubuntu image temporary fix by @enjoy-binbin in #2189
  4. Bump zstd to v1.5.6 by @aleksraiden in #2204
  5. Add semantics checker for KQIR by @PragmaTwice in #2207
  6. Add a simple pass manager for KQIR by @PragmaTwice in #2226
  7. Bump Snappy to 1.2.0 by @aleksraiden in #2224
  8. (ci) Bump apache/skywalking-eyes to 0.6.0 by @aleksraiden in #2241
  9. Enable CI for kvrocks2redis by @Zakelly in #2175
  10. ci(pr-lint): force the PR title to follow the conventional commits by @PragmaTwice in #2342
  11. Upload build artifact for the sonar scan by @git-hulk in #2321
  12. Fix missing temp dir while running sonar scan by @git-hulk in #2322
  13. Bump Snappy v1.2.1 by @aleksraiden in #2325
  14. Set ENABLE_NEW_ENCODING=FALSE in CI workflow by @PragmaTwice in #2333
  15. (ci) Remove 'ps aux' command from kvrocks2redis test by @aleksraiden in #2337
  16. chore: bump rocksdb to v9.2.1 by @aleksraiden in #2327
  17. chore: bump jsoncons to v176.0 by @aleksraiden in #2328
  18. test: fix listpack decoding in kvrocks2redis integration tests by @PragmaTwice in #2345
  19. build: set ENABLE_NEW_ENCODING=FALSE when building kvrocks2redis by @PragmaTwice in #2350
  20. chore: bump rocksdb to v9.2.2 by @aleksraiden in #2351

Documentation Changes

  1. Add more documents in KQIR by @PragmaTwice in #2196
  2. Fix invalid url in docs by @mapleFU in #2192
  3. Update "Type Conversion" doc by @PragmaTwice in #2235
  4. Fix doc typo by @Yangsx-1 in #2240
  5. Add more docs for the dynamic column family by @mapleFU in #2298

Contributors

aleksraiden, lloydzhou, and 23 other contributors
Assets 2
Loading

2.8.0

08 Mar 02:47
@caipengbo caipengbo
v2.8.0
311398a
Compare
Choose a tag to compare
2.8.0

Important Notice

Dependency Updates: RocksDB is bumped from 8.8.1 to 8.11.3 in this version.

Highlights:

Set the default value of redis-cursor-compatible to yes, so from this version, Kvrocks' SCAN will be compatible with Redis by default. We also proudly announce that RESP3 is fully supported now, you can enable the RESP3 feature via set resp3-enabled yes.

In addition, the cluster now can use the rocksdb write-batch to migrate slot data between instances via the APPLYBATCH command, which is faster and consumes less CPU compared to the replay command way.

It is worth mentioning that the community is working on the RedisSearch module, and we look forward to seeing RedisSearch in a future release.

New Features

Bug Fixes

  • Fix redundant ERR prefix in cluster redirect error message by @git-hulk in #1928
  • Avoid to fetch old value in SET command if NX/XX/GET/KEEPTTL is not set by @git-hulk in #1968
  • Fix parse BITCOUNT by @tisonkun in #1979
  • Fix RESET arity and command flags by @enjoy-binbin in #2018
  • Fix crash in zset store getkeys, fix zdiff/bzmpop range, add tests by @enjoy-binbin in #2051
  • Bugfix: BITCOUNT/BITPOS negative handling fixing by @mapleFU in #2069
  • Fix incorrent bytes to calculate length of ziplist/listpack 32bit string by @git-hulk in #2124

Improvements

Misc

Read more

Contributors

aleksraiden, pleaseshutup, and 19 other contributors
Assets 2
Loading

2.7.0

17 Dec 11:08
@git-hulk git-hulk
v2.7.0
33b4959
Compare
Choose a tag to compare
2.7.0

Important Notice

Dependency Updates: RocksDB is bumped from 8.3.3 to 8.8.1 in this version.

Highlights:

We're thrilled to announce that the JSON data structure is supported now, huge thanks to @PragmaTwice and all contributors for making this happen. Also, basic capabilities of Functions have been supported to make up for the lack of modularity of the EVAL-series commands.

From this release, the speedb is allowed to be used as an alternative to rocksdb, you have a try as simple as to append the build option -DENABLE_SPEEDB=ON. Additionally, dynamic change number of worker threads is also supported and users can change it via CONFIG SET workers {NUMBER}. Furthermore, numerous new commands and bug fixes have been implemented.

NOTICE: JSON and BloomFilter is not ready for the cluster mode yet.

New Features

Bug Fixes

Improvements

Misc

New Contributors

Read more

Contributors

aleksraiden, skyitachi, and 21 other contributors
Assets 2
Loading

2.6.0

26 Sep 07:42
@PragmaTwice PragmaTwice
v2.6.0
cac0ade
Compare
Choose a tag to compare
2.6.0

Important Notice

Dependency Updates: RocksDB is bumped from 8.3.2 to 8.3.3 in this version.

Configuration Changes: The default value of max-io-mb is changed to 0 (unlimited), and rocksdb.level_compaction_dynamic_level_bytes changed to yes from no.

Highlights: In this release, we have added support for the BloomFilter data structure and most of its commands (prefixed with BF.). Special thanks to @zncleon and @mapleFU. Additionally, the RESTORE command has been supported by @git-hulk to facilitate various data migration scenarios, such as using RedisShake upon Kvrocks. Moreover, TLS can now secure not only server connections but also replication connections. Furthermore, numerous new commands and bug fixes have been implemented. We extend our gratitude to all contributors!

New Features

Improvements

Bug Fixes

Read more

Contributors

aleksraiden, ovaistariq, and 21 other contributors
Assets 2
Loading

2.5.1

05 Aug 12:47
@git-hulk git-hulk
v2.5.1
83eeef4
Compare
Choose a tag to compare
2.5.1

Highlights: This release only contains bug fixes since 2.5.0 and some critical issues need to be noticed. For #1615, the data race when reading the cluster slot's info may cause the server to crash in cluster mode, and #1568 / #1593 will also crash the server if users pass the wrong arguments.

Bug Fixes

Contributors

ovaistariq, git-hulk, and 3 other contributors
Assets 2
Loading

2.5.0

12 Jul 01:55
@git-hulk git-hulk
v2.5.0
1e199d2
Compare
Choose a tag to compare
2.5.0

Important Notice

Dependency Updates: RocksDB is bumped from 8.1.1 to 8.3.2 in this version.

Highlights: We have supported Redis-compatible SCAN cursors so that you can use any Redis client library to perform the SCAN far painless than before. Enable it via redis-cursor-compatible in the configuration file. Many thanks to @jihuayu. Meanwhile, lots of ZSET commands are newly supported, like ZMPOP, ZRANGESTORE, BZMPOP, ZUNION. Many thanks to @infdahai for driving them and other contributors for implementing them. The detailed changes and contributors are listed below.

New Features

Improvements

Bug Fixes

Build & CI related

Misc

New Contributors

Full Changelog: v2.4.0...v2.5.0-rc2

Contributors

aleksraiden, rueian, and 20 other contributors
Assets 2
Loading

2.4.0

03 May 03:49
@PragmaTwice PragmaTwice
v2.4.0
d539acd
Compare
Choose a tag to compare
2.4.0

Note: In this release, we have upgraded RocksDB to version 8.1.1. In addition, we have introduced a new storage encoding format that can accommodate 64-bit element count and expiration time in milliseconds. To enable this feature, you must include -DENABLE_NEW_ENCODING=ON during the build process (which is not enabled by default). Once the new encoding is enabled, data stored using the old encoding will remain accessible for reading and writing as usual. However, any new data will be written using the new encoding format.

New Features

Improvements

Bug Fixes

Build & CI related

  • Automatically build and push the docker image when tagging a new version by @git-hulk in #1247
  • Revert "Automatically build and push the docker image when tagging a new version" by @git-hulk in #1248
  • Add header filter for clang-tidy checks by @PragmaTwice in #1249
  • Automatically set the env MACOSX_DEPLOYMENT_TARGET to the current macOS version unless the env is already set by @git-hulk in #1250
  • Enable LuaJIT to improve the performance of scripts by @xiaobiaozhao in #1241
  • build: Drop travis workflow by @tisonkun in #1258
  • Use nproc instead of default for job numbers in docker build to avoid timeout by @PragmaTwice in #1272
  • build: package sources in a folder by @tisonkun in #1332
  • Fix clang-tidy warnings by @torwig in #1334
  • Add clang-tidy checking for unit tests by @PragmaTwice in #1337
  • Add some readability checks and fix all reports in current codebase by @PragmaTwice in #1348
  • Fix stream_metadata enum -Wint-in-bool-context warning by @enjoy-binbin in #1349
  • Enable readability-identifier-naming in clang-tidy by @PragmaTwice in #1383
  • Fix member name convension for readability-identifier-naming by @PragmaTwice in #1385
  • Remove underscore suffix for public member name by @PragmaTwice in #1386
  • Check static class member and type alias name in readability-identifier-naming by @PragmaTwice in #1387
  • Check method and namespace names in readability-identifier-naming by @PragmaTwice in #1388
  • Check function names in readability-identifier-naming by @PragmaTwice in #13...
Read more

Contributors

aleksraiden, DenizPiri, and 14 other contributors
Assets 2
Loading

2.3.0

22 Feb 13:05
@git-hulk git-hulk
v2.3.0
8eb4a2c
Compare
Choose a tag to compare
2.3.0

For this release, we now upgrade the RocksDB to 7.9.2 which will significantly improve the compaction performance. Another to be noticed, Kvrocks cluster mode will persist the cluster nodes' topology in the local file by default, you can disable it in flight by config set persist-cluster-nodes-enable no.

New Features

  • Add config for log cleaner by @IoCing in #1171
  • Persist the cluster nodes info after applying the cluster topology by @git-hulk in #1219
  • Add ability to set log level via config file by @torwig in #1230

Improvements

Bug Fixes

  • Fix logging initialized before loading by @mapleFU in #1061
  • Harden list_test.go by @tisonkun in #1141
  • Fix redis cli test will fail if cannot read response at once by @git-hulk in #1129
  • Fix the script load should return SHA as the bulk string by @git-hulk in #1133
  • Fix format output in Server::GetRocksDBStatsJson by @PragmaTwice in #1142
  • Increase blocking timeout to harden the blocking pop test case by @git-hulk in #1128
  • Fix wrongly append the ERR prefix in no script error by @git-hulk in #1162
  • Check if the thread is joinable before joining in the replication thread by @git-hulk in #1195
  • Fix SockConect doesn't resolve domain before connecting by @git-hulk in #1183
  • Fix error message passing in Server::Start by @PragmaTwice in #1225
  • Fix deadlock when unblocking connection on stream by @torwig in #1228

Refine Code Style

New Contributors

Contributors

aleksraiden, kukey, and 16 other contributors
Assets 2
Loading

2.2.0

10 Nov 03:47
@PragmaTwice PragmaTwice
v2.2.0
17b31e7
Compare
Choose a tag to compare
2.2.0

Notice

For this release, we are still using rocksdb 6.29, but we will update to rocksdb 7 in the next release (as in the unstable branch).

New Features

  • Allow using LuaJIT to improve the performance of the Lua script (#697)
  • Add the new command GETDEL (#771)
  • Add the new command SMISMEMBER (#778)
  • Add the new data structure Redis STREAM (#745)
  • Add TLS support for kvrocks connections (#797)
  • Add the new command HELLO (#881)
  • Add the new command EVAL_RO (#782)
  • Add the new command DISK (#882)
  • Add the new command GETEX (#961)
  • Support CLI options for the kvrocks server (#1037)
  • Add the new command options for ZADD (#1022)
  • Add ZLIB dependency for compression in rocksdb (#1078)

Improvements

  • Switch the default value of log-dir to stdout (#754)
  • Add fpm packaging to x.py (#752)
  • Propagate sanitizer flags to all cmake deps (#759)
  • Move runtest to x.py with customized server/cli path (#765)
  • Add StatusOr for error handling in modern C++ style (#768)
  • Add ParseInt in StatusOr style (#787)
  • Show function name in backtrace (#796)
  • Optimize get bitmap string and remove redundant num2bit array (#793)
  • Allows Kvrocks to listen to only the unix socket (#809)
  • Update dependency to latest version (lz4, libevent, gtest) (#828)
  • Export the rocksdb write options (#885)
  • Support quoted string and inline comment in config (#849)
  • Add the FilterBlobByKey for SubKeyFilter (#902)
  • Avoid using Get when iterating sub-keys during migration (#906)
  • Support EXAT/PEXAT option in the set command (#901)
  • Return from HRANGE if the key wasn't found (#923)
  • Update jemalloc to 5.3.0 (#940)
  • Use ParseInt to replace sto*, ato* (#924)
  • Add UniqueFD and ScopeExit (#973)
  • Align georadius behavior of redis (#993)
  • Align the expire option behavior of redis (#1017)
  • Support config set backup-dir new-dir (#1026)
  • Use a consistent way to get the current time (#1038)
  • Add support for showing used_memory_startup (#1044)
  • Improve stack trace format and clean main.cc (#1053)
  • Add command parser (#1032)
  • Add WriteBatchInspector to inspect WriteBatch (#1069)
  • Improve libunwind linking in CMake (#1079)

Bug Fixes

  • Fix didn't return the out of range error in cluster import command(#767)
  • Fix should forbid to create of the default namespace (#772)
  • Fix CMake configuration error in snappy (#803)
  • Fix inline protocol don't allow LF only EOL (#808)
  • Fix successor commands won't be processed before receiving the next read event (#839)
  • Fix georadius should return an empty set if the key does not exist (#845)
  • Fix config set rocksdb.blob_garbage_collection_age_cutoff (#914)
  • Fix ping with arg should return a bulk string (#933)
  • Fix server cannot exit properly when enabling cluster mode (#969)
  • Fix shouldn't lowercase the configuration value in kvrocks2redis(#971)
  • Fix don't duplicate killed clients which have already flagged kCloseAfterReply (#1020)
  • Fix properly parse cmake version (#1043)
  • Fix rename-command only the first one works(#1047)
  • Fix CAS and CAD return statuses (#1055)
  • Fix random crash in the migration test case (#1068)

Test Refactoring

TCL tests are now moved to Go, refer to #811.

Misc

  • Replace cpplint with clang-format (#979)
  • Refactor source file structure (#989)
  • Update C++ standard to 17 (#1006)
  • Use clang-tidy instead of cppcheck (#1025)
Assets 2
Loading

2.1.0

03 Aug 13:37
@git-hulk git-hulk
v2.1.0
2cf4e16
Compare
Choose a tag to compare
2.1.0

New features

  • Support source packaging by x.py (#729)
  • Support to use lz4 compression in RocksDB (#584)
  • Allow to send logs to stdout/stderr (#613)
  • LPOP/RPOP support pop multi elements (#594)
  • Add LMove command (#577)
  • Support to listen on the IPV6 address (#554)
  • Add support for listening to unix socket (#531)
  • Support BITOP command (#471)
  • Support unlink command (#517)

Improvements

  • replace shell scripts with x.py (#725)
  • Use lock_guard to replace the manual mutex lock/unlock (#723)
  • Build docker image with travis ci (#724)
  • Return values instead of passing pointers (#722)
  • Don't call GetCurrentTime in expired and type when possible (#709)
  • Remove the slice ToString() when encoding or decoding the key (#707)
  • Use the const reference for avoiding copy (#704)
  • Check sanitizer status for TCL tests in CI (#698)
  • CMake: support ninja as a build backend (#626)
  • Build static lib for glog (#618)
  • Improve using of ASan and TSan in CMake build (#599)
  • PSYNC based on Unique Replication Sequence ID (#538)
  • Remove handwritten makefiles to keep only one build system (#576)
  • Use FetchContent instead of ExternalProject and git submodules in CMake (#564)
  • Upgrade the rocksdb version to 6.29.5(latest version for 6.x) (#555)
  • Support to send auth command before migrating slots (#514)
  • Allow to dynamic resize the level_compaction_dynamic_level_bytes and level base size (#497)

Bugfixes

  • Make the set command consistent with redis (#730)
  • Fix some time dependencies in replication.tcl (#688)
  • Fix get rocksdb ops tcl test case (#668)
  • Fix typos in config file (#669)
  • Fix replication tests should wait for server restarted (#660)
  • Fix the role command can't be used when loading data (#661)
  • Fix Wrongly parsed the RESP empty/null array (#652)
  • Fix compile warnings in kvrocks (#653)
  • Fix unmatched iterator in Cluster::SetClusterNodes (#646)
  • Fix load namespaces from the config file would ignore case (#642)
  • Fix RocksDB can't auto resume after disk quota exceeded error (#628)
  • Fix don't swallow the error when creating column families failed (#620)
  • Compile lua by C++ compilers to avoid memory leaks (#614)
  • Fix the slave may crash when restoring the db (#606)
  • Fix leaks reported by ASan (#605)
  • Fix can't find jemalloc when building with makefile on MacOSX (#557)
  • Stop sending files if failed in full synchronization (#539)
  • Avoid requirepass and masterauth conflicts with the namespace tokens (#507)
  • Fix slave can't resync with master after password change (#520)
  • Fix rocksdb cmake depends should use list instead of string (#518)
  • Fix spop command should return bulk string instead of array when without count(#515)
  • Fix wrong replication state (#506)
  • Avoid flush all redis db in kvrocks2redis (#498)
  • Fix failing in setting thread name with string more than 16 byte (#496)
  • Avoid accessing slot_migrate_ before it is created (#472)
  • Fix the connection fd was used after free (#479)

Others

  • Add use case CIRCL.lu (#546)
  • Change license to Apache 2.0 (#533)
  • Add DISCLAIMER (#558)
  • Apply Apache License header for Tcl files (#548)
  • Add .asf.yaml file (#545)
  • Apply Apache License header for source code (#543)
Assets 2
Loading