Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option to disable write stall #251

Merged
merged 2 commits into from
Aug 3, 2021

Conversation

Connor1996
Copy link
Member

  • Add an option disable-write-stall to disable write stall. It avoids RocksDB from being the write stall state, but still keeps the logic of decreasing/increasing compaction threads unchanged as before.
  • Use base_background_compactions to decide max_compactions when decrease/increase compaction threads.

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@Connor1996 Connor1996 changed the title Add option to disable write stall Add an option to disable write stall Jul 29, 2021
Copy link
Member

@tabokie tabokie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest looks good.

db/db_impl/db_impl_compaction_flush.cc Outdated Show resolved Hide resolved
Copy link
Collaborator

@yiwu-arbug yiwu-arbug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall.

Have you tested both of disable_write_stall and base_background_compactions are dynamic changeable?

options/cf_options.cc Show resolved Hide resolved
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@Connor1996
Copy link
Member Author

LGTM overall.

Have you tested both of disable_write_stall and base_background_compactions are dynamic changeable?

Add two test cases to cover the two configs.

@@ -829,7 +829,7 @@ WriteStallCondition ColumnFamilyData::RecalculateWriteStallConditions(
name_.c_str(), vstorage->estimated_compaction_needed_bytes(),
write_controller->delayed_write_rate());
} else {
assert(write_stall_condition == WriteStallCondition::kNormal);
assert(write_stall_condition == WriteStallCondition::kNormal || mutable_cf_options.disable_write_stall);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Connor1996 Minor: should be else if (write_stall_condition == WriteStallCondition::kNormal) ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each branch has its own precondition, which is "write_stall_condition is unset" here. For this specific spot, breaking the precondition might not cause any trouble, but it's a mistake nonetheless.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disable_write_stall is a kind of "write_stall_condition is unset", using else if needs to duplicate the logic in else branch.

tabokie pushed a commit to tabokie/rocksdb that referenced this pull request Aug 23, 2021
* add option to disable write stall

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
tabokie added a commit that referenced this pull request Aug 25, 2021
* add option to disable write stall

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

Co-authored-by: Connor <zbk602423539@gmail.com>
tabokie added a commit to tabokie/rocksdb that referenced this pull request Sep 2, 2021
This reverts commit 32b29a1.

Signed-off-by: tabokie <xy.tao@outlook.com>
tabokie added a commit that referenced this pull request Sep 3, 2021
…#258)

* Revert "Add an option to disable write stall (#251) (#252)"

This reverts commit 32b29a1.

Signed-off-by: tabokie <xy.tao@outlook.com>

* use ignore_unknown_options regardless of RocksDB version

Signed-off-by: tabokie <xy.tao@outlook.com>

* disable only for lower version and fix test

Signed-off-by: tabokie <xy.tao@outlook.com>
@tabokie tabokie mentioned this pull request May 9, 2022
39 tasks
tabokie pushed a commit to tabokie/rocksdb that referenced this pull request May 12, 2022
* add option to disable write stall

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
tabokie pushed a commit that referenced this pull request May 12, 2022
* add option to disable write stall

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
v01dstar pushed a commit to v01dstar/rocksdb that referenced this pull request Mar 5, 2024
* add option to disable write stall

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants