Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow to query DB stall status #226

Merged
merged 4 commits into from
Feb 20, 2021
Merged

allow to query DB stall status #226

merged 4 commits into from
Feb 20, 2021

Conversation

hicqu
Copy link

@hicqu hicqu commented Feb 18, 2021

Signed-off-by: qupeng qupeng@pingcap.com

This PR adds a new property is-write-stalled to query whether the column family is in write stall or not.

Why it's necessary?

In TiKV there is a compaction filter used for GC, in which DB::write is called. So if we can query whether the DB instance is stalled or not, we can skip to create more compaction filter instances to save some resources.

Copy link
Member

@tabokie tabokie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind explain why we need this specifically? And can that be fulfilled with a callback instead?

include/rocksdb/utilities/stackable_db.h Outdated Show resolved Hide resolved
@yiwu-arbug
Copy link
Collaborator

yiwu-arbug commented Feb 18, 2021

Can we implement this as a DB property? That would avoid adding the new DB method, and I think it would be easier to maintain.

db/db_impl/db_impl.h Outdated Show resolved Hide resolved
Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: qupeng <qupeng@pingcap.com>
Copy link
Collaborator

@yiwu-arbug yiwu-arbug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM.

db/internal_stats.cc Outdated Show resolved Hide resolved
Signed-off-by: qupeng <qupeng@pingcap.com>
@tabokie tabokie merged commit 0d1ac74 into tikv:6.4.tikv Feb 20, 2021
@hicqu hicqu deleted the db-query-stall branch February 20, 2021 07:15
@tabokie tabokie mentioned this pull request May 9, 2022
39 tasks
tabokie pushed a commit to tabokie/rocksdb that referenced this pull request May 12, 2022
This PR adds a new property is-write-stalled to query whether the column family is in write stall or not.

In TiKV there is a compaction filter used for GC, in which DB::write is called. So if we can query whether the DB instance is stalled or not, we can skip to create more compaction filter instances to save some resources.

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
tabokie pushed a commit that referenced this pull request May 12, 2022
This PR adds a new property is-write-stalled to query whether the column family is in write stall or not.

In TiKV there is a compaction filter used for GC, in which DB::write is called. So if we can query whether the DB instance is stalled or not, we can skip to create more compaction filter instances to save some resources.

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
v01dstar pushed a commit to v01dstar/rocksdb that referenced this pull request Feb 21, 2024
This PR adds a new property is-write-stalled to query whether the column family is in write stall or not.

In TiKV there is a compaction filter used for GC, in which DB::write is called. So if we can query whether the DB instance is stalled or not, we can skip to create more compaction filter instances to save some resources.

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
v01dstar pushed a commit to v01dstar/rocksdb that referenced this pull request Feb 22, 2024
compaction_filter: add bottommost_level into context (tikv#160)

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

add range for compaction filter context (tikv#192)

* add range for compaction filter context

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

allow no_io for VersionSet::GetTableProperties (tikv#211)

* allow no_io for VersionSet::GetTableProperties

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

expose seqno from compaction filter and iterator (tikv#215)

This PR supports to access `seqno` for every key/value pairs in compaction filter or iterator.
It's helpful to enhance GC in compaction filter in TiKV.

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

allow to query DB stall status (tikv#226)

This PR adds a new property is-write-stalled to query whether the column family is in write stall or not.

In TiKV there is a compaction filter used for GC, in which DB::write is called. So if we can query whether the DB instance is stalled or not, we can skip to create more compaction filter instances to save some resources.

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

Fix compatibilty issue with Titan

Signed-off-by: v01dstar <yang.zhang@pingcap.com>

filter deletion in compaction filter (tikv#344)

And delay the buffer initialization of writable file to first actual write.

---------

Signed-off-by: tabokie <xy.tao@outlook.com>

Adjustments for compaptibilty with 8.10.facebook

Signed-off-by: v01dstar <yang.zhang@pingcap.com>

Adjust tikv related changes with upstream

Signed-off-by: v01dstar <yang.zhang@pingcap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants