Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: properly handle deletes in ss stores #17533

Merged
merged 5 commits into from
Aug 28, 2023

Conversation

alexanderbez
Copy link
Contributor

@alexanderbez alexanderbez commented Aug 25, 2023

Description

The PebbleDB and SQLite SS backends did not handle range deletes correctly as illustrated by #17505. This PR addresses handling deletes correctly for those aforementioned backends.

Changelog

  • For SQLite, we handle this via a new tombstone column where the value indicates the version it's being deleted at (if not already tombstoned). This allows us to query and iterate correctly.
  • For PebbleDB, we essentially do the exact same thing we do for SQLite, except we modify the value to contain the tombstone as a suffix.
  • Fix iteration lower bound bug for PebbleDB
  • Add more test coverage

ref: #17505
credit: @yihuang for the test updates


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • run make lint and make test
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

@yihuang
Copy link
Collaborator

yihuang commented Aug 25, 2023

there's an inconsistency in version 0 handling, rocksdb backend treat it as latest version, but other backends seems treat it normally:

	if version == 0 {
		ver = math.MaxUint64
	} else {
		ver = version
	}

@alexanderbez
Copy link
Contributor Author

alexanderbez commented Aug 25, 2023

That's correct and intentional. First version is 1. I noticed the VersionDB version did this and I didn't see a reason to do it. I also didn't want to work with int64 pointers.

Do you think it's worth doing?

@yihuang
Copy link
Collaborator

yihuang commented Aug 26, 2023

That's correct and intentional. First version is 1. I noticed the VersionDB version did this and I didn't see a reason to do it. I also didn't want to work with int64 pointers.

Do you think it's worth doing?

I mean the inconsistency between different backends, maybe change rocksdb backend behavior to the same as the other ones?

@alexanderbez alexanderbez marked this pull request as ready for review August 28, 2023 19:05
@alexanderbez alexanderbez requested a review from a team as a code owner August 28, 2023 19:05
@github-actions
Copy link
Contributor

@alexanderbez your pull request is missing a changelog!

@alexanderbez alexanderbez merged commit 7c0a1ba into feature/store-v2 Aug 28, 2023
45 of 46 checks passed
@alexanderbez alexanderbez deleted the bez/ss-fix-deletes branch August 28, 2023 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants