feat: implement object stores #206

yihuang · 2024-03-23T08:59:37Z

Description

Module parameters are loaded multiple times for each tx, although the raw bytes are cached, the decoding process is not.

Support an object store that is transient stores for interface{}, applications can cache objects directly, without doing unnecessary encoding/decoding.

Caching module parameters is an important use case for this, there are also many use cases out there.

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

confirmed the correct type prefix in the PR title
confirmed ! in the type prefix if API or client breaking change
confirmed all author checklist items have been addressed
reviewed state machine logic
reviewed API design and naming
reviewed documentation is accurate
reviewed tests and test coverage
manually tested (if applicable)

CHANGELOG.md

types/context.go

store/cachemulti/store.go

+func (cms Store) GetKVStore(key types.StoreKey) types.KVStore {
+	store, ok := cms.getCacheWrap(key).(types.KVStore)
+	if !ok {
+		panic(fmt.Sprintf("store with key %v is not KVStore", key))


store/rootmulti/store.go

-	store := types.KVStore(s)
+	store, ok := s.(types.KVStore)
+	if !ok {
+		panic(fmt.Sprintf("store with key %v is not KVStore", key))


mmsqe

need use storetypes.GKVStore[V] in runtime right?

store/types/store.go

yihuang · 2024-03-24T02:49:55Z

need use storetypes.GKVStore[V] in runtime right?

dependers to be changed in other PR.

store/internal/btreeadaptor.go

…in store (cosmos#19770)" This reverts commit ff30f6e.

Solution: - revert the iavl bump, v1.0.2 is the version referenced by outer packages

store/types/validity.go

generic interface generic btree generic cachekv generic transient store support ObjStore changelog Update CHANGELOG.md Signed-off-by: yihuang <huang@crypto.com> object store key Apply review suggestions fix merge conflict fix snapshot revert dependers value validation seems still need to replace store object store in api gas accounting for object store

Signed-off-by: yihuang <huang@crypto.com>

baseapp/baseapp.go

Signed-off-by: yihuang <huang@crypto.com>

store/types/validity.go

+// AssertValidValueLength checks if the value length is within length limit
+func AssertValidValueLength(l int) {
+	if l > MaxValueLength {
+		panic(errors.New("value is too large"))


Reason for Cherry-pick: - Migrate Block STM (Parallel Transaction Execution) Original Commit Message: generic interface generic btree generic cachekv generic transient store support ObjStore changelog Update CHANGELOG.md Signed-off-by: yihuang <huang@crypto.com> object store key Apply review suggestions fix merge conflict fix snapshot revert dependers Original author: yihuang <huang@crypto.com> ----------- Cherry-picked-by: zsystm <actor93kor@gmail.com>

…-org-chain#235) Reason for Cherry-pick: - Migrate Block STM (Parallel Transaction Execution) - Solve dependency problem after adding ObjectStore (crypto-org-chain#206) Original Commit Message: * Revert "build(deps): Bump github.com/cosmos/iavl from 1.0.1 to 1.1.1 in store (cosmos#19770)" This reverts commit ff30f6e. * Problem: store package has dependency issues Solution: - revert the iavl bump, v1.0.2 is the version referenced by outer packages Original author: yihuang <huang@crypto.com> ----------- Cherry-picked-by: zsystm <actor93kor@gmail.com>

github-actions bot added the C:Store label Mar 23, 2024

yihuang requested a review from mmsqe March 23, 2024 09:00

yihuang commented Mar 23, 2024

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

github-actions bot reviewed Mar 23, 2024

View reviewed changes

types/context.go Outdated Show resolved Hide resolved

yihuang changed the title ~~Problem: kv store require unnesserary encoding/decoding objects~~ Problem: transient store require unnesserary encoding/decoding objects Mar 23, 2024

yihuang changed the title ~~Problem: transient store require unnesserary encoding/decoding objects~~ Problem: duplicated parameters decoding Mar 23, 2024

yihuang changed the title ~~Problem: duplicated parameters decoding~~ Problem: duplicated parameters decoding put big pressure on memory allocations Mar 23, 2024

yihuang mentioned this pull request Mar 23, 2024

[Feature]: caching parameters and other stuff in transient object store cosmos/cosmos-sdk#19841

Closed

github-advanced-security bot found potential problems Mar 23, 2024

View reviewed changes

mmsqe reviewed Mar 24, 2024

View reviewed changes

store/types/store.go Show resolved Hide resolved

yihuang force-pushed the object-store branch from 6fbc727 to 036a782 Compare March 24, 2024 02:48

github-advanced-security bot found potential problems Mar 24, 2024

View reviewed changes

store/internal/btreeadaptor.go Fixed Show fixed Hide fixed

store/internal/btreeadaptor.go Fixed Show fixed Hide fixed

mmsqe approved these changes Mar 25, 2024

View reviewed changes

yihuang force-pushed the object-store branch 2 times, most recently from 9348ebf to 7fc4763 Compare March 25, 2024 02:09

yihuang enabled auto-merge (squash) March 25, 2024 02:09

yihuang added 2 commits March 25, 2024 11:29

Revert "build(deps): Bump github.com/cosmos/iavl from 1.0.1 to 1.1.1 …

c5e1ada

…in store (cosmos#19770)" This reverts commit ff30f6e.

Problem: store package has dependency issues

8f4e817

Solution: - revert the iavl bump, v1.0.2 is the version referenced by outer packages

github-advanced-security bot found potential problems Mar 25, 2024

View reviewed changes

store/types/validity.go Fixed Show fixed Hide fixed

yihuang force-pushed the object-store branch from f872044 to 5cf108d Compare March 25, 2024 03:40

yihuang added 2 commits March 25, 2024 11:44

fix interface

9c4a626

Merge branch 'release/v0.50.x' into object-store

2e58bbd

Signed-off-by: yihuang <huang@crypto.com>

yihuang commented Mar 25, 2024

View reviewed changes

baseapp/baseapp.go Outdated Show resolved Hide resolved

yihuang added 2 commits March 25, 2024 11:47

Update baseapp/baseapp.go

5c6f18d

Signed-off-by: yihuang <huang@crypto.com>

fix obj store

9a25b09

github-advanced-security bot found potential problems Mar 25, 2024

View reviewed changes

fix test

8592be8

bump go-version in ci

f3421db

github-actions bot added the Type: CI label Mar 25, 2024

yihuang added 2 commits March 25, 2024 13:39

fix e2e tests

a9b39e9

gofump

34a1473

yihuang mentioned this pull request Mar 25, 2024

[EPIC] block-stm integration crypto-org-chain/ethermint#432

Closed

7 tasks

yihuang added 2 commits March 25, 2024 13:46

fix lint

47c1065

fix test

7b66002

github-actions bot added the C:x/group label Mar 25, 2024

yihuang merged commit 90f3048 into crypto-org-chain:release/v0.50.x Mar 25, 2024
43 of 45 checks passed

yihuang deleted the object-store branch March 25, 2024 06:15

yihuang changed the title ~~Problem: duplicated parameters decoding put big pressure on memory allocations~~ feat: implement object store Jun 5, 2024

yihuang changed the title ~~feat: implement object store~~ feat: implement object stores Jun 5, 2024

tac0turtle mentioned this pull request Jun 12, 2024

[Feature (collections)]: decoded object cache cosmos/cosmos-sdk#19223

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement object stores #206

feat: implement object stores #206

yihuang commented Mar 23, 2024 •

edited

Loading

mmsqe left a comment

yihuang commented Mar 24, 2024

feat: implement object stores #206

feat: implement object stores #206

Conversation

yihuang commented Mar 23, 2024 • edited Loading

Description

Author Checklist

Reviewers Checklist

mmsqe left a comment

Choose a reason for hiding this comment

yihuang commented Mar 24, 2024

yihuang commented Mar 23, 2024 •

edited

Loading