Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UCT/RDMACM: fix uct_rdmacm_cm_cqs hash key #6495

Merged
merged 1 commit into from
Mar 16, 2021

Conversation

evgeny-leksikov
Copy link
Contributor

@evgeny-leksikov evgeny-leksikov commented Mar 15, 2021

What

fix uct_rdmacm_cm_cqs hash key

Why ?

pd->handle is always 0 in container

How ?

make safer key

@@ -192,6 +192,11 @@ static ucs_status_t uct_rdmacm_cm_create_dummy_qp(struct rdma_cm_id *id,
return UCS_OK;
}

static uint64_t uct_rdamcm_pd_get_key(const struct ibv_pd *pd)
{
return (uintptr_t)pd ^ pd->handle;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. can you pls add comment about container?
  2. maybe the key should be both pd and handle? and not a xor of them? there is some chance that pd1 and pd2 will be considered same because xor value would be the same. so need to make sure each of them is also compared.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean 86-bit key? maybe just set (or xor) the handle to hight 32 bits of 64-bit key initialized by the pd ptr?

return (uintptr_t)pd ^ ((uint64_t)pd->handle << 32);

or

uint64_t key64;
uint32_t *key32_p = &key64;

key32_p[0] = pd->handle;
key32_p[1] = (uint32_t)(uintptr_t)pd;
return key64;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, i mean a struct of {pd, handle}
if we use xor, or reduce the pointer to 32 bit, it's not a 100% guarantee that we get the right PD, so could end up using CQ object with wrong PD

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just to clarify - should not pd->handle guarantee this in normal case? I cant find any mention in docs usage examples or purposes of this handle. If yes, then it does not makes sence to me otherwise we can even do simpler:

return pd->handle ? pd->handle : pd;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use GUID as key:
ibv_get_device_guid(struct ibv_device *device);

@yosefe
Copy link
Contributor

yosefe commented Mar 15, 2021

bot:pipe:retest

@brminich
Copy link
Contributor

@avildema, can we check some simple CS test inside container in CI somehow?

@yosefe yosefe merged commit 1eac449 into openucx:master Mar 16, 2021
changchengx added a commit to changchengx/ucx that referenced this pull request Jan 24, 2022
Backport from master branch:
    openucx#6495

Signed-off-by: Evgeny Leksikov <evgenylek@mellanox.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants