Skip to content
shamisp edited this page Oct 28, 2014 · 1 revision

boostrap flow:

worker_get_address

GOAL: find boostrap transport in each "domain" and use the one with lowest latency.

  1. bootable transport holds the following:

    1. supports connect_to_iface
    2. supports short active message
  2. For each device, select bootable transport with lowest latency Assumption: if a remote device is reachable with one transport, this remote device is also reachable with other transports which run on the same local device.

  3. if a transport is selected more than once, use tl->iface_is_reachable() to eliminate devices which are on the same network.

  4. return a blob which contains records like this:

    • transport name
    • performance attributes: latency,bandwidth
    • iface address

ep_connect

GOAL: find best method to reach the destination address, for each of: short_am, long_am, rma, amo.

  1. find the boostrap method (will not neccesarily be used):
    for each local resource:
    for each remote address:
    if reachable and can-connect-to-iface:
    compute-score
    save-the-combination-which-had-best-score-amongst-which-can-connect-to-iface => "booststrap"

  2. find methods for each class:
    for "class" in [short_am, long_am, rma, amo]:
    for each local resource:
    for each remote address:
    if reachable:
    compute score based on local and remote capabilities / performance attributes.
    use-the-combination-which-has-the-best-score
    if the selected resource can connect to iface address:
    connect directly
    else
    boostrap[class] = selected-resource (local-index & remote-address)
    need-boostrap = 1

if need-boostrap:
start-boostrap

boostrap process

GOAL: exchange endpoint-specific address

  • send active message to remote-address: - my iface address is X - my ep address is Y

  • remote also sends: - my iface address is X - my ep address is Y

  • when getting a message from remote side:

    • search in hash table, key is: iface address
      • if exists: connect

ucp problems - tag matching

  • need to pass sender ep id from sender to receiver so the receiver could send replies:
    • rndv protocol
    • credits
    • cancel protocol
    • active message based protocols

options: - pass ep_id as part of the protocol for relevant messages - will it be needed for RMA/AMO emulation as well? yes - need to know to whom send - maybe could avoid TID for RMA/AMO, assuming order? - need TID+Sender only for: GET, FADD, SWAP, CSWAP

- pass uct_ep_h to AM callback
    - could be temporary handle
    - ud - already have it. rc - fast lookup. dc - need to build AH.
    - for dc - need to pass DCT number as immediate, ah could be created from LID.
    - could have "send" flag as to whether the transport should pass "sender handle" to am.
Clone this wiki locally