Skip to content

Rendezvous Protocol threshold for multilane mode

Dmitry edited this page Mar 3, 2020 · 7 revisions

Assumptions:

  1. Sender & receiver are configured by same manner & using same HW (latency, registration costs, etc. on both sides are equal).
  2. Eager & RNDV protocols use different lanes
  3. Cost of transfer single message is latency + overhead

Definitions:

  • ecost/rcost – eager/rndv registration cost
  • egro/rgro – eager/rndv registration growth
  • ebw/rbw – eager/rndv bandwidth
  • eover/rover – eager/rndv overhead
  • elat/rlat – eager/rndv latency
  • etime/rtime – eager/rndv delivery time
  • d - differentiate factor (bit less 1)

bw=∑n0lanei→bandwidth

cost=∑n0mdi→registration cost

gro=∑n0mdi→registration grow

Eager protocol (zcopy):

  1. Registration (eager reg cost + size * eager reg grow)

  2. Transfer (size/bw + eager overhead + eager latency)

    EAGER:etime = ecost + size • egro + sizeebw+eover

RNDV AM protocol (zcopy):

  1. Registration on sender size (rndv reg cost + size * rndv reg grow)

  2. RTS (rndv latency + rndv overhead)

  3. RTR (rndv latency + rndv overhead)

  4. Transfer (size/bw + rndv latency + rndv overhead + rndv latency)

    AM:rtime=d(rcost + size • rgro + 4rlat + 3rover + sizerbw)

RNDV RMA protocol (get):

  1. Registration on sender size (rndv reg cost + size * rndv reg grow)

  2. RTS (rndv latency + rndv overhead)

  3. Registration on receiver size (rndv reg cost + size * rndv reg grow)

  4. Transfer (size/bw + rndv latency + rndv overhead + rndv latency)

  5. ACK (rndv latency + rndv overhead)

    RMA:rtime=d(2(rcost + size • rgro) + 4rlat + 3rover + sizerbw)

To unify RNDV formulas let’s add one more argument: rrc – receive registration cost, which is 0 for AM RNDV and 1 for RMA RNDV, then RNDV time may be calculated as:

RNDV:rtime=d((1+rrc)(rcost + size • rgro) + 4rlat + 3rover + sizerbw)

Transformations:

EAGER:etime = ecost + size • egro + sizeebw+ eover = size(egro+1ebw)+ ecost + eover

RNDV:rtime=d((1 + rrc)(rcost + size • rgro) + 4rlat + 3rover + sizerbw) = d(size•rgro(1 + rrc)+rcost(1 + rrc) + 4rlat + 3rover + sizerbw) = d(size(rgro(1 + rrc) + 1rbw) + rcost(1 + rrc) + 4rlat + 3rover) = size • d(rgro(1 + rrc) + 1rbw) + d(rcost(1 + rrc) + 4rlat + 3rover)

Have equation:

etime == rtime

size(egro + 1ebw) + ecost + eover = size • d(rgro(1 + rrc) + 1rbw) + d(rcost(1 + rrc) + 4rlat + 3rover)

size(egro + 1ebw) - size • d(rgro(1 + rrc) + 1rbw) = d(rcost(1 + rrc) + 4rlat + 3rover) - ecost - eover

size(egro + 1ebw - d(rgro(1 + rrc) + 1rbw)) = d(rcost(1 + rrc) + 4rlat + 3rover) - ecost - eover

size = d(rcost(1 + rrc) + 4rlat + 3rover) - ecost - eoveregro + 1ebw - d(rgro(1 + rrc) + 1rbw)

Clone this wiki locally