Single server - multiple clients example

[scuser22]

Hi,

1. What is the correct approach to implement a single server, multiple clients usecase?
   For example, a single server reads data from multiple clients (N) in different nodes.

1a) Does the server needs to have N endpoints and each client has a single endpoint?

1b) Does such sample usecase available in fabtests?

Thank you in advance.

[shefty]

Welcome.

The best option sounds like you want to use MSG endpoints. Each client would have a single endpoint that it uses to connect. The server would create a passive endpoint, listen on it, and accept new connections on a separate endpoint - 1 endpoint per client.

Fabtests specific to this will have the string 'msg' in the name, though several tests support msg endpoints. You could start with functional/msg.c. The server only accepts a single client connection in this example, but the general flow is there.

Because fabtest uses a lot a common code, some of the underlying calls won't be needed as they apply to different endpoints. Specifically, you can ignore AV related calls, and the cntr APIs to start with.

[scuser22]

Thank you for the quick and detailed reply. I have a followup question.

I am thinking about utilizing the RMA capability provided by the fabric. Server can read data from clients using RMA based on some predefined strategies. For example,

1. Clients exchange address to server via oob.
2. Server adds clients address and associate an endpoint per client.
3. Performs key exchange.
4. Use the endpoint associated with client to perform RMA read.

Ideally, then the client can continue the application execution and remove the overhead associated with message transfer.
Does the following approach make sense? Does fabtest provide support for such usecases(1 to N RMA) ?

Thank you in advance.

[shefty]

Nearly all of the fabtests are a single client-server model. We have higher level middleware for more complex testing. The multinode test is an exception.

For msg endpoints, connections must be setup in-band. You could exchange address/keys for RMA out of band, as part of the connection setup, or in-band after the connection is done. In similar cases to yours, I've seen it exchanged as part of the CM data.

If you want endpoint setup to be OOB, you would need to switch to RDM endpoints. If you did that, the server could use a single endpoint to talk with all clients. Clients would provide their address to the server OOB, the server would insert it into its local address vector, and it could being transferring data. (Clients do not need to have the server's address in their address vector unless they want to send to the server). The fabtests with 'rdm' in the name support this endpoint model, and there is code for OOB address exchange.

Note that it will be difficult for the two sides to guarantee that RMA read data is available without some higher-level message exchange. And having that message exchange on a connection separate from where RMA is occurring is usually racy.