-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Semantics of MPI Rank Parameters #138
Comments
Note that as of this writing, #87 is rather narrow; a new version with a wider intent (to clarify where, throughout the standard, null processes are permitted) will be be written shortly and the issue updated accordingly. |
We should give serious consideration to the idea of adding a new MPI type:
|
I agree with that.
Martin Ruefenacht and I also discussed yesterday.
I propose
MPI_Rank be added.
MPI_Rank === C int for MPI-4.
Therefore, these benefits accrue
0) Clarifies it as a signed integer; with < 0 values reserved for MPI usage
[such as MPI_PROC_NULL]
1) No loss of backward compatibility with any existing MPI codes
2) We advise people to write new codes using MPI_Rank
3) In some years, Forum willing, we deprecate the syntax of using int
4) Some more years after that, Forum willing, we allow MPI_Rank to be >= C
int
Its a slow but steady path
I suggest we also offer the manifest constant in MPI 4 called MPI_MAX_RANK;
which is 2^31 -1 in my opinion on most MPI's,
but OK, why not have it.
What is more, we need the actual type for more modern language bindings :-)
Regards,
Tony
…On Tue, Jun 4, 2019 at 6:31 AM Dan Holmes ***@***.***> wrote:
We should give serious consideration to the idea of adding a new MPI type:
MPI_Rank.
// possible mpi.h extract
#define MPI_Rank uint64_t;
#define MPI_PROC_NULL (2**63-1);
int MPI_Send(..., MPI_Rank dest, ...);
// possible alternative mpi.h extract
#define MPI_Rank int;
#define MPI_PROC_NULL (-1);
int MPI_Send(..., MPI_Rank dest, ...);
main() {
MPI_Rank myRank, wSize;
MPI_Init();
MPI_Comm_rank(MPI_COMM_WORLD, myRank);
MPI_Comm_size(MPI_COMM_WORLD, wSize);
if ((MPI_Rank)0 == myRank) {
for (MPI_Rank r = (MPI_Rank)1; r < wSize; ++r) {
MPI_Recv(..., r, ...);
}
} else {
MPI_Send(..., (MPI_Rank)0, ...);
}
MPI_Finalize();
}
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#138?email_source=notifications&email_token=AAG7ZCMS2RHCUU24UF5VLR3PYY77LA5CNFSM4HSYYUIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW4EOZA#issuecomment-498616164>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAG7ZCNKVHIORTB3XKB2VELPYY77LANCNFSM4HSYYUIA>
.
--
Anthony Skjellum, PhD
skjellum@gmail.com
Cell: +1-205-807-4968
|
@tonyskjellum
Q: if I use MPI_COMM_CONNECT/MPI_COMM_ACCEPT to repeatedly add new MPI jobs to my universe and I keep merging the intercomms into intracomms forcing MPI to find bigger numbers for rank, what happens when I force MPI to exceed MPI_MAX_RANK/MPI_MAX_COMM_SIZE? Is it an error of class MPI_ERR_MAX_RANK/MPI_ERR_MAX_COMM_SIZE? Does MPI_MAX_RANK/MPI_MAX_COMM_SIZE depend on how much memory MPI has already used? |
If your goal is to have Remember: one of the key things of making the C11 For example, here's the "simple" case of C++ function overloading for MPI_Accumulate (even abiding by the "all 'count' params will be the same type" restriction): MPI_Accumulate(const void *origin_addr, int origin_count, MPI_Datatype origin_datatype, int target_rank, MPI_Aint target_disp, int target_count, MPI_Datatype target_datatype, MPI_Op op, MPI_Win win);
MPI_Accumulate(const void *origin_addr, MPI_Count origin_count, MPI_Datatype origin_datatype, int target_rank, MPI_Aint target_disp, MPI_Count target_count, MPI_Datatype target_datatype, MPI_Op op, MPI_Win win);
MPI_Accumulate(const void *origin_addr, int origin_count, MPI_Datatype origin_datatype, MPI_Rank target_rank, MPI_Aint target_disp, int target_count, MPI_Datatype target_datatype, MPI_Op op, MPI_Win win);
MPI_Accumulate(const void *origin_addr, MPI_Count origin_count, MPI_Datatype origin_datatype, MPI_Rank target_rank, MPI_Aint target_disp, MPI_Count target_count, MPI_Datatype target_datatype, MPI_Op op, MPI_Win win); Things get dicey with C11 |
@jsquyres this is precisely my reason for bring up this idea (again, this is not its first outing/rodeo). This is the combinatorial problem that drove us towards the function pointers interface. This is also the only opportunity we have to say "new binding or old binding; make your choice". Old = like now. New = MPI_COUNT and MPI_RANK for all appropriate parameters. No mixing. That is, we only support two backend signatures: your first one and your last one. |
Also, anything that currently takes int but is actually asking for a displacement or an offset should be upgraded to MPI_AINT in the same move. Still one new backend signature. |
@jsquyres it's possible to nest void foo_int_int(int X, int Y);
void foo_int_long(int X, long Y);
void foo_long_int(long X, int Y);
void foo_long_long(long X, long Y);
#define foo_long(X, Y) _Generic((Y), \
long: foo_long_long, \
int: foo_long_int, \
default: foo_long_int \
)(X, Y)
#define foo_int(X, Y) _Generic((Y), \
long: foo_int_long, \
int: foo_int_int, \
default: foo_int_int \
)(X, Y)
#define foo(X, Y) _Generic((X), \
long: foo_long(X, Y), \
int: foo_int(X, Y), \
default: foo_int(X, Y) \
) |
Also note that that whether MPI should have |
I’m flexible ... do you think it is not int now in most MPIs? :-)
Anthony Skjellum, PhD
205-807-4968
… On Jun 4, 2019, at 9:36 AM, Jeff Squyres ***@***.***> wrote:
If your goal is to have sizeof(MPI_Rank) > sizeof(int) someday, then if #137 (BigCount) is accepted into MPI-4, you're going to have a nightmare of overloaded bindings -- particularly in C.
Remember: one of the key things of making the C11 _Generic workable is that all "count" arguments will be int or all "count" arguments will be MPI_Count -- we're not supporting multiple types of "count" argument in the same function call. If you need to overload another parameter type -- MPI_Rank, you're making a nightmare for C11 _Generic, and indeed, you're even making life complicated for Fortran and C++.
For example, here's the "simple" case of C++ function overloading for MPI_Accumulate (even abiding by the "all 'count' params will be the same type" restriction):
MPI_Accumulate(const void *origin_addr, int origin_count, MPI_Datatype origin_datatype, int target_rank, MPI_Aint target_disp, int target_count, MPI_Datatype target_datatype, MPI_Op op, MPI_Win win);
MPI_Accumulate(const void *origin_addr, MPI_Count origin_count, MPI_Datatype origin_datatype, int target_rank, MPI_Aint target_disp, MPI_Count target_count, MPI_Datatype target_datatype, MPI_Op op, MPI_Win win);
MPI_Accumulate(const void *origin_addr, int origin_count, MPI_Datatype origin_datatype, MPI_Rank target_rank, MPI_Aint target_disp, int target_count, MPI_Datatype target_datatype, MPI_Op op, MPI_Win win);
MPI_Accumulate(const void *origin_addr, MPI_Count origin_count, MPI_Datatype origin_datatype, MPI_Rank target_rank, MPI_Aint target_disp, MPI_Count target_count, MPI_Datatype target_datatype, MPI_Op op, MPI_Win win);
Things get dicey with C11 _Generic -- I don't know exactly what that would look like, but I suspect you would have to nest them...? 🤷♂
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
@jsquyres This is one of the reasons it might just be time to break API. Leave MPI-4.x alone and define all new functions going forward as using MPI_Rank for the rank and MPI_Count for counts. Anything else is a hack at best. |
We decided MPI_Count was bigger or same as MPI_Aint ... do I had just used count for both kinds .. Martin lies MPI_Displ or such as a synonym either for MPI_Aint or MPI_Count
Anthony Skjellum, PhD
205-807-4968
… On Jun 4, 2019, at 9:44 AM, Dan Holmes ***@***.***> wrote:
Also, anything that currently takes int but is actually asking for a displacement or an offset should be upgraded to MPI_AINT in the same move. Still one new backend signature.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
We are nowhere near needing to have a rank that is larger than |
@jeffhammond I can see the utility of specifying a rank type separately from some general integer from an API and application maintainability perspective, but I entirely agree that should be discussed in a different ticket (especially since it will undoubtedly prove to be incredibly controversial). I doubt anyone is seriously arguing that we need support for 64-bit ranks, other than from the perspective of being 64-bit clean. Even the cluster with the most cores (Sunway TiahuLight) has several orders of magnitude less cores than supported by 32-bit signed integers. Of course, |
I don't really have an opinion here on My understanding of the rationale of this proposal is twofold:
|
In any case, discussion on |
Problem
MPI variously defines 'rank' parameters (e.g. source, destination, root, and target) as 'integer' or 'non-negative integer', occasionally using both within the same chapter. This has no readily-apparent reason and the definition should be unified, perhaps within the context of the Semantic Terms section.
Chapters describing ranks as 'integer':
Chapters describing ranks as 'non-negative integer':
Chapters using both descriptions:
The above isn't meant to be a comprehensive list, but does adequately exhibit the problem.
Proposal
Define 'rank' as an integer in the range 0 to N, where N is the size of a group or communicator. Should #87 be accepted, this definition can be expanded to include
MPI_PROC_NULL
.Changes to the Text
The Semantic Terms section will need to be updated with the definition for 'rank' and all current descriptions of 'rank' parameters will be changed correspondingly.
Impact on Implementations
None
Impact on Users
None; it may be a bit more clear what values are permitted for rank parameters.
Alternatives
More conservatively, either 'integer' or 'non-negative integer' may be chosen as the definitive description of the type of a rank parameter and used throughout the standard. I recommend 'integer', since
MPI_PROC_NULL
andMPI_ANY_SOURCE
are commonly implemented as negative integers.The text was updated successfully, but these errors were encountered: