Separate malloc hints for signal updates

[naveen-rn]

In OpenSHMEM-1.5, we have SHMEM_MALLOC_SIGNAL_REMOTE as a single hint for both the signal operation, we might want to deprecate this or provide another two constants SHMEM_MALLOC_SADD_REMOTE and SHMEM_MALLOC_SPUT_REMOTE, where SPUT and SADD is for signal PUT and signal ADD respectively.

Not sure, whether we can do this for OpenSHMEM-1.5.

[manjugv]

@naveen-rn What is the motivation to have different hints for different signals? do you plan to implement different strategies based on the hint?

[naveen-rn]

yes.

[manjugv]

@naveen-rn More information about the implementation strategies and why it requires more specific hint might help. Let's discuss this in the next working group (March 2nd).

[jamesaross]

Some thoughts for discussion...

There are two hints now which can be bitwise OR'd, resulting in 2^2 possible hints. This proposal would increase the number of hints to 2^3.

For completeness, we also have the SHMEM_MALLOC_ATOMICS_REMOTE hint for all atomic operations, so should we add the following hints as well?

SHMEM_MALLOC_AFETCH_REMOTE
SHMEM_MALLOC_ASET_REMOTE
SHMEM_MALLOC_ACOMPARE_SWAP_REMOTE
SHMEM_MALLOC_ASWAP_REMOTE
SHMEM_MALLOC_AFETCH_INC_REMOTE
SHMEM_MALLOC_AINC_REMOTE
SHMEM_MALLOC_AFETCH_ADD_REMOTE
SHMEM_MALLOC_AADD_REMOTE
SHMEM_MALLOC_AFETCH_AND_REMOTE
SHMEM_MALLOC_AAND_REMOTE
SHMEM_MALLOC_AFETCH_OR_REMOTE
SHMEM_MALLOC_AOR_REMOTE
SHMEM_MALLOC_AFETCH_XOR_REMOTE
SHMEM_MALLOC_AXOR_REMOTE
SHMEM_MALLOC_AFETCH_NBI_REMOTE
SHMEM_MALLOC_ACOMPARE_SWAP_NBI_REMOTE
SHMEM_MALLOC_ASWAP_NBI_REMOTE
SHMEM_MALLOC_AFETCH_INC_NBI_REMOTE
SHMEM_MALLOC_AFETCH_ADD_NBI_REMOTE
SHMEM_MALLOC_AFETCH_AND_NBI_REMOTE
SHMEM_MALLOC_AFETCH_OR_NBI_REMOTE
SHMEM_MALLOC_AFETCH_XOR_NBI_REMOTE

One could imagine implementing different strategies based on the hint. But, as an implementer, I'd really rather not. Combined with this proposal, the specification would then have 2^24 hints. Maybe that's too many, but is this proposal for just 2^3 hints enough? At what point has sufficient optional hint complexity been achieved?

There's no shortage of potential usage scenarios for shmem_malloc'd symmetric memory and I think we need to consider the benefits of lengthening the list of optional hints to the API, which may only confuse application developers without certain performance or semantic improvements.

Perhaps these and future optional constants--which are becoming more narrowly focused on particular implementations--would be better placed into the implementation-defined shmemx.h?

#ifdef SHMEMX_MALLOC_SADD_REMOTE
p = shmem_malloc_with_hints(size, SHMEMX_MALLOC_SADD_REMOTE);
#else
p = shmem_malloc_with_hints(size, SHMEM_MALLOC_SIGNAL_REMOTE);
#endif

[naveen-rn]

I was thinking about this same logic for sometime. Consider the following analysis:

1. You don't need different hints for atomics - because you couldn't split the atomics implementation. Atomics are atomic with respect one another. So, I'm not sure how could an/any implementation - provide such a semantic guarantee with different implementations for different atomics.

2. put-with-signal is atomic with respect to itself and its own type of operation - essentially signal_set and signal_add are separate operations. AFAIU, we just contained them inside the same AP for usability purpose, but they are different ops.

So, the reason for splitting hints for signaling is that the operation types are not tightly coupled, while AMOs are tightly coupled.

[jdinan]

In no particular order: (1) Why is this assigned to me? (2) What?! (3) Atomics can be implemented differently based on datatype (ducks for cover). (4) No, I would not recommend this as a section committee edit.