Operation Sessions Extension

[jdinan]

Problem

The OpenSHMEM library sees individual operations as they occur. If the application will perform a series of operations (e.g. a loop of puts), it can be beneficial to inform the runtime.

Proposed Solution

Allow users to provide a performance hint that they will issue a set of operations. This information can be used to optimize the behavior of the OpenSHMEM runtime. For example, sessions may allow the runtime to aggregate operations or completions.

Proposed API

Straw man API:

void shmem_ctx_start_session(shmem_ctx_t ctx);
void shmem_ctx_end_session(shmem_ctx_t ctx);

The start session operation informs the OpenSHMEM library that operations will be issued on the given context. The end session operation informs the OpenSHMEM library that the last operation in the session has been issued on the given context. Ending a session does not guarantee completion or ordering, quiet/fence on the context must still be used to achieve these guarantees.

[davidozog]

Here are the WIP slides from the RMA WG (Min's) discussion today:
https://docs.google.com/presentation/d/1rwONNPFlHMv36eLJSWm8TnGE6nrEvVwpLARaIFH1RmY/edit?usp=sharing

Key takeaways/questions for next time:

1. Do we want bundling to be more "implicit" (the OpenSHMEM library decides how to bundle operations) or "explicit" (the user says exactly which ops are bundled and/or when they are shipped).
2. Should bundles be associated with contexts?
3. Should collectives be supported for bundling?

[davidozog]

Here are the slides we covered in the committee meeting today:
https://docs.google.com/presentation/d/1Eco0_yFRS__b-kHy9dM7vO4yDw951_Gc685divy_JZs/edit?usp=sharing

I created a gist that attempts to capture the ideas that came up today. Feedback is welcome:
https://gist.github.com/davidozog/9be1e52dd425567f86ccb1b050446baf

(@minsii @jdinan @nspark @naveen-rn @manjugv)

[nspark]

Personally, I like Option 2 from your slides. You present:

shmem_ctx_t my_ctx;
shmem_bundle_start(my_ctx);

shmem_put(my_ctx, ...);
shmem_atomic_inc(my_ctx, ...);
/* ... more RMA/AMO ... */

shmem_bundle_stop(my_ctx);

The shmem_bundle*_t types seem unnecessary at the current stage of design. I'm hesitant to add structure types and fields if all we're (currently) specifying amounts to a boolean flag.

[nspark]

If something more explicit w.r.t. bundle configuration is desired (e.g., to differentiate message aggregation and NIC doorbell chaining), we may want to revisit @naveen-rn's shmem_ctx_create_ext, which took a structure-base configuration argument. If the context is the object that "does the bundling," then there shouldn't be a shmem_bundle_t type. A sketch:

shmem_ctx_config_t ctx_config = {
  .bundle = SHMEM_CTX_BUNDLE;
};
shmem_ctx_t ctx;
long ret = shmem_ctx_create_ext(SHMEM_CTX_CONFIG, SHMEM_CTX_CONFIG_BUNDLE, &config, &ctx);
if (ret | SHMEM_CTX_CONFIG_BUNDLE)
  printf("Cannot create bundle-enabled context!\n");
else
  printf("Some other error in shmem_ctx_create_ext\n");

Maybe, eventually, we'll have something like:

#define SHMEM_CTX_BUNDLE_AGGREGATE (1l << 1)
#define SHMEM_CTX_BUNDLE_CHAIN     (1l << 2)
#define SHMEM_CTX_BUNDLE (SHMEM_CTX_BUNDLE_AGGREGATE | SHMEM_CTX_BUNDLE_CHAIN)

[nspark]

If bundling itself is a hint, then it seems like the start/stop operations on a bundle-enabled context (I don't like "bundled context") should themselves be hints. An implementation that supports bundling may choose to:

• Only bundle nonblocking operations between explicit start/stop regions
• Bundle nonblocking operations eagerly

[nspark]

Re: when to bundle: Should blocking, non-fetching atomics be bundled? They are in many ways like nonblocking operations. Unlike blocking RMA, which may require a copy of the local buffer, these operations take their only payload as a value parameter. I imagine there could be meaningful benefits from NIC doorbell chaining these operations, for example; however, there's no nonblocking, non-fetching AMO operations in the API.

[naveen-rn]

The shmem_bundle*_t types seem unnecessary at the current stage of design.

@nspark I agree with this statement. AFAIU, we have two levels of bundling features - chaining and aggregation.

Chaining is entirely an one-sided semantics. Meaning, all we need is the source process to group the RMA and AMO events together. To do this, a simple session_start/stop routine is sufficient. I would prefer the following core semantics for chaining:

1. Chaining is an hint for the implementation and performed on a context
2. Chain all NBI RMA, NBI PWS, NBI AMO, and non-fetching blocking AMO operations together
3. Type of chains are implementation defined - meaning, a chain of a particular set of events (like: NBI RMA-only or NBI-AMO only), or a chain of events to a particular target, or a chain of events based on data size, or a single chain per session can be created based on the underlying network
4. All chains within a session are broken explicitly using memory ordering operations like shmem_quiet (or shmem_local_complete and shmem_pe_quiet - if available)
5. Also, session_stop operation implicitly breaks all existing chains within a context
6. Session_stop() doesn't guarantee remote completion (need more discussion) - meaning there is no explicit shmem_quiet() within a session_stop() operation

Aggregation is entirely a different beast to handle at this point. Also, aggregation is sort of a two-sided operation - meaning, the source can aggregate events together and send the aggregated message to the target and it is the responsibility of the target process to unpack the aggregated message. Though, this push-pull sort of semantics can be introduced in SHMEM - we need to discuss more on the usage models.

That said, we can easily extend the core chaining-only sessions approach to support aggregation in future.