An application that wishes to use RDMA will ideally want to use multiple communication primitives to connect participating nodes, such as a ring buffer, hashmap, or replicated memory region. These abstractions might have multiple instantiations on each node, each with its own local state and RDMA registered memory regions; each of these would need to be linked to corresponding instantiations on other nodes with their own local state and RDMA memory regions. The process of establishing this link is nontrivial, as it involves setting up RDMA queue pairs and connections, registering memory regions, and exchanging permissions. In order to manage this complexity, we developed a library of RDMA communication objects, which we call LOCO. LOCO allows programmers to create classes of reusable RDMA objects that maintain state on multiple nodes in the network without worrying about linking to remote objects or handling RDMA address offsets or permissions; as such, LOCO exports a quasi- shared memory programming model on top of a distributed system.
The installation of LOCO and the comparison software is relatively simple. The first thing that needs to be done is to get an experiment started with however many nodes you need to use for your experiment. For the development of each of these protocols, experiment setups were used through Cloudlab.
See the README.md in the scripts/ directory for information on how to configure the Cloudlab experiment.
In future work we may extend the setup script to work on other environments (other than Cloudlab).
Broken Pipe at Port 22
One or more nodes are inaccessible via SSH(Port 22). Reboot/Reload might work.
- timeout during setup at
StrictHostKeyChecking
the experiment was created incorrectly, terminate and restart the experiment.
Some dependencies need to be re-enabled after rebooting.
For organization of this repo, please refer to REPO_ORG.md