GraphLab RPC

GraphLab RPC primary design goal was to provide a convenient and easy to use asynchronous communication system between identical binaries running on different machines over a distributed network. It therefore provides MPI-like capabilities together with RPC functionality. The GraphLab distributed implementation is built on top of this RPC library.

GraphLab RPC uses extensive template meta-programming techniques to provide an IDL-free (http://en.wikipedia.org/wiki/Interface_description_language) RPC system, allowing arbitrary functions to be called on program running on remote machines (Note that all machines must be running the same binary).

For instance, this is a particularly interesting example:

#include <iostream>
#include <graphlab/rpc/dc.hpp>
#include <graphlab/rpc/dc_init_from_mpi.hpp>
using namespace graphlab;

int main(int argc, char ** argv) {
  mpi_tools::init(argc, argv);

  dc_init_param param;
  if (init_param_from_mpi(param) == false) {
    return 0;
  }
  distributed_control dc(param);
  
  if (dc.procid() == 0 && dc.numprocs() >= 2) {
    dc.remote_call(1, printf, "%d + %f = %s\n", 1, 2.0, "three");
  }
  dc.barrier();
}

Once the distributed_control object is created, dc.procid() provides the current machine number, while dc.numprocs() provide the total number of machines.

The if-condition is therefore entered by only the first machine, which performs a remote call to the second machine (the first argument of remote_call is the target machine ID). The second machine will then execute the equivalent of

  printf("%d + %f = %s\n", 1, 2.0, "three");

We will discuss the different aspects of the RPC library seperately:

• Spawning and Initialization
  Initialization and Starting a distributed program using GraphLab RPC
• Basic RPC Usage
  Basic usage of the RPC library. Calling of simple functions.
• Distributed Objects
  Advanced usage of the RPC library. Creating and managing distributed object contexts.

Examples

The tests/ directory include a collection of seven RPC examples demonstrating all the key features.

• RPC Example 1: Basic Synchronous RPC rpc_example1.cpp
• RPC Example 2: Asynchronous RPC with Built-in Serialization rpc_example2.cpp
• RPC Example 3: Asynchronous RPC with Struct POD Serialization rpc_example3.cpp
• RPC Example 4: Asynchronous RPC with Manual Serialization rpc_example4.cpp
• RPC Example 4: Asynchronous RPC to printf rpc_example5.cpp
• RPC Example 4: Asynchronous RPC with any rpc_example6.cpp
• RPC Example 4: Distributed Object rpc_example7.cpp