graphlab::distributed_control Class Reference [GraphLab RPC]

#include <dc.hpp>

List of all members.

Classes
struct	function_call_block
Public Member Functions
	distributed_control (dc_init_param initparam)
	distributed_control (const std::vector< std::string > &machines, const std::string &initstring, procid_t curmachineid, size_t numhandlerthreads=DEFAULT_NUMHANDLERTHREADS, dc_comm_type commtype=DEFAULT_COMMTYPE)
procid_t	procid () const
	returns the id of the current processor
procid_t	numprocs () const
	returns the number of processors in total.
	BOOST_PP_REPEAT (6, RPC_INTERFACE_GENERATOR,(remote_call, dc_impl::remote_call_issue, STANDARD_CALL)) BOOST_PP_REPEAT(6
FAST_CALL	BOOST_PP_REPEAT (6, RPC_INTERFACE_GENERATOR,(control_call, dc_impl::remote_call_issue,(FAST_CALL|CONTROL_PACKET))) BOOST_PP_REPEAT(6
FAST_CALL dc_impl::function_ret_type < __GLRPC_FRESULT >::type STANDARD_CALL	BOOST_PP_REPEAT (6, REQUEST_INTERFACE_GENERATOR,(typename dc_impl::function_ret_type< __GLRPC_FRESULT >::type fast_remote_request, dc_impl::remote_request_issue, FAST_CALL)) BOOST_PP_REPEAT(6
FAST_CALL dc_impl::function_ret_type < __GLRPC_FRESULT >::type STANDARD_CALL dc_impl::function_ret_type < __GLRPC_FRESULT >::type FAST_CALL CONTROL_PACKET size_t	calls_sent () const
size_t	calls_received () const
size_t	bytes_sent () const
size_t	network_bytes_sent () const
size_t	bytes_received () const
bool	is_master_rank () const
procid_t	master_rank () const
template<typename F , F f>
void	register_rpc (std::string c)
void	comm_barrier (procid_t targetmachine)
void	comm_barrier ()
long long int	get_total_bytes_sent ()
template<typename U >
void	send_to (procid_t target, U &t, bool control=false)
template<typename U >
void	recv_from (procid_t source, U &t, bool control=false)
template<typename U >
void	broadcast (U &data, bool originator, bool control=false)
template<typename U >
void	gather (std::vector< U > &data, procid_t sendto, bool control=false)
template<typename U >
void	all_gather (std::vector< U > &data, bool control=false)
template<typename U >
void	gather_partition (const std::vector< U > &local_contribution, std::vector< std::vector< U > > &ret_partition, bool control=false)
void	barrier ()
void	full_barrier ()
std::map< std::string, size_t >	gather_statistics ()
void	fill_metrics ()
metrics	get_metrics ()
void	reset_metrics ()
void	report_metrics (imetrics_reporter &reporter)
Static Public Member Functions
static unsigned char	set_sequentialization_key (unsigned char newkey)
static unsigned char	new_sequentialization_key ()
static unsigned char	get_sequentialization_key ()
	gets the current sequentialization key. This function is not generally useful.
Public Attributes
	RPC_INTERFACE_GENERATOR
	fast_remote_call
FAST_CALL	REQUEST_INTERFACE_GENERATOR
FAST_CALL dc_impl::function_ret_type < __GLRPC_FRESULT >::type	remote_request
FAST_CALL dc_impl::function_ret_type < __GLRPC_FRESULT >::type STANDARD_CALL	REQUEST_INTERFACE_GENERATOR
FAST_CALL dc_impl::function_ret_type < __GLRPC_FRESULT >::type STANDARD_CALL dc_impl::function_ret_type < __GLRPC_FRESULT >::type	control_request
long long int	total_bytes_sent
Friends
class	dc_dist_object
	the callback given to the comms class. Called when data is inbound
void	dc_recv_callback (void *tag, procid_t src, const char *buf, size_t len)
	the callback given to the comms class. Called when data is inbound

---

Detailed Description

graphlab::distributed_control is the primary distributed RPC object. This class initializes distributed communication, as well as provide basic RPC routines and collective operations.

In addition to the documented functions, the following RPC routines are provided.

void distributed_control::remote_call(procid_t targetmachine, function, ...)

remote_call performs a non-blocking RPC call to the target machine to run the provided function pointer. All arguments are transmitted by value and must be serializable.

• targetmachine: The ID of the machine to run the function on
• function: The function to run on the target machine

void distributed_control::fast_remote_call(procid_t targetmachine, function, ...)

fast_remote_call is the same as remote_call, but the receiver completes the function call using the receiving thread instead of a thread pool. This should only be used if the function is short and does not block.

• targetmachine: The ID of the machine to run the function on
• function: The function to run on the target machine

void distributed_control::control_call(procid_t targetmachine, function, ...)

Same as remote_call, but calls performed using the control_call do not contribute to the call counter and has no effect on graphlab::async_consensus.

• targetmachine: The ID of the machine to run the function on
• function: The function to run on the target machine

RetType distributed_control::remote_request(procid_t targetmachine, function, ...)

remote_request performs a blocking RPC call to the target machine to run the provided function pointer. All arguments are transmitted by value and must be serializable. This call only returns when the target machine completes the function call. The return value of the call is serialized and returned.

• targetmachine: The ID of the machine to run the function on
• function: The function to run on the target machine

RetType distributed_control::fast_remote_request(procid_t targetmachine, function, ...)

fast_remote_request is the same as remote_request, but the receiver completes the function call using the receiving thread instead of a thread pool. This should only be used if the function is short and does not block.

• targetmachine: The ID of the machine to run the function on
• function: The function to run on the target machine

void distributed_control::control_request(procid_t targetmachine, function, ...)

Same as remote_request, but calls performed using the control_request do not contribute to the call counter and has no effect on graphlab::async_consensus.

• targetmachine: The ID of the machine to run the function on
• function: The function to run on the target machine

Definition at line 152 of file dc.hpp.

---

Member Function Documentation

template<typename U >

void graphlab::distributed_control::all_gather	(	std::vector< U > &	data,
		bool	control = false
	)			[inline]

Each machine creates a vector 'data' with size equivalent to the number of machines. Each machine then fills the entry data[procid()] with information that it wishes to communicate. After calling all_gather(), all machines will return with identical vectors 'data', where data[i] contains the information machine i stored.

Definition at line 779 of file dc.hpp.

void graphlab::distributed_control::barrier

(

)

A regular barrier equivalent to MPI_Barrier. A machine entering this barrier will wait until every machine reaches this barrier before continuing. Only one thread from each machine should call the barrier.

See also:

full_barrier

template<typename U >

void graphlab::distributed_control::broadcast	(	U &	data,
		bool	originator,
		bool	control = false
	)			[inline]

This function allows one machine to broadcasts a variable to all machines.

The originator calls broadcast with data provided in in 'data' and originator set to true. All other callers call with originator set to false.

The originator will then return 'data'. All other machines will receive the originator's transmission in the "data" parameter.

This call is guaranteed to have barrier-like behavior. That is to say, this call will block until all machines enter the broadcast function.

Note:

Behavior is undefined if more than one machine calls broadcast with originator set to true.

Behavior is undefined if multiple threads on the same machine call broadcast simultaneously. If multiple-thread broadcast is necessary, each thread should use its own instance of the services class.

Definition at line 769 of file dc.hpp.

void graphlab::distributed_control::comm_barrier

(

procid_t

targetmachine

)

This comm barrier is not a true "barrier" but is essentially a sequentialization point. It guarantees that all calls from this machine to the target machine performed before the comm_barrier() call are completed before any call sent after the comm barrier() call.

void graphlab::distributed_control::comm_barrier

(

)

This is a convenience function which broadcasts a comm_barrier()

Note:

having all machines call the comm barrier does not guarantee that all calls have been processed. Basically 'p' local barriers do not result in a global barrier.

void graphlab::distributed_control::fill_metrics

(

)

Fills metrics information. All machines must call this function simultaneously. Only proc 0 will have metrics

void graphlab::distributed_control::full_barrier

(

)

Similar to the barrier(), but provides additional guarantees that all calls issued prior to this barrier are completed before returning.

Note:

This function could return prematurely if other threads are still issuing function calls since we cannot differentiate between calls issued before the barrier and calls issued while the barrier is being evaluated. Therefore, when used in a multithreaded scenario, the user must ensure that all other threads which may perform operations using this object are stopped before the full barrier is initated.

See also:

barrier

template<typename U >

void graphlab::distributed_control::gather	(	std::vector< U > &	data,
		procid_t	sendto,
		bool	control = false
	)			[inline]

Collects information contributed by each machine onto one machine. data must be of length data[numprocs]. My data is stored in data[dc.procid()]. when function returns, machine sendto will have the complete vector where data[i] is the data contributed by machine i. All machines must have the same parameter for "sendto"

Definition at line 774 of file dc.hpp.

template<typename U >

void graphlab::distributed_control::gather_partition	(	const std::vector< U > &	local_contribution,
		std::vector< std::vector< U > > &	ret_partition,
		bool	control = false
	)			[inline]

This function is takes a vector of local elements T which must be comparable and constructs a vector of length numprocs where each element is a subset of the local contribution from that machine and the union of all elements in the union of all local contributions and all entries are unique:

Usage: Each process reads the files that are stored locally and wants to know which subset of local files to read even when multiple processes see the same files.

Definition at line 784 of file dc.hpp.

std::map<std::string, size_t> graphlab::distributed_control::gather_statistics

(

)

Gather RPC statistics. All machines must call this function at the same time. However, only proc 0 will return values

metrics graphlab::distributed_control::get_metrics

(

)

[inline]

returns metrics information collected by fill_metrics Only proc 0 will have metrics

Definition at line 726 of file dc.hpp.

bool graphlab::distributed_control::is_master_rank

(

)

const [inline]

Returns true if this is the process with the lowest ID currently running on this machine in this working directory

Definition at line 470 of file dc.hpp.

procid_t graphlab::distributed_control::master_rank

(

)

const [inline]

Returns the lowest ID of all the processes currently running on this machine in this working directory

Definition at line 479 of file dc.hpp.

static unsigned char graphlab::distributed_control::new_sequentialization_key

(

)

[static]

Creates a new sequentialization key, returning the old value. All remote calls/remote requests made by the current thread will be sequentially processed by the remote machines.

Essentially all RPC calls made using the same key value will sequentialize. However, since new_sequentialization_key() uses a very naive key selection system, we recommend the use of set_sequentialization_key() especially in the case of multi-threaded code.

User should oldval = new_sequentialization_key() ... ... do stuff ... set_sequentialization_key(oldval) All RPC calls in while the key is set will be sequentialized on the receiving machine.

template<typename U >

void graphlab::distributed_control::recv_from	(	procid_t	source,
		U &	t,
		bool	control = false
	)			[inline]

A blocking recv_from. Must be matched with a send_to call from the target before both source and target resumes.

Definition at line 764 of file dc.hpp.

template<typename F , F f>

void graphlab::distributed_control::register_rpc

(

std::string

c

)

[inline]

registers a portable RPC call.

Definition at line 487 of file dc.hpp.

void graphlab::distributed_control::report_metrics

(

imetrics_reporter &

reporter

)

[inline]

Dumps the metric information to a reporter Only proc 0 will have metrics

Definition at line 737 of file dc.hpp.

template<typename U >

void graphlab::distributed_control::send_to	(	procid_t	target,
		U &	t,
		bool	control = false
	)			[inline]

This is a blocking send_to. It send an object T to the target machine, but waits for the target machine to call recv_from before returning. Functionally similar to MPI's matched sending/receiving

Definition at line 759 of file dc.hpp.

static unsigned char graphlab::distributed_control::set_sequentialization_key

(

unsigned char

newkey

)

[static]

Sets the sequentialization key to the new value, returning the previous value. When the key is set to an arbitrary non-zero value, all remote calls/remote requests made by the current thread will be sequentially processed by the remote machines.

All RPC calls made using the same key value will sequentialize.

User should oldval = set_sequentialization_key(newval) ... ... do stuff ... set_sequentialization_key(oldval)

---

The documentation for this class was generated from the following file:

• src/graphlab/rpc/dc.hpp