tensorflowonspark.TFManager module

class TFManager(address=None, authkey=None, serializer='pickle', ctx=None)[source]

Bases: multiprocessing.managers.BaseManager

Python multiprocessing.Manager for distributed, multi-process communication.

connect(address, authkey)[source]

Connect to a multiprocess.Manager.

Args:
address

unique address to the TFManager, either a unique connection string for ‘local’, or a (host, port) tuple for remote.

authkey

string authorization key

Returns:

A TFManager instance referencing the remote TFManager at the supplied address.

start(authkey, queues, mode='local')[source]

Create a new multiprocess.Manager (or return existing one).

Args:
authkey

string authorization key

queues

INTERNAL_USE

mode

‘local’ indicates that the manager will only be accessible from the same host, otherwise remotely accessible.

Returns:

A TFManager instance, which is also cached in local memory of the Python worker process.