tensorflowonspark.reservation module

This module contains client/server methods to manage node reservations during TFCluster startup.

class Client(server_addr)[source]

Bases: tensorflowonspark.reservation.MessageSocket

Client to register and await node reservations.

Args:
server_addr

a tuple of (host, port) pointing to the Server.

await_reservations()[source]

Poll until all reservations completed, then return cluster_info.

close()[source]

Close the client socket.

get_reservations()[source]

Get current list of reservations.

register(reservation)[source]

Register reservation with server.

request_stop()[source]

Request server stop.

server_addr = None

address of server

sock = None

socket to server TCP connection

class MessageSocket[source]

Bases: object

Abstract class w/ length-prefixed socket send/receive functions.

receive(sock)[source]

Receive a message on sock.

send(sock, msg)[source]

Send msg to destination sock.

class Reservations(required)[source]

Bases: object

Thread-safe store for node reservations.

Args:
required

expected number of nodes in the cluster.

add(meta)[source]

Add a reservation.

Args:
meta

a dictonary of metadata about a node

done()[source]

Returns True if the required number of reservations have been fulfilled.

get()[source]

Get the list of current reservations.

remaining()[source]

Get a count of remaining/unfulfilled reservations.

class Server(count)[source]

Bases: tensorflowonspark.reservation.MessageSocket

Simple socket server with length-prefixed pickle messages.

Args:
count

expected number of nodes in the cluster.

await_reservations(sc, status={}, timeout=600)[source]

Block until all reservations are received.

done = False

boolean indicating if server should be shutdown.

get_server_ip()[source]

Returns the value of TFOS_SERVER_HOST environment variable (if set), otherwise defaults to current host/IP.

get_server_ports()[source]

Returns a list of target ports as defined in the TFOS_SERVER_PORT environment (if set), otherwise defaults to 0 (any port).

TFOS_SERVER_PORT should be either a single port number or a range, e.g. ‘8888’ or ‘9997-9999’

reservations = None

List of reservations managed by this server.

start()[source]

Start listener in a background thread

Returns:

address of the Server as a tuple of (host, port)

start_listening_socket()[source]

Starts the registration server socket listener.

stop()[source]

Stop the Server’s socket listener.