adaptdl.reducer module

class adaptdl.reducer.Future(reducer, key)

Bases: object

result()

class adaptdl.reducer.Reducer(rank, replicas, root_host, root_port)

Bases: object

Simple asynchronous (all)reduce operations on python objects. Assumes all invokations to allreduce, allreduce_async, and Future.result happen in the same order across all processes.

allreduce(obj, reduce_fn=<function default_reduce_fn>)

allreduce_async(obj, reduce_fn=<function default_reduce_fn>)

broadcast(obj)

Broadcast a value from replica 0 to all other replicas. Currently uses allreduce with left-projection.

adaptdl.reducer.default_reduce_fn(a, b)