Skip to content

Distributed stochastic gradient (GRPC/Kubernetes) and comparison with Hogwild.

License

Notifications You must be signed in to change notification settings

zifeo/distributed-sgd

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Distributed SDG

This project showcases a distributed implementation of the stochastic gradient descent algorithm. It introduces a synchronous and an asynchronous version. On the one hand, the synchronous version includes a master node which ensures that the computation of the gradient and the update steps of SGD are coordinated among worker nodes. On the other hand, in the asynchronous version the worker nodes perform computation on their own and frequently exchange weight updates among each other. The master only manage the start and end of the full algorithm (e.g. split the work and collect the result).

Getting started

cd data
./download.sh
cd ..
sbt
> run
> test
> scalafmt

SGD settings can be modified in src/main/resources/application.conf.

Running on Kubernetes

./build.sh
./run.sh -async
./run.sh -sync

SGD settings can be modified in kube/config-async.conf and kube/config-async.conf.

Dataset

RCV1 can be downloaded from here.

References

  • RECHT, Benjamin, RE, Christopher, WRIGHT, Stephen, et al. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In : Advances in neural information processing systems. 2011. p. 693-701.

About

Distributed stochastic gradient (GRPC/Kubernetes) and comparison with Hogwild.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published