Communication-Efficient Distributed Online Learning
This project provides a distributed online learning framework based on Apache Storm. The goal is to provide an easy to install and configure framework with which a stream-based online learning service can be provided. That includes a building block like structure, where a user can put together the model type, update rule, loss function, etc., either using already implemented methods, or by implementing her own methods.
The distributed online learning framework has been developed within the FERARI project (ferari-eu.org) as part of an efficient streaming complex event processing framework that allows distribution not only within a site using Apache Storm, but also over geographically distributed sites. If you are interested, check out the FERARI open source repository (https://bitbucket.org/sbothe-iais/ferari).
This repository has been just forked from the FERARI repository. We stripped off additional features that have nothing to do with distributed online learning and fitted the classes to a scenario, where the whole learning happens within one Storm topology (in FERARI, the entire system consists of many separate Storm topologies).
We also provide a few sample topologies that can be directly run after downloading the repository.
Our plans for the future
The project has been developed within the FERARI project and we want to thank the whole consortium for their great efforts. The adaption of the learning framework to a single topology mode and the implementation of many learning algorithms has been done by @rania_briq, the necessary framework parts have been developed by @Moritz Fürneisen. Thanks, guys!
Our most immediate goal is to refactor the code to fit the single topology mode better.
Our near future goals are to provide a reasonable interface for the output of the learners, so that an actual service can be set up, as well as improving the monitoring dashboard (with it you can monitor the current error and communication of the topology).
On a theoretical side, we just implemented kernel models and we will include more learning algorithms and model compression techniques for that case. Moreover, there are experiments on doing kernel outlier detection within the framework.
A rather novel idea is to replace the averaging operation of models (which is the current way of aggregating the models to a synchronized variant) with another merging operation, the Radon point. For batch learning, one can show that this is quite efficient, theoretically and practically. Experiments are currently going on that analyze this operation in online learning.
A really interesting direction for future work is to employ methods from transfer learning to cope with problems where the data does not come from the same data distribution but from different data sources. Your ideas would be very welcome!
I want to see a running example
In the examples folder, you'll find several topologies (in the topologies package) that should run out of the box - if not, I screwed with the code, in which case it would be great if you could drop me a note. I'll then try to fix it ASAP.
I want to get involved
Awesome, thank you! Feel free to contribute, refactor, and improve. If you have questions, don't hesitate to contact me Michael Kamp.
I want to use this in my research
Great! If you want to use this software in your research, please cite one of the following papers.
Kamp, Michael, et al. "Communication-efficient distributed online prediction by dynamic model synchronization." Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer Berlin Heidelberg, 2014.
Kamp, Michael, et al. "Communication-Efficient Distributed Online Learning with Kernels." Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer International Publishing, 2016.
Module core provides general interfaces that are independent of runtime adaption.
The basic components are local state which accepts messages and may contact a coordinator if needed.
The coordinator in turn can send updated model information to the instances of local state.
Of course it is completely feasible to contact the coordinator after each message to the local models.
However this is not recommended as it will become a bottleneck when processing large amounts of data in a distributed fashion.
For further information please read the Overview Slides
Every pair of local/global models expressed using the interfaces defined in core should be able to run on different runtime adaptions. At the moment, there is an adaption to Apache Storm.
Of course, we don't mind people providing more adaptations...
Maven is used for building the project and maintaining dependencies.
There are modules for core, runtime adaptions and examples.
The runtime adaptions module has a submodule for each adaption.
Examples will have a submodule for their implementations of local and global states as well as submodules deploying it to different runtime adaptions.
The copyright lies with the authors and contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.