Dataflow matrix machines as generalized recurrent neural networks

[anhinga_anhinga]

A year ago I posted about dataflow programming and linear models of computation:

http://anhinga-anhinga.livejournal.com/82757.html

It turns out that those dataflow matrix machines are a fairly powerful generalization of recurrent neural networks.

Comments

[anhinga_anhinga]

> Do you have a simple hint of what is that matrix transformation which would apply the neuron functions to appropriate data and produce a new computational matrix. Or, may be an example?

Currently, we use a very simple form of Self. It has two arguments, and on the up movement it simply
adds them together. Its output is connected to one of its own inputs with weight 1, so this allows it to act as an accumulator (to keep the computed value through time).

The other input of Self accepts additive contributions from other neurons. So on the down movement, those additive contributions are added together, and on the up movement they are added to the previously existing matrix.

***

A couple of simple examples are in the Appendix D.2 on the page 6 of https://arxiv.org/abs/1610.00831

The second of those examples is also implemented in here: https://github.com/jsa-aerial/DMM/tree/master/examples/dmm