Note on Computation by Biological Plausible Learning

from lecture of Cengiz Penleven 2019

Philosophy

Neural dynamics can be a substrate of computation. The neural dynamics and plasticity dynamics can both do optimization, and the biological constraint form a source of constraint on variables.

For a dynamic system, if it has an Lyapnov function, then the dynamics can do some mimization / maximization.

The biological plausibility means the plascity happens on local site, requires information that are only local.

PCA from Hebbian Learning

Simplest implementation of Hebbian Learning is just as

$$ W_{ij}\leftarrow W_{ij}+y_ix_j $$

Essentially, it’s doing

multi-PC PCA from Hebbian anti-Hebbian Learning

Similarity Matching (classical MDS) from Hebbian anti-Hebbian Learning

Sparse coding