Note on Computation by Biological Plausible Learning
from lecture of Cengiz Penleven 2019
Philosophy
Neural dynamics can be a substrate of computation. The neural dynamics and plasticity dynamics can both do optimization, and the biological constraint form a source of constraint on variables.
For a dynamic system, if it has an Lyapnov function, then the dynamics can do some mimization / maximization.
The biological plausibility means the plascity happens on local site, requires information that are only local.
PCA from Hebbian Learning
Simplest implementation of Hebbian Learning is just as
\[W_{ij}\leftarrow W_{ij}+y_ix_j\]Essentially, it’s doing