8 items tagged
Motivation Given the popularity and power of diffusion models, the theoretical formulation of these models are not in unison. Because multiple groups have derived these models from different background, there exist multiple formulations, SDE, ODE, Markov Chain, Non-markov chain etc.
Hippo: Recurrent Memory with Optimal Polynomial Projection Motivation Hidden state in RNN represents a form of memory of the past. For a sequence, a natural way to represent the past sequence is to project it onto an orthonormal basis set. Here depending on the different emphasis of the past, we could define different measures on the time axis and define the basis set based on this measure. Then we can keep track of the projection coefficient on this basis when observing new data points.
[TOC] Motivation S4 sequence model is rising in the sequence modelling field. It dominates on long sequence modelling over RNN, LSTM and transformers. It’s both mathematically elegant and useful, and it’s trending, so why not write about it.
Stability Theory When a system has control, then comes the questions of whether we could make it stable under the control. Problem Setup A control affine system is this $$ \dot x =f(x)+\sum_i g_i(x)\bar u_i=f(x)+g(x)\bar u $$Interpretation:
Basic Notions Def Topological Equivalence: 2 dynamic systems are topological equivalence when there is a homeomorphism between their solutions. Def Conjugate: Two maps are connected by $$g=h^{-1}\circ f \circ h$$.
Bifurcation Normal Form Invariance and Stable Manifold Lyapnov Stability Theory Feedback Stablization
Stability Theory Motivation We want to know for a dynamic system, in this note majorly autonomous system, when it is stable? The meaning of stability? If it’s stable how to prove so. Majorly we are going to use Lyapnov functions and spectral properties of linearized system to prove.
Invariance Properties of an Invariance Set Stable and Unstable Manifold Theorem