Motivation Given the popularity and power of diffusion models, the theoretical formulation of these models are not in unison. Because multiple groups have derived these models from different background, there exist multiple formulations, SDE, ODE, Markov Chain, Non-markov chain etc.
Jan 16, 2023
Hippo: Recurrent Memory with Optimal Polynomial Projection Motivation Hidden state in RNN represents a form of memory of the past. For a sequence, a natural way to represent the past sequence is to project it onto an orthonormal basis set. Here depending on the different emphasis of the past, we could define different measures on the time axis and define the basis set based on this measure. Then we can keep track of the projection coefficient on this basis when observing new data points.
Jul 25, 2022
[TOC] Motivation S4 sequence model is rising in the sequence modelling field. It dominates on long sequence modelling over RNN, LSTM and transformers. It’s both mathematically elegant and useful, and it’s trending, so why not write about it.
Jul 17, 2022
Stability Theory When a system has control, then comes the questions of whether we could make it stable under the control. Problem Setup A control affine system is this $$ \dot x =f(x)+\sum_i g_i(x)\bar u_i=f(x)+g(x)\bar u $$Interpretation:
May 10, 2021
Basic Notions Def Topological Equivalence: 2 dynamic systems are topological equivalence when there is a homeomorphism between their solutions. Def Conjugate: Two maps are connected by $$g=h^{-1}\circ f \circ h$$.
May 9, 2021
Bifurcation Normal Form Invariance and Stable Manifold Lyapnov Stability Theory Feedback Stablization
Apr 9, 2021
Stability Theory Motivation We want to know for a dynamic system, in this note majorly autonomous system, when it is stable? The meaning of stability? If it’s stable how to prove so. Majorly we are going to use Lyapnov functions and spectral properties of linearized system to prove.
Apr 9, 2021
Invariance Properties of an Invariance Set Stable and Unstable Manifold Theorem
Apr 9, 2021