TOC {:toc} Note on Online Regression Algorithm Least Square Problem Classical least square linear regression is $$ \hat \beta_{ls}=\arg\min_\beta\|y-X\beta\|^2_2 $$ With regularizations it becomes a ridge or lasso regression problem
TOC {:toc} L-BFGS algorithm Motivation L-BFGS is one of the not so simple optimization algorithm that we may encounter in large scale optimization problems. Not so simple means it’s not simply a first order algorithm, and the deviation from that is well motivated by theoretical arguments. So this note target to understand this algorithm
Notes on Visual Imagery Definition: Recreate the sensory world in mind in absense of physical stimuli. Usage in daily cognition Closely related to memory. We solve some cognitive task by recreating the visual scene in mind and examine the mind picture! Some tasks are memory about spatial some are feature memory! Usage in creative work Provides another way of thinking, other than verbal and logical induction. Intuition Characteristics of Imagery Is the representation spatial or propositional?
Note on Local Feature Descriptors Before the advent of convolutional neural network, many techniques to represent and detect local features has been invented. As lower level feature detector, many of them are strongly mathematically motivated. Some are still used in some Computer Vision tasks as preprocessing step.
Note on Patch Based Shape Interpretation These are 2 related papers both employ a patch based approach to tackle shape from shading problem. Typically patches have simpler appearance, thus they are easier to collect the statistics on or fit a model on. The spirit is to find a local explanation for patches in an image. However, as there will be ambiguity in local patches, the algorithm should not over-commiting to any one of the explanation and keep the distribution of possible shapes. And then take these local shape proposals and see which can stitch together and make sense globally.
Note on Categorization and Concepts From lecture notes from Science of Behavior Configuration The relative configuration of a single elements Example: Face What defines a face? Components Essential feature Configural property Relative Invariance to many change in Stimuli
Some Computation on Sphere (Updating) Motivation Recently, in research, we encounter quite a few statistical problems on sphere. For example, Head direction tuning 3d direction of object 3d direction of body parts Some 3d tuning There are many standard statistical operations on Euclidean space, like getting mean, standard deviation and generate uniform distribution, fitting a model etc. We can perform these operation without thinking.
Note on CNN Interpretability 2 major way of interpreting CNN Feature visualization: See what a hidden neuron is interested in Attribution: See what part of image activate a filter or detector Activation Atlas These works try to find a tool kits for visualizing DeepNN and building up a human-computer interface of DeepNN.
Based on Goldstein Book Chapter and lecture from Jeff Beck Note on Forms of Memory Definition pin down can be very tricky! Definition Retaining, retrieving, using information after the original information (stimuli) does not present. (Inner view) Any process that some past experience has an effect on the way the subject think and behave in the future. (Outer View) Thus can generalize into even non-animated things! Memory of magnet Use of Memory Longterm Memory Human: Remember things relevant for life. (name, pw, birthday, info about others, address, knowledge) Ecological: cache for food, foraging location. Shorterm Memory Continuity of awareness Different forms Memory has many forms.
Note on Optimization on Manifold Manifold is locally similar to $\R^n$ flat space, but globally not. Manifold is locally homeomorphic to a Euclidean space of the same dimension. Besides there is Riemann logrithm map that connect the local vector space $T_p$ to the neighbourhood of $p$ on the manifold. Thus, many local optimization algorithm that work on flat $\R^n$ space can work the same on neighborhood of a manifold. The unique thing of working on a manifold is how to transport the local direction information in one neighborhood into another neighborhood.