Motivations Many CNN models have become the bread and butter in modern deep learning pipeline. Here I’m summarizing some famous CNN structure and their key innovations as I use them.
Jan 17, 2021
Note on Neural Tuning and Information Given a stimuli with $D$ intrinsic dimensions, we consider how one neuron or a population of neurons is informative about this stimulus space. Specific Information (Mutual Information) Setup for specific information computation is easy given a certain response $r$ , compute the reduction of entropy of stimuli $\mathbb s$ .
Aug 24, 2020
Note on MiniMax (Updating) Motivation This is a very traditional way of solving turn based game like chess or tic-tac-toc. It’s climax is Deep Blue AI in playing chess. Note, some people think about GAN training procedure as a min-max game between G and D, which is also interesting.
May 18, 2020
Notes on Cortical Waves Methods
May 6, 2020
Reinforcement Learning deals with environment and rewards. Agents have a set of actions to interact with environment (state $s_i$), and the environment will be changed by these actions $a_j$, from time to time, there will be reward coming from environment!
Mar 20, 2020
Note on Photometric Reasoning Shape $\hat n$, lighting $l$, reflectance $\rho$ affect image appearance $I$. Can we infer them back? $$ I=\rho<\hat n,l> $$ How much does shading and photometric effects tell us about shape, in natural settings.
Mar 17, 2020
Note on GAN Note with reference to the Youtube lecture series Hongyi Li. Architecture Developments Self Attention Used in Self-Attention GAN and BigGAN class Self_Attn(nn.Module): """ Self attention Layer""" def __init__(self,in_dim,activation): super(Self_Attn,self).__init__() self.chanel_in = in_dim self.activation = activation self.query_conv = nn.Conv2d(in_channels = in_dim , out_channels = in_dim//8 , kernel_size= 1) self.key_conv = nn.Conv2d(in_channels = in_dim , out_channels = in_dim//8 , kernel_size= 1) self.value_conv = nn.Conv2d(in_channels = in_dim , out_channels = in_dim , kernel_size= 1) self.gamma = nn.Parameter(torch.zeros(1)) self.softmax = nn.Softmax(dim=-1) # def forward(self,x): """ inputs : x : input feature maps( B X C X W X H) returns : out : self attention value + input feature attention: B X N X N (N is Width*Height) """ m_batchsize,C,width ,height = x.size() proj_query = self.query_conv(x).view(m_batchsize,-1,width*height).permute(0,2,1) # B X CX(N) proj_key = self.key_conv(x).view(m_batchsize,-1,width*height) # B X C x (*W*H) energy = torch.bmm(proj_query,proj_key) # transpose check attention = self.softmax(energy) # BX (N) X (N) proj_value = self.value_conv(x).view(m_batchsize,-1,width*height) # B X C X N out = torch.bmm(proj_value,attention.permute(0,2,1) ) out = out.view(m_batchsize,C,width,height) out = self.gamma*out + x return out,attention Style GAN BigGAN Conditional GAN Text Conditioning Text is processed and combined with noise vector.
Mar 7, 2020
Note on Hardware Based Computational Photography Now we have far more computational power than before! Besides, many images will go through complex algorithms as postprocessing. But we can also optimize camera measurement, so that results look even better.
Feb 27, 2020
Computational Photography TOC {:toc} Basically, enhance image by computation! Intersection of 3 fields Optics Vision Graphics Majorly two kinds of work Co-design camera and image processing (optics + vision) Use Vision to help Graphics to help generate better image faster! CG2REAL CG rendering is very computational intensive!
Feb 27, 2020
Motivation This is a brief analytical note about how physical self movement of eye / camera will induce optic flow in a static environment. And then discuss how a system can separate these two components instantaneously.
Feb 17, 2020