Blog Posts

Advent of Code 2024: A Gleam Retrospective

Gleam is a rather new functional language. How does it fare in AoC? What about its scientific-computing ecosystem?

Writing Advice for Fledging Machine Learning Researchers

Some bullet-point advice regarding paper-writing I would give to my younger self.

The 'use' Expression in Gleam

How can we emulate the behavior of Python's `with` and Rust `?` in Gleam?

Volume Forms and Probability Density Functions Under Change of Variables

From elementary probability theory, it is well known that a probability density function (pdf) is not invariant under an arbitrary ...

The Invariance of the Hessian and Its Eigenvalues, Determinant, and Trace

In deep learning, the Hessian and its downstream quantities are observed to be not invariant under reparametrization. This makes the ...

Convolution of Gaussians and the Probit Integral

Gaussian distributions are very useful in Bayesian inference due to their (many!) convenient properties. In this post we take a ...

The Last Mile of Creating Publication-Ready Plots

In machine learning papers, plots are often treated as afterthought---authors often simply use the default Matplotlib style, resulting in an ...

Modern Arts of Laplace Approximations

The Laplace approximation (LA) is a simple yet powerful class of methods for approximating intractable posteriors. Yet, it is largely ...

Chentsov's Theorem

The Fisher information is often the default choice of the Riemannian metric for manifolds of probability distributions. In this post, ...

The Curvature of the Manifold of Gaussian Distributions

The Gaussian probability distribution is central in statistics and machine learning. As it turns out, by equipping the set of ...

Hessian and Curvatures in Machine Learning: A Differential-Geometric View

In machine learning, especially in neural networks, the Hessian matrix is often treated synonymously with curvatures. But, from calculus alone, ...

Optimization and Gradient Descent on Riemannian Manifolds

One of the most ubiquitous applications in the field of differential geometry is the optimization problem. In this article we ...

Minkowski's, Dirichlet's, and Two Squares Theorem

Application of Minkowski's Theorem in geometry problems, Dirichlet's Approximation Theorem, and Two Squares Theorem.

Reduced Betti number of sphere: Mayer-Vietoris Theorem

A proof of reduced homology of sphere with Mayer-Vietoris sequence.

Brouwer's Fixed Point Theorem: A Proof with Reduced Homology

A proof of special case (ball) of Brouwer's Fixed Point Theorem with Reduced Homology.

Natural Gradient Descent

Intuition and derivation of natural gradient descent.

Fisher Information Matrix

An introduction and intuition of Fisher Information Matrix.

Introduction to Annealed Importance Sampling

An introduction and implementation of Annealed Importance Sampling (AIS).

Gibbs Sampler for LDA

Implementation of Gibbs Sampler for the inference of Latent Dirichlet Allocation (LDA)

Boundary Seeking GAN

Training GAN by moving the generated samples to the decision boundary.

Least Squares GAN

2017 is the year GAN loss its logarithm. First, it was Wasserstein GAN, and now, it's LSGAN's turn.

CoGAN: Learning joint distribution with GAN

Original GAN and Conditional GAN are for learning marginal and conditional distribution of data respectively. But how can we extend ...

Wasserstein GAN implementation in TensorFlow and Pytorch

Wasserstein GAN comes with promise to stabilize GAN training and abolish mode collapse problem in GAN.

InfoGAN: unsupervised conditional GAN in TensorFlow and Pytorch

Adding Mutual Information regularization to a GAN turns out gives us a very nice effect: learning data representation and its ...

Maximizing likelihood is equivalent to minimizing KL-Divergence

We will show that doing MLE is equivalent to minimizing the KL-Divergence between the estimator and the true distribution.

Variational Autoencoder (VAE) in Pytorch

With all of those bells and whistles surrounding Pytorch, let's implement Variational Autoencoder (VAE) using it.

Generative Adversarial Networks (GAN) in Pytorch

Pytorch is a new Python Deep Learning library, derived from Torch. Contrary to Theano's and TensorFlow's symbolic operations, Pytorch uses ...

Theano for solving Partial Differential Equation problems

We all know Theano as a forefront library for Deep Learning research. However, it should be noted that Theano is ...

Linear Regression: A Bayesian Point of View

You know the drill, apply mean squared error, then descend those gradients. But, what is the intuition of that process ...

MLE vs MAP: the connection between Maximum Likelihood and Maximum A Posteriori Estimation

In this post, we will see what is the difference between Maximum Likelihood Estimation (MLE) and Maximum A Posteriori (MAP).

Conditional Generative Adversarial Nets in TensorFlow

Having seen GAN, VAE, and CVAE model, it is only proper to study the Conditional GAN model next!

KL Divergence: Forward vs Reverse?

KL Divergence is a measure of how different two probability distributions are. It is a non-symmetric distance function, and each ...

Conditional Variational Autoencoder: Intuition and Implementation

An extension to Variational Autoencoder (VAE), Conditional Variational Autoencoder (CVAE) enables us to learn a conditional distribution of our data, ...

Variational Autoencoder: Intuition and Implementation

Variational Autoencoder (VAE) (Kingma et al., 2013) is a new perspective in the autoencoding business. It views Autoencoder as a ...

Deriving Contractive Autoencoder and Implementing it in Keras

Contractive Autoencoder is more sophisticated kind of Autoencoder compared to the last post. Here, we will dissect the loss function ...

Many flavors of Autoencoder

Autoencoder is a family of methods that answers the problem of data reconstruction using neural net. There are several variation ...

Level Set Method Part II: Image Segmentation

Level Set Method is an interesting classical (pre deep learning) Computer Vision method based on Partial Differential Equation (PDE) for ...

Level Set Method Part I: Introduction

Level Set Method is an interesting classical (pre deep learning) Computer Vision method based on Partial Differential Equation (PDE) for ...

Residual Net

In this post, we will look into the record breaking convnet model of 2015: the Residual Net (ResNet).

Generative Adversarial Nets in TensorFlow

Let's try to implement Generative Adversarial Nets (GAN), first introduced by Goodfellow et al, 2014, with TensorFlow. We'll use MNIST ...

How to Use Specific Image and Description when Sharing Jekyll Post to Facebook

Normally, random subset of pictures and the site's description will be picked when we shared our Jekyll blog post URL ...

Deriving LSTM Gradient for Backpropagation

Deriving neuralnet gradient is an absolutely great exercise to understand backpropagation and computational graph better. In this post we will ...

Convnet: Implementing Maxpool Layer with Numpy

Another important building block in convnet is the pooling layer. Nowadays, the most widely used is the max pool layer. ...

Convnet: Implementing Convolution Layer with Numpy

Convnet is dominating the world of computer vision right now. What make it special of course the convolution layer, hence ...

Implementing BatchNorm in Neural Net

BatchNorm is a relatively new technique for training neural net. It gaves us a lot of relaxation when initializing the ...

Implementing Dropout in Neural Net

Dropout is one simple way to regularize a neural net model. This is one of the recent advancements in Deep ...

Beyond SGD: Gradient Descent with Momentum and Adaptive Learning Rate

There are many attempts to improve Gradient Descent: some add momentum, some add adaptive learning rate. Let's see what's out ...

Implementing Minibatch Gradient Descent for Neural Networks

Let's use Python and Numpy to implement Minibatch Gradient Descent algorithm for a simple 3-layers Neural Networks.

Paralellizing Monte Carlo Simulation in Python

Monte Carlo simulation is all about quantity. It can take a long time to complete. Here's how to speed it ...

Scrapy as a Library in Long Running Process

Scrapy is a great web crawler framework, but it's tricky to make it runs as a library in a long-running ...

Gaussian Anomaly Detection

In Frequentist and Bayesian Way

Slice Sampling

An implementation example of Slice Sampling for a special case: unimodal distribution with known inverse PDF

Rejection Sampling

Rejection is always painful, but it's for the greater good! You can sample from a complicated distribution by rejecting samples!

Metropolis-Hastings

An implementation example of Metropolis-Hastings algorithm in Python.

Gibbs Sampling

Example of Gibbs Sampling implementation in Python to sample from a Bivariate Gaussian.

Twitter Authentication with Tweepy and Flask

A tutorial on how to do Twitter OAuth authentication in Flask web application.

Deploying Wagtail App

In this post, I'll show you how to deploy our blog and how to solve some common problems when deploying ...

Developing Blog with Wagtail

My experience on building this blog using Wagtail CMS, with zero Django knowledge. Let’s code our blog!

Setting Up Wagtail Development Environment

My experience on building a blog using Wagtail CMS, with zero Django knowledge. I’ll walk you through from scratch up ...