Notes

Here is a collection of miscellaneous notes:

Learning information theory with hangman. For a game of hangman with a three-letter word, in which X is the probability space of the first letter (26 elements) and Y is the probability space of the last two leters (26^2 elements), I consider various information theoretic quantities (e.g. joint entropy H(X,Y), conditional entropy H(Y|X), and the mutual information I(X;Y)). Updated 11/26/20.

Implementation of artificial boundary conditions in simulations of the 1D and 2D Fokker-Planck equations. The Fokker-Planck equation (FPE) describes the time-evolution of a probability distribution subject to deterministic drift and stochastic diffusion. Simulating this partial differential equation on a grid requires some care, especially at the boundaries, to ensure no probability “leaks out.” Here I explicitly report how to properly implement boundary conditions to ensure that no leakage occurs with matrix equations for a 1D FPE equation, and qualitatively explain how this approach may be extended to 2D (and higher-dimensional) FPE equations. Updated 10/6/20.