The structure tensor is the local covariance matrix field of the gradient vector field. It encodes the local anisotropy of an image. At the heart of anisotropic filtering and corner detection. en.wikipedia.org/wiki/Structure…

Oldies but goldies: Carl de Boor, On calculating with B-splines, 1971. Introduced (with Cox) an efficient algorithm for evaluating B-splines, which revolutionized Computer Aided Geometric Design. en.wikipedia.org/wiki/De_Boor%2…

The Douglas-Rachford algorithm (the dual of ADMM) computes the projection on the intersection of two convex sets. But it also works well in the non-convex case! epubs.siam.org/doi/abs/10.113… regularize.wordpress.com/2017/02/24/the…

Fun fact (...) of the day: the pairing btw (functions, measures) is equal to the classical inner product with the covariance when applied to quadratics functions.

Oldies but goldies: Joseph Fourier, Theorie analytique de la chaleur, 1822. Introduces sines and cosines series as an approximation method and derives the heat equation PDE. en.wikipedia.org/wiki/Joseph_Fo…

Hear, hear Paris! 🥖 A cool new opportunity to learn 📚from AI apostles⭐ like Francis Bach, Gabriel Peyré, Olivier Cappé, Mathieu Blondel Pierre Senellart and many others and get a master degree! 🎓

30 more days to sign up at masteriasd.eu

Hilbert space-filling curve defines a continuous map from a segment to a square through an iterative refinement process. Often used to store and traverse 2D arrays. en.wikipedia.org/wiki/Space-fil… en.wikipedia.org/wiki/Hilbert_c…

Oldies but goldies: Mikhail Gromov, Groups of Polynomial Growth and Expanding Maps, 1981. Defines a distance in the space of metric spaces. Allows one to study the convergence of spaces. en.wikipedia.org/wiki/Gromov%E2…

Monge and Kantorovitch Optimal Transport are equivalent when the measures are supported on the same number of points. en.wikipedia.org/wiki/Assignmen… en.wikipedia.org/wiki/Doubly_st…

Oldies but goldies: H Robbins, S Monro, A Stochastic Approximation Method, 1951. Early appearance of the stochastic gradient method, which is the workhorse of many large-scale ML methods. en.wikipedia.org/wiki/Stochasti… en.wikipedia.org/wiki/Stochasti…

Stationary Gaussian fields are characterized by their power spectrum. Parametric spectra (e.g., Gabor) define procedural texture noises which are used in computer graphics. en.wikipedia.org/wiki/Gaussian_… graphics.cs.kuleuven.be/publications/G…

Oldies but goldies: Paul Erdős, Alfred Renyi, On Random Graphs, 1959. Defines the basic model for random graphs and studies some of its properties. en.wikipedia.org/wiki/Erd%C5%91…

The KAN hype has shown many people are thinking transformers still use MLPs‼️

However all the big models we know have switched to GLUs, such as Gemma (GeGLU), LLama (SwiGLU) and Palm (SwiGLU).

These 'activation functions' actually take two linear projections and multiply them

Integral curves of a vector field join stationary points of the field. They segment the space in cells of the Morse-Smale complex, corresponding to common pairs of stationary points. en.wikipedia.org/wiki/Integral_… en.wikipedia.org/wiki/Morse%E2%…

Oldies but goldies: A. S. Householder, Unitary Triangularization of a Nonsymmetric Matrix, 1958. Introduces a numerically stable way to compute the QR factorization of a matrix. en.wikipedia.org/wiki/Household…

SNE and t-SNE perform dimensionality reduction by minimizing relative entropy (KL) between high and low dimensional spaces. t-SNE improves over SNE by replacing Gaussians with students’ t-distributions in the low dimensional space. en.wikipedia.org/wiki/T-distrib…

Oldies but goldies: I.M. Sobol, Distribution of points in a cube and approximate evaluation of integrals, 1967. Introduces the most well-known low-discrepancy quasi-random sequence. en.wikipedia.org/wiki/Sobol_seq…

Oldies but goldies: Hastings, Monte Carlo Sampling Methods Using Markov Chains and Their Applications, 1970. Metropolis-Hasting is a simple way to sample from a density known only up to a constant. en.wikipedia.org/wiki/Metropoli…