While I try to write those with care, there is no guarantees of correctness and if you spot an error or a doubtful statement, kindly let me know by opening an issue on GitHub.
2021
July 5GMRES & Conjugate Gradient – pt. II: GMRES and Krylov subspaces, Arnoldi and Lanczos methods, conjugate gradient method, and code.
July 1GMRES & Conjugate Gradient – pt. I: Direct and iterative methods for solving a linear system of equations, including GMRES, with code.
June 29QR for eigen decomposition: Power method and the QR iteration, with code.
June 25Gram-Schmidt Orthogonalisation: Gram-Schmidt, Modified Gram-Schmidt, QR factorisation, with code.
2019
December 9CV Ridge – pt. II: Trying to generalise the LOO-CV trick for the Ridge regression.
December 8CV Ridge – pt. I: Recycling computations in Ridge regression with
LOO-CV .
2018
December 13Matrix inversion lemmas: Investigating the Woodbury formula and the Sherman-Morrison formula.
December 8Splitting methods: Splitting methods in optimisation, proximal methods, and the Alternating Direction Method of Multipliers (ADMM).
October 28First order methods: First order methods, minimising sequence, admissible direction, and the Generalised Projected Gradient Descent (again).
October 27Mirror descent algorithm: The Generalised Projected Gradient Descent (GPGD) and the Mirror Descent Algorithm (MDA).
October 10Projected gradient descent: Normal cone, Euclidean projection, and the Projected Gradient Descent (PGD).
October 9Convex analysis – pt. III: Strict and strong convexity, Bregman divergences, and the link between Lipschitz continuity and strong convexity.
September 24Convex analysis – pt. II: The convex conjugate, Fenchel's inequality, and the Fenchel-Moreau theorem.
September 23Convex analysis – pt. I: The subdifferential and the First-order Optimality Condition (FOC).
September 13Convex Optimisation – intro: Introduction to the general convex minimisation problem and generic iterative methods.
2016
March 5RKHS – pt. II: Probabilistic reasoning with kernel embeddings.
March 2RKHS – pt. I: Introduction to Reproducing Kernel Hilbert Spaces (RKHS) and embedding of distributions.