Inverse problems

Blur a spectrum of three peaks with a Gaussian three samples wide, add noise at a thousandth of the peak height, and undo the blur by solving the linear system exactly: the result swings to 10¹³. The blur matrix is invertible on paper, but it damps fine detail so strongly that its smallest singular values are below 10⁻¹⁵ of the largest, and inverting it multiplies the noise in those directions by the reciprocal. Add a small penalty on the size of the solution, which is Tikhonov regularization, and the peaks come back within 7 % of the truth, where the blurred data were 36 % off. The measurement depends continuously on the cause, but the cause does not depend continuously on the measurement. That is what ill-posed means.

The same shape turns up wherever the instrument sits downstream of what you want: a micrograph blurred by the microscope's point spread function, the density beneath a gravity survey, a CT slice of a turbine blade, the relaxation times hidden in a battery's impedance spectrum.

In Python, numpy.linalg.svd shows the singular values and gives Tikhonov regularization for small problems; for large ones, scipy.sparse.linalg.lsqr and lsmr take the penalty as their damp argument. scikit-image deconvolves with skimage.restoration.richardson_lucy and wiener and reconstructs tomograms with skimage.transform.iradon. In Julia, svd from LinearAlgebra serves small problems, and Krylov.jl provides lsqr and lsmr with a regularization parameter for large ones.

Choose the regularization parameter from the noise level, with the discrepancy principle or the L-curve, never by eye, which picks the answer you expected. Start with the singular values of a small blur matrix, which show which details the data carry, then Tikhonov regularization and its parameter, then deconvolution of images and tomography. The decompositions are linear algebra, and a well-posed fit is curve fitting.

What belongs here

Problems where the measurement is the effect and the cause is wanted, and a naive inversion amplifies the noise: ill-posedness, Tikhonov and total variation regularization, choosing the regularization parameter, deconvolution of blurred signals and images, and tomographic reconstruction. Ordinary well-posed fits belong to curve-fitting.