Squared exponential kernel

The squared exponential kernel defines a stationary gaussian process and results in a smooth prior over the space of functions that can be sampled from the gaussian process. In 1 dimension we write:

\begin{align*}
\displaystyle
K(x, x') &= \sigma^{2}\, \exp\left(-\frac{{d}^{2}}{2 l^{2}}\right),\quad d=x'-x
\end{align*}

And in $D$ dimensions with the covariance matrix $\Sigma$ between the different dimensions:

\begin{align*}
\displaystyle
K(\mathbf{x}, \mathbf{x'}) &= \Sigma^{2}\, \exp\left(-\frac{1}{2} \sum_{d=1}^D \frac{{(x'_d-x_d)}^{2}}{l_d^{2}}\right)
\end{align*}

The parameter $l$ is the characteristic lengthscale of the process. As one can see on the figure below, the larger the value of $l$ the “further” the kernel takes non-negligible values.