We can define the Horseshoe prior in the general context of a gaussian scale mixture model :

\begin{align*}
  z_i &\sim \operatorname{Normal}(x_i^T\beta,\; \sigma^2\, \omega_i^2)\\
  \sigma^2 &\sim \pi(\sigma^2)\, \mathrm{d}\sigma^2\\
  \omega_i &\sim \pi(\omega_i)\, \mathrm{d}\omega_i\\
  \beta_j &\sim \operatorname{Normal}(0,\; \lambda_j^2\, \tau^2\, \sigma^2)\\
 \lambda_{j}  &\sim \operatorname{C}^{+}(0, 1)\\
 \tau &\sim \operatorname{C}^+(0,1)\\
\end{align*}

It is characterized by the global ( $τ$ ) and local ( $λ_{j}$ ) shrinkage parameters that follow a half-Cauchy distribution.

How does it work?

Sampling

We can use the fact that the Half-Cauchy distribution can be written as the mixture of two inverse-gamma distributions to write the conditional posterior distributions for the local shrinkage parameters.

Horsehoe prior

How does it work?

Sampling