Recent Articles

Bayes factors

In statistics, the use of Bayes factors is a Bayesian alternative to classical hypothesis testing.

Given a model selection problem in which we have to choose between two models M_{1 and M2, on the basis of a data vector x. The Bayes factor K is given by}

<math>K = \frac{p(x|M_1)}{p(x|M_2)}.<math>

In a sense this is a likelihood-ratio test. Generally, the models M_{1 and M2 will be parametrised by vectors of parameters <math>\theta_1<math> and <math>\theta_2<math>; thus K is given by}

<math>K = \frac{p(x|M_1)}{p(x|M_2)} = \frac{\int \,p(\theta_1|M_1)p(x|\theta_1 M_1)d\theta_1}{\int \,p(\theta_2|M_2)p(x|\theta_2 M_2)d\theta_2}.<math>

A value of K > 1 means that the data indicate that M₁ is more likely than M₂ and vice versa. Harold Jeffreys gave a scale for interpretation of K:

K	Strength of evidence
< 1	Negative (supports M2)
1 to 3	Barely worth mentioning
3 to 12	Positive
12 to 150	Strong
> 150	Very strong

Many Bayesian statisticians would use a Bayes factor as part of making a choice, but would also combine it with their estimates of the prior probability of each of the models and the loss functions associated with making the wrong choice.

Example

Suppose we have a random variable which produces either a success or a failure. We want to consider a model M₁ where the probability of success is q=1/2, and another model M₂ where q is completely unknown and we take a prior distribution for q which is uniform on [0,1]. We take a sample of 200, and find 115 success and 85 failures. The likelihood is

<math>{200 \choose 115}q^{115}(1-q)^{85}.<math>

So we have

<math>P(X=115|M_1)={200 \choose 115}\left({1 \over 2}\right)^{200}=0.00595...<math>

but

<math>P(X=115|M_2)=\int_{q=0}^1 1{200 \choose 115}q^{115}(1-q)^{85}dq = {1 \over 201} = 0.00497...<math>

The ratio is then 1.197..., which is "barely worth mentioning" even if it points very slightly towards M₁.

This is not the same as a classical likelihood ratio test, which would have found the maximum likelihood estimate for q, namely 115/200=0.575, and from that get a ratio of 0.1045..., and so pointing towards M₂. A frequentist hypothesis test would have produced an even more dramatic result, saying that that M₁ could be rejected at the 5% confidence level, since the probability of getting 115 or more successes from a sample of 200 if q=1/2 is 0.0200..., and as a two-tailed test of getting a figure as extreme as or more extreme than 115 is 0.0400... Note that 115 is more than two standard deviations away from 100.

M₂ is a more complex model than M₁ because it has a free parameter which allows it to model the data more closely. The ability of Bayes factors to take this into account is a reason why Bayesian inference has been put forward as a theoretical justification for and generalisation of Occam's razor, reducing Type I errors.

See also

• Bayesian model comparison