Statistics 5102 (Geyer, Spring 2013) Examples: Bayesian Inference
Point Estimates
Posterior medians require the computer, except when the posterior distribution
is symmetric (in which case the median is the center of symmetry).
Here is the example from the slides. The data are Binomial(n,
p) and the prior distribution for
p is Beta(α1, α2)
Posterior PDF
When you have a computer, there is no point in not plotting the whole
posterior PDF. For the same example above
Interval Estimates
Equal Tails
The most obvious Bayesian competitor of frequentist confidence intervals
is the interval between the α ⁄ 2
and 1 − α ⁄ 2 quantiles of the posterior distribution of
the parameter of interest. This makes a 100 (1 − α) % credible
interval for the parameter of interest.
Again, the data are Binomial(n,
p) and the prior distribution for
p is Beta(α1, α2)
The shaded area under the curve has posterior probability
conf.level. The unshaded areas on either side
have posterior probability α ⁄ 2.
With the data, hyperparameter, and confidence level given in the form
before editing, the unshaded area to the left is too small to be seen
The next most obvious Bayesian competitor of frequentist confidence intervals
is the level set of
the posterior PDF
of the parameter of interest that has probability 1 − α.
This makes a 100 (1 − α) % credible
interval for the parameter of interest.
Again, the data are Binomial(n,
p) and the prior distribution for
p is Beta(α1, α2)
The shaded area under the curve has posterior probability
conf.level.
Unlike the equal tailed interval, the
HPD
region automatically switches from two-sided to one-sided as appropriate.
With the data, hyperparameter, and confidence level given in the form
before editing, the
HPD
is one-sided, going all the way to zero.
Two Intervals Compared
Same as in the two preceding sections except we put both intervals on one plot.
Hypothesis Tests
One Sample, One Tailed
The most obvious Bayesian competitor of frequentist P-values
is the Bayes factor comparing the hypotheses.
The hypotheses (models) are
H0
=
m1
:
p ≥ p0 H1
=
m2
:
p < p0
Again, the data are Binomial(n, p).
The prior distribution for
p is Beta(α1, α2)
conditioned on whichever hypothesis we are doing.
One Sample, Two Tailed
The hypotheses (models) are
H0
=
m1
:
p = p0 H1
=
m2
:
p ≠ p0
Again, the data are Binomial(n, p).
The prior distribution for m1 is concentrated
at the point p0.
The prior distribution for m2 is
p is Beta(α1, α2).
Two Sample, Two Tailed
Now the data are xi, i = 1, 2,
where the xi are independent
and xi
is Binomial(ni,
pi).
The hypotheses (models) are
H0
=
m1
:
p1 = p2 H1
=
m2
:
p1 ≠ p2
The prior distribution for m1 forces
p1 = p2 = p,
in which case the distribution of
x1 + x2
is Binomial(n1 +
n2, p).
For model m2 we consider
p1 and p2 a priori independent,
and we use the same Beta(α1, α2)
for both parameters. In model m1 we use the
prior Beta(α3, α4)
for the only parameter.
Two Sample, One Tailed
Now the data are xi, i = 1, 2,
where the xi are independent
and xi
is Binomial(ni,
pi).
The hypotheses (models) are
H0
=
m1
:
p1 ≥ p2 H1
=
m2
:
p1 < p2
We use the same prior distribution
Beta(α1, α2)
for both parameters.