"Bayesian Statistics" is the school of thought that combines prior beliefs with the likelihood of a hypothesis to arrive at posterior beliefs. The first edition of Peter Lee's book appeared in 1989, but the subject has moved ever onwards, with increasing emphasis on Monte Carlo based techniques. This new fourth edition looks at recent techniques such as variational methods, Bayesian importance sampling, approximate Bayesian computation and Reversible Jump Markov Chain Monte Carlo (RJMCMC), providing a concise account of the way in which the Bayesian approach to statistics develops as well as how it contrasts with the conventional approach. The theory is built up step by step, and important notions such as sufficiency are brought out of a discussion of the salient features of specific examples. This edition: includes expanded coverage of Gibbs sampling, including more numerical examples and treatments of OpenBUGS, R2WinBUGS and R2OpenBUGS; presents significant new material on recent techniques such as Bayesian importance sampling, variational Bayes, Approximate Bayesian Computation (ABC) and Reversible Jump Markov Chain Monte Carlo (RJMCMC); provides extensive examples throughout the book to complement the theory presented; and, accompanied by a supporting website featuring new material and solutions. More and more students are realizing that they need to learn Bayesian statistics to meet their academic and professional goals. This book is best suited for use as a main text in courses on Bayesian statistics for third and fourth year undergraduates and postgraduate students.
Preface Preface to the First Edition 1 Preliminaries 1.1 Probability and Bayes' Theorem 1.2 Examples on Bayes' Theorem 1.3 Random variables 1.4 Several random variables 1.5 Means and variances 1.6 Exercises on Chapter 2 Bayesian inference for the normal distribution 2.1 Nature of Bayesian inference 2.3 Several normal observations with a normal prior 2.4 Dominant likelihoods 2.5 Locally uniform priors 2.6 Highest density regions 2.7 Normal variance 2.8 HDRs for the normal variance 2.9 The role of sufficiency 2.10 Conjugate prior distributions 2.11 The exponential family 2.12 Normal mean and variance both unknown 2.13 Conjugate joint prior for the normal distribution 2.14 Exercises on Chapter 2 3 Some other common distributions 3.1 The binomial distribution 3.2 Reference prior for the binomial likelihood 3.3 Jeffreys' rule 3.4 The Poisson distribution 3.5 The uniform distribution 3.6 Reference prior for the uniform distribution 3.7 The tramcar problem 3.8 The first digit problem; invariant priors 3.9 The circular normal distribution 3.10 Approximations based on the likelihood 3.11 Reference posterior distributions 3.12 Exercises on Chapter 3 4 Hypothesis testing 4.1 Hypothesis testing 4.2 One-sided hypothesis tests 4.3 Lindley's method 4.4 Point (or sharp) null hypotheses with prior information 4.5 Point null hypotheses for the normal distribution 4.6 The Doogian philosophy 4.7 Exercises on Chapter 4 5Two-sample problems 5.1 Two-sample problems -- both variances unknown 5.2 Variances unknown but equal 5.3 Variances unknown and unequal (Behrens--Fisher problem) 5.4 The Behrens--Fisher controversy 5.5 Inferences concerning a variance ratio 5.6 Comparison of two proportions; the 2 × 2 table 5.7 Exercises on Chapter 5 6 Correlation, regression and the analysis of variance 6.1 Theory of the correlation coefficient 6.2 Examples on the use of the correlation coefficient 6.3 Regression and the bivariate normal model 6.4 Conjugate prior for the bivariate regression model 6.5 Comparison of several means -- the one way model 6.6 The two way layout 6.7 The general linear model 6.8 Exercises on Chapter 6 7 Other topics 7.1 The likelihood principle 7.2 The stopping rule principle 7.3 Informative stopping rules 7.4 The likelihood principle and reference priors 7.5 Bayesian decision theory 7.6 Bayes linear methods 7.7 Decision theory and hypothesis testing 7.8 Empirical Bayes methods 7.9 Exercises on Chapter 7 8 Hierarchical models 8.1 The idea of a hierarchical model 8.2 The hierarchical normal model 8.3 The baseball example 8.4 The Stein estimator 8.5 Bayesian analysis for an unknown overall mean 8.6 The general linear model revisited 8.7 Exercises on Chapter 8 9 The Gibbs sampler and other numerical methods 9.1 Introduction to numerical methods 9.2 The EM algorithm 9.3 Data augmentation by Monte Carlo 9.4 The Gibbs sampler 9.5 Rejection sampling 9.6 The Metropolis--Hastings algorithm 9.7 Introduction to WinBUGS and OpenBUGS 9.8 Generalized linear models 9.9 Exercises on Chapter 9 10 Some approximate methods 10.1 Bayesian importance sampling 10.2 Variational Bayesian methods: simple case 10.3 Variational Bayesian methods: general case 10.4 ABC : Approximate Bayesian Computation 10.5 Reversible Jump Markov Chain Monte Carlo 10.6 Exercises on Chapter 10 Appendix A common statistical distributions A.1 Normal distribution A.2 Chi-squared distribution A.3 Normal approximation to chi-squared A.4 Gamma distribution A.5 Inverse chi-squared distribution A.6 Inverse chi distribution A.7 Log chi-squared distribution A.8 Student's t distribution A.9 Normal/chi-squared distribution A.10 Beta distribution A.11 Binomial distribution A.12 Poisson distribution A.13 Negative binomial distribution A.14 Hypergeometric distribution A.15 Uniform distribution A.16 Pareto distribution A.17 Circular normal distribution