Bayesian statistical modelling

Peter Congdon0470018755, 9780470018750, 9780470035931

Bayesian methods combine the evidence from the data at hand with previous quantitative knowledge to analyse practical problems in a wide range of areas. The calculations were previously complex, but it is now possible to routinely apply Bayesian methods due to advances in computing technology and the use of new sampling methods for estimating parameters. Such developments together with the availability of freeware such as WINBUGS and R have facilitated a rapid growth in the use of Bayesian methods, allowing their application in many scientific disciplines, including applied statistics, public health research, medical science, the social sciences and economics.
Following the success of the first edition, this reworked and updated book provides an accessible approach to Bayesian computing and analysis, with an emphasis on the principles of prior selection, identification and the interpretation of real data sets.
The second edition: Provides an integrated presentation of theory, examples, applications and computer algorithms. Discusses the role of Markov Chain Monte Carlo methods in computing and estimation. Includes a wide range of interdisciplinary applications, and a large selection of worked examples from the health and social sciences. Features a comprehensive range of methodologies and modelling techniques, and examines model fitting in practice using Bayesian principles. Provides exercises designed to help reinforce the reader’s knowledge and a supplementary website containing data sets and relevant programs.
Bayesian Statistical Modelling is ideal for researchers in applied statistics, medical science, public health and the social sciences, who will benefit greatly from the examples and applications featured. The book will also appeal to graduate students of applied statistics, data analysis and Bayesian methods, and will provide a great source of reference for both researchers and students.
Praise for the First Edition:
“It is a remarkable achievement to have carried out such a range of analysis on such a range of data sets. I found this book comprehensive and stimulating, and was thoroughly impressed with both the depth and the range of the discussions it contains.” – ISI – Short Book Reviews
“This is an excellent introductory book on Bayesian modelling techniques and data analysis” – Biometrics
“The book fills an important niche in the statistical literature and should be a very valuable resource for students and professionals who are utilizing Bayesian methods.” – Journal of Mathematical Psychology

---

Table of contents :
Bayesian Statistical Modelling……Page 4
Contents……Page 8
Preface……Page 16
1.1 The Bayes Approach and its Potential Advantages……Page 18
1.2 Expressing Prior Uncertainty About Parameters and Bayesian Updating……Page 19
1.3 MCMC Sampling and Inferences from Posterior Densities……Page 22
1.4 The Main MCMC Sampling Algorithms……Page 26
1.4.1 Gibbs Sampling……Page 29
1.5 Convergence of MCMC Samples……Page 31
1.7 The Present Book……Page 35
References……Page 36
2.1 Introduction: The Formal Approach to Bayes Model Choice and Averaging……Page 42
2.2 Analytic Marginal Likelihood Approximations and the Bayes Information Criterion……Page 45
2.3 Marginal Likelihood Approximations from the MCMC Output……Page 47
2.4 Approximating Bayes Factors or Model Probabilities……Page 53
2.5 Joint Space Search Methods……Page 55
2.6 Direct Model Averaging by Binary and Continuous Selection Indicators……Page 58
2.7 Predictive Model Comparison via Cross-Validation……Page 60
2.8 Predictive Fit Criteria and Posterior Predictive Model Checks……Page 63
2.9 The DIC Criterion……Page 65
2.10 Posterior and Iteration-Specific Comparisons of Likelihoods and Penalised Likelihoods……Page 67
2.11 Monte Carlo Estimates of Model Probabilities……Page 69
References……Page 74
3.1 Introduction……Page 80
3.2 Univariate Normal with Known Variance……Page 81
3.2.1 Testing Hypotheses on Normal Parameters……Page 83
3.3 Inference on Univariate Normal Parameters, Mean and Variance Unknown……Page 86
3.4 Heavy Tailed and Skew Density Alternatives to the Normal……Page 88
3.5 Categorical Distributions: Binomial and Binary Data……Page 91
3.5.1 Simulating Controls Through Historical Exposure……Page 93
3.6 Poisson Distribution for Event Counts……Page 96
3.7 The Multinomial and Cirichlet Densities for Categorical and Proportional Data……Page 99
3.8 Multivariate Continuous Data: Multivariate Normal and t Densities……Page 102
3.8.1 Partitioning Multivariate Priors……Page 104
3.8.2 The Multivariate t Density……Page 105
3.9 Applications of Standard Densities: Classification Rules……Page 108
3.10 Applications of Standard Densities: Multivariate Discrimination……Page 115
Exercises……Page 117
References……Page 119
4.1 The Context for Bayesian Regression Methods……Page 126
4.2 The Normal Linear Regression Model……Page 128
4.2.1 Unknown Regression Variance……Page 129
4.3 Normal Linear Regression: Variable and Model Selection, Outlier Detection and Error form……Page 133
4.3.1 Other Predictor and Model Search Methods……Page 135
4.4 Bayesian Ridge Priors for Multicollinearity……Page 138
4.6 Binary and Binomial Regression……Page 140
4.6.1 Priors on Regression Coefficients……Page 141
4.6.2 Model Checks……Page 143
4.7 Latent Data Sampling for Binary Regression……Page 146
4.8 Poisson Regression……Page 149
4.8.1 Poisson Regression for Contingency Tables……Page 151
4.8.2 Log-Linear Model Selection……Page 156
4.9 Multivariate Responses……Page 157
Exercises……Page 160
References……Page 163
5.1 Hierarchical Priors for Pooling Strength and in General Linear Model Regression……Page 168
5.2 Hierarchical Priors: Conjugate and Non-Conjugate Mixing……Page 169
5.3 Hierarchical Priors for Normal Data with Applications in Meta-Analysis……Page 170
5.3.1 Prior for Second-Stage Variance……Page 172
5.4 Pooling Strength under Exchangeable Models for Poisson Outcomes……Page 174
5.4.1 Hierarchical Prior Choices……Page 175
5.4.2 Parameter Sampling……Page 176
5.5 Combining Information for Binomial Outcomes……Page 179
5.6 Random Effects Regression for Overdispersed Count and Binomial Data……Page 182
5.7 Overdispersed Normal Regression: The Scale-Mixture Student t Model……Page 186
5.8 The Normal Meta-Analysis Model Allowing for Heterogeneity in Study Design or Patient Risk……Page 190
5.9 Hierarchical Priors for Multinomial Data……Page 193
5.9.1 Histogram Smoothing……Page 194
Exercises……Page 196
References……Page 200
6.1 Introduction: The Relevance and Applicability of Discrete Mixtures……Page 204
6.2 Discrete Mixtures of Parametric Densities……Page 205
6.2.1 Model Choice……Page 207
6.3 Identifiability Constraints……Page 208
6.4 Hurdle and Zero-Inflated Models for Discrete Data……Page 212
6.5 Regression Mixtures for Heterogeneous Subpopulations……Page 214
6.6 Discrete Mixtures Combined with Parametric Random Effects……Page 217
6.7 Non-Parametric Mixture Modelling via Dirichlet Process Priors……Page 218
6.8 Other Non-Parametric Priors……Page 224
Exercises……Page 229
References……Page 231
7.1 Introduction: Applications with Categoric and Ordinal Data……Page 236
7.2 Multinomial Logit Choice Models……Page 238
7.3 The Multinomial Probit Representation of Interdependent Choices……Page 241
7.4 Mixed Multinomial Logit Models……Page 245
7.5 Individual Level Ordinal Regression……Page 247
7.6 Scores for Ordered Factors in Contingency Tables……Page 252
Exercises……Page 254
References……Page 255
8.1 Introduction: Alternative Approaches to Time Series Models……Page 258
8.2 Autoregressive Models in the Observations……Page 259
8.2.1 Priors on Autoregressive Coefficients……Page 261
8.2.2 Initial Conditions as Latent Data……Page 263
8.3 Trend Stationarity in the AR1 Model……Page 265
8.4 Autoregressive Moving Average Models……Page 267
8.5 Autoregressive Errors……Page 270
8.6 Multivariate Series……Page 272
8.7.1 Observation-Driven Autodependence……Page 274
8.7.2 INAR Models……Page 275
8.7.3 Error Autocorrelation……Page 276
8.8 Dynamic Linear Models and Time Varying Coefficients……Page 278
8.8.1 Some Common Forms of DLM……Page 281
8.8.2 Priors for TIme-Specific Variances or Interventions……Page 284
8.8.3 Nonlinear and Non-Gaussian State-Space Models……Page 285
8.9 Models for Variance Evolution……Page 290
8.9.1 ARCH and GARCH Models……Page 291
8.9.2 Stochastic Volatility Models……Page 292
8.10 Modelling Structural Shifts and Outliers……Page 294
8.10.1 Markov Mixtures and Transition Functions……Page 296
8.11 Other Nonlinear Models……Page 299
Exercises……Page 302
References……Page 305
9.1 Introduction: Implications of Spatial Dependence……Page 314
9.2 Discrete Space Regressions for Metric Data……Page 315
9.3 Discrete Spatial Regression with Structured and Unstructured Random Effects……Page 320
9.3.1 Proper CAR Priors……Page 323
9.4 Moving Average Priors……Page 328
9.5 Multivariate Spatial Priors and Spatially Varying Regression Effects……Page 330
9.6 Robust Models for Discontinuities and Non-Standard Errors……Page 334
9.7 Continuous Space Modelling in Regression and Interpolation……Page 338
Exercises……Page 342
References……Page 346
10.1 Approaches to Modelling Nonlinearity……Page 350
10.2 Nonlinear Metric Data Models with Known Functional Form……Page 352
10.3 Box–Cox Transformations and Fractional Polynomials……Page 355
10.4 Nonlinear Regression Through Spline and Radial Basis Functions……Page 359
10.4.1 Shrinkage Models for Spline Coefficients……Page 362
10.4.2 Modelling Interaction Effects……Page 363
10.5 Application of State-Space Priors in General Additive Nonparametric Regression……Page 367
10.5.1 Continuous Predictor Space Prior……Page 368
10.5.2 Discrete Predictor Space Priors……Page 370
Exercises……Page 376
References……Page 379
11.1 Introduction: Nested Data Structures……Page 384
11.2.1 The Multilevel Normal Linear Model……Page 386
11.2.2 General Linear Mixed Models for Discrete Outcomes……Page 387
11.2.3 Multinomial and Ordinal Multilevel Models……Page 389
11.2.4 Robustness Regarding Cluster Effects……Page 390
11.2.5 Conjugate Approaches for Discrete Data……Page 391
11.3 Heteroscedasticity in Multilevel Models……Page 396
11.4 Random Effects for Crossed Factors……Page 398
11.5 Panel Data Models: The Normal Mixed Model and Extensions……Page 404
11.5.1 Autocorrelated Errors……Page 407
11.5.2 Autoregression in y……Page 408
11.6.1 Binary Panel Data……Page 410
11.6.2 Repeated Counts……Page 412
11.6.3 Panel Categorical Data……Page 414
11.7 Growth Curve Models……Page 417
11.8 Dynamic Models for Longitudinal Data: Pooling Strength Over Units and Times……Page 420
11.9 Area APC and Spatiotemporal Models……Page 424
11.9.1 Age–Period Data……Page 425
11.9.3 Age–Area–Period Data……Page 426
11.9.4 Interaction Priors……Page 427
Exercises……Page 430
References……Page 435
12.1 Introduction: Latent Traits and Latent Classes……Page 442
12.2 Factor Analysis and SEMS for Continuous Data……Page 444
12.2.1 Identifiability Constraints in Latent Trait (Factor Analysis) Models……Page 446
12.3 Latent Class Models……Page 450
12.3.1 Local Dependence……Page 454
12.4 Factor Analysis and SEMS for Multivariate Discrete Data……Page 458
12.5 Nonlinear Factor Models……Page 464
Exercises……Page 467
References……Page 469
13.1 Introduction……Page 474
13.2 Parametric Survival Analysis in Continuous Time……Page 475
13.2.1 Censored Observations……Page 476
13.2.2 Forms of Parametric Hazard and Survival Curves……Page 477
13.2.3 Modelling Covariate Impacts and Time Dependence in the Hazard Rate……Page 478
13.3 Accelerated Hazard Parametric Models……Page 481
13.4 Counting Process Models……Page 483
13.5 Semiparametric Hazard Models……Page 486
13.5.1 Priors for the Baseline Hazard……Page 487
13.5.2 Gamma Process Prior on Cumulative Hazard……Page 489
13.6 Competing Risk-Continuous Time Models……Page 492
13.7 Variations in Proneness: Models for Frailty……Page 494
13.8 Discrete Time Survival Models……Page 499
Exercises……Page 503
References……Page 504
14.1 Introduction: Types of Missingness……Page 510
14.2 Selection and Pattern Mixture Models for the Joint Data-Missingness Density……Page 511
14.3 Shared Random Effect and Common Factor Models……Page 515
14.4 Missing Predictor Data……Page 517
14.5 Multiple Imputation……Page 520
14.6.1 Hierarchical Models for Response and Non-Response by Strata……Page 523
14.6.2 Regression Frameworks……Page 527
14.7 Missingness with Mixtures of Continuous and Categorical Data……Page 533
14.8 Missing Cells in Contingency Tables……Page 535
14.8.1 Ecological Inference……Page 536
Exercises……Page 543
References……Page 546
15.2 Measurement Error in Both Predictors and Response in Normal Linear Regression……Page 550
15.2.1 Prior Information on X or its Density……Page 552
15.2.2 Measurement Error in General Linear Models……Page 554
15.3 Misclassification of Categorical Variables……Page 558
15.4 Simultaneous Equations and Instruments for Endogenous Variables……Page 563
15.5 Endogenous Regression Involving Discrete Variables……Page 567
Exercises……Page 571
References……Page 573
A1.1 Procedure for Compiling and Running Programs……Page 578
A1.2 Generating Simulated Data……Page 579
A1.3 Other Advice……Page 580
Index……Page 582
Wiley Series in Probability and Statistics……Page 591