Bayes Linear Statistics: Theory & Methods

Michael Goldstein, David Wooff9780470015629, 0470015624

Bayesian methods combine information available from data with any prior information available from expert knowledge. The Bayes linear approach follows this path, offering a quantitative structure for expressing beliefs, and systematic methods for adjusting these beliefs, given observational data. The methodology differs from the full Bayesian methodology in that it establishes simpler approaches to belief specification and analysis based around expectation judgements. Bayes Linear Statistics presents an authoritative account of this approach, explaining the foundations, theory, methodology, and practicalities of this important field.The text provides a thorough coverage of Bayes linear analysis, from the development of the basic language to the collection of algebraic results needed for efficient implementation, with detailed practical examples.The book covers:• The importance of partial prior specifications for complex problems where it is difficult to supply a meaningful full prior probability specification.• Simple ways to use partial prior specifications to adjust beliefs, given observations.• Interpretative and diagnostic tools to display the implications of collections of belief statements, and to make stringent comparisons between expected and actual observations.• General approaches to statistical modelling based upon partial exchangeability judgements.• Bayes linear graphical models to represent and display partial belief specifications, organize computations, and display the results of analyses.Bayes Linear Statistics is essential readingfor all statisticians concerned with the theory and practice of Bayesian methods. There is an accompanying website hosting free software and guides to the calculations within the book.

---

Table of contents :
Bayes Linear Statistics……Page 4
Contents……Page 10
Preface……Page 20
1 The Bayes linear approach……Page 24
1.1 Combining beliefs with data……Page 25
1.2 The Bayesian approach……Page 26
1.3 Features of the Bayes linear approach……Page 29
1.4 Example……Page 30
1.4.2 Prior inputs……Page 31
1.4.3 Adjusted expectations……Page 32
1.4.5 Adjusted variances……Page 33
1.4.6 Checking data inputs……Page 34
1.4.9 Further diagnostics for the adjusted versions……Page 35
1.4.10 Summary of basic adjustment……Page 36
1.4.11 Diagnostics for collections……Page 37
1.4.12 Exploring collections of beliefs via canonical structure……Page 40
1.4.13 Modifying the original specifications……Page 42
1.4.14 Repeating the analysis for the revised model……Page 43
1.4.15 Global analysis of collections of observations……Page 45
1.4.16 Partial adjustments……Page 47
1.4.17 Partial diagnostics……Page 50
1.4.18 Summary……Page 52
1.5 Overview……Page 53
2.1 Expectation as a primitive……Page 56
2.2 Discussion: expectation as a primitive……Page 58
2.3 Quantifying collections of uncertainties……Page 60
2.4.1 Example: oral glucose tolerance test……Page 62
2.5 Qualitative and quantitative prior specification……Page 64
2.6.2 Identifying relevant prior information……Page 65
2.6.3 Sources of variation……Page 66
2.6.5 The qualitative representation……Page 67
2.6.6 Graphical models……Page 69
2.7.1 Prior expectations……Page 71
2.7.2 Prior variances……Page 72
2.7.3 Prior covariances……Page 74
2.8 Discussion: on the various methods for assigning expectations……Page 75
3.1 Adjusted expectation……Page 78
3.2 Properties of adjusted expectation……Page 79
3.3 Adjusted variance……Page 80
3.4 Interpretations of belief adjustment……Page 81
3.5 Foundational issues concerning belief adjustment……Page 83
3.6 Example: one-dimensional problem……Page 86
3.7 Collections of adjusted beliefs……Page 87
3.8.1 Algebraic example……Page 88
3.8.2 Oral glucose tolerance test……Page 92
3.8.3 Many oral glucose tolerance tests……Page 96
3.9.1 Canonical directions for the adjustment……Page 98
3.9.2 The resolution transform……Page 100
3.9.3 Partitioning the resolution……Page 102
3.9.4 The reverse adjustment……Page 103
3.9.6 The adjusted belief transform matrix……Page 104
3.10 The geometric interpretation of belief adjustment……Page 105
3.11.2 Algebraic example……Page 107
3.11.3 Oral glucose tolerance test……Page 113
3.12 Further reading……Page 116
4.1 Discrepancy……Page 118
4.1.1 Discrepancy for a collection……Page 119
4.2 Properties of discrepancy measures……Page 121
4.3.2 Detecting degeneracy……Page 123
4.3.3 Oral glucose tolerance test……Page 125
4.4.1 Adjustment discrepancy……Page 127
4.4.2 Adjustment discrepancy for a collection……Page 128
4.4.3 Maximal discrepancy……Page 129
4.4.4 Construction over a basis……Page 130
4.5.1 Simple one-dimensional problem……Page 131
4.5.2 Oral glucose tolerance test……Page 132
4.6.1 The size of an adjustment for a collection……Page 136
4.7 The bearing for an adjustment……Page 137
4.7.2 Representing discrepancy vectors as bearings……Page 138
4.9 Size diagnostics……Page 139
4.10 Geometric interpretation……Page 140
4.11 Linear likelihood……Page 141
4.12.1 Algebraic example……Page 142
4.12.2 Oral glucose tolerance test……Page 143
5.1 Partial adjustment……Page 148
5.2 Partial variance……Page 150
5.3 Partial resolution transforms……Page 151
5.4 Relative belief adjustment……Page 152
5.5 Example: oral glucose tolerance test……Page 153
5.5.1 Performing an initial adjustment……Page 154
5.5.3 Partial canonical directions……Page 155
5.5.5 Relative belief adjustment……Page 156
5.5.6 Withdrawing quantities from the adjustment……Page 157
5.6 Partial bearings……Page 158
5.8 Bearing and size for a relative adjustment……Page 160
5.9 Path correlation……Page 161
5.10.1 The initial observed adjustment……Page 162
5.10.2 Observed partial expectations……Page 163
5.10.3 The size of the partial adjustment……Page 164
5.10.4 The bearing for the partial adjustment……Page 165
5.10.5 The path correlation for the partial adjustment……Page 166
5.11.1 The data trajectory……Page 167
5.12 The canonical trajectory……Page 168
5.13 Detection of systematic bias……Page 169
5.14.1 Anscombe data sets……Page 170
5.14.2 Regression with correlated responses……Page 176
5.15 Bayes linear sufficiency and belief separation……Page 189
5.16 Properties of generalized conditional independence……Page 191
5.17 Properties of belief separation……Page 192
5.18.1 Exploiting separation……Page 195
5.18.2 Heart of the transform……Page 196
5.19 Further reading……Page 199
6.1 Exchangeability……Page 200
6.2 Coin tossing……Page 203
6.3 Exchangeable belief structures……Page 206
6.4 The representation theorem……Page 208
6.5 Finite exchangeability……Page 211
6.6 Example: oral glucose tolerance test……Page 212
6.7.1 Introduction……Page 214
6.7.2 Error structure and specifications……Page 215
6.7.3 Regression coefficient specifications……Page 216
6.8 Adjusting exchangeable beliefs……Page 217
6.9 Predictive sufficiency for exchangeable models……Page 218
6.10 Bayes linear sufficiency for sample means……Page 219
6.11 Belief adjustment for scalar exchangeable quantities……Page 220
6.12 Canonical structure for an exchangeable adjustment……Page 221
6.12.1 Standard form for the adjustment……Page 223
6.12.2 Further properties of exchangeable adjustments……Page 224
6.13 Algebraic example……Page 225
6.13.2 Coherence……Page 226
6.13.3 Bayes linear sufficiency……Page 227
6.14.1 Bayes linear sufficiency……Page 228
6.14.2 Adjustment……Page 229
6.14.3 Resolution transforms……Page 231
6.14.4 Resolution partition for exchangeable cases……Page 233
6.14.5 Data diagnostics……Page 234
6.14.6 Sample size choice……Page 235
6.14.8 Data diagnostics for an equivalent linear space……Page 237
6.14.9 Compatibility of data sources……Page 238
6.15 Predictive adjustment……Page 241
6.16.2 Mean component adjustment……Page 243
6.16.3 Variance reduction for a predictive adjustment……Page 244
6.16.4 Observed exchangeable adjustments……Page 246
6.16.5 Path diagnostics……Page 249
6.17 Example: predictive analysis for exchangeable regressions……Page 251
6.17.1 Choice of canonical directions……Page 253
6.18 Further reading……Page 254
7.1 Respecting exchangeability……Page 256
7.2 Adjustments respecting exchangeability……Page 257
7.3 Example: simple algebraic problem……Page 258
7.3.2 Resolution transform……Page 259
7.4 Co-exchangeable adjustments……Page 261
7.5 Example: analysing further exchangeable regressions……Page 263
7.5.1 The resolution envelope……Page 266
7.6.1 Model……Page 267
7.6.2 Specifications……Page 271
7.6.4 Analysis……Page 274
8.1 Assessing a population variance with known population mean……Page 288
8.2 Assessing a population variance with unknown population mean……Page 289
8.3 Choice of prior values……Page 291
8.4 Example: oral glucose tolerance test……Page 294
8.5 Adjusting the population residual variance in multiple linear regression: uncorrelated errors……Page 296
8.5.1 Sample information……Page 297
8.6 Example: Anscombe data sets……Page 299
8.7 Adjusting the population residual variance in multiple linear regression: correlated errors……Page 300
8.8 Example: regression with correlated responses……Page 301
8.9 Example: analysing exchangeable regressions……Page 303
8.10 Adjusting a collection of population variances and covariances……Page 305
8.11 Direct adjustment for a population variance matrix……Page 306
8.12 Example: regression with correlated responses……Page 307
8.13 Separating direct adjustment for population variances and for correlation structure……Page 308
8.13.1 Assessing the equivalent sample size……Page 309
8.14 Example: oral glucose tolerance test……Page 310
8.15 Two-stage Bayes linear analysis……Page 311
8.17 Example: analysing exchangeable regressions……Page 313
8.18 Further reading……Page 315
9 Belief comparison……Page 316
9.1 Comparing variance specifications……Page 317
9.1.1 Rank-degenerate case……Page 319
9.2 Example: variance comparison……Page 321
9.2.1 Canonical structure for the comparison……Page 322
9.2.3 Comparisons for further constructed quantities……Page 324
9.3 Comparing many variance specifications……Page 325
9.4 Example: comparing some simple nested hypotheses……Page 327
9.5.1 General belief transforms……Page 329
9.5.2 Properties of general belief transforms……Page 330
9.5.3 Adjusted belief transforms as general belief transforms……Page 332
9.5.4 Example: adjustment of exchangeable structures……Page 333
9.5.5 Example: analysing exchangeable regressions……Page 334
9.6 Comparing expectations and variances……Page 335
9.7 Geometric interpretation……Page 337
9.8 Residual forms for mean and variance comparisons……Page 338
9.8.1 Rank-degenerate case……Page 340
9.9 The observed comparison……Page 341
9.9.1 Combined directions……Page 342
9.10 Example: mean and variance comparison……Page 343
9.10.1 The observed comparison……Page 346
9.11 Graphical comparison of specifications……Page 347
9.11.1 Belief comparison diagram……Page 348
9.11.2 The observed comparison……Page 350
9.11.4 Residual belief comparison diagrams……Page 352
9.12 Example: exchangeable regressions……Page 354
9.12.1 Basic canonical analysis……Page 355
9.12.2 Mean and residual comparisons……Page 356
9.13 Comparisons for exchangeable structures……Page 360
9.13.1 The observed comparison……Page 361
9.13.2 Example: exchangeable regressions……Page 363
9.14 Example: fly population dynamics……Page 365
9.14.2 Differences for the residual part of the average……Page 366
9.14.3 Differences for the residual part of the average……Page 367
9.15 Assessing robustness of specifications……Page 369
9.15.1 Sensitivity analyses for expectations……Page 370
9.15.2 Example: robustness analysis for exchangeable regressions……Page 372
9.15.3 Sensitivity analyses for variances……Page 373
9.15.4 Example: robustness analysis for variance specifications……Page 374
9.16 Further reading……Page 376
10 Bayes linear graphical models……Page 378
10.1 Directed graphical models……Page 379
10.2 Operations on directed graphs……Page 381
10.3 Quantifying a directed graphical model……Page 384
10.4 Undirected graphs……Page 385
10.5 Example……Page 387
10.5.2 Reading properties from the diagram……Page 390
10.5.3 Alternative diagrams……Page 391
10.5.4 Diagrams for inference and prediction……Page 393
10.6 Displaying the flow of information……Page 395
10.6.1 Node shading……Page 396
10.6.2 Arc labelling……Page 397
10.6.3 Tracking information as it is received……Page 399
10.6.4 Example……Page 400
10.7 Displaying diagnostic information……Page 406
10.7.1 Node diagnostics……Page 408
10.7.2 Arc diagnostics……Page 410
10.7.4 Interpreting diagnostic warnings……Page 411
10.7.5 Example: inference and prediction……Page 412
10.8 Local computation: directed trees……Page 418
10.8.1 Propagation……Page 420
10.8.2 Example……Page 421
10.9 Junction trees……Page 422
10.10 Sequential local computation on the junction tree……Page 423
10.12.1 Problem summary……Page 425
10.12.2 Identifying the quantities of interest……Page 426
10.12.3 Modelling……Page 427
10.12.4 Initialization values and specifications……Page 429
10.12.5 Examining the generated model……Page 435
10.12.6 Basic adjustment……Page 437
10.12.7 Exploration via graphical models……Page 439
10.13 Local computation for global adjustment of the junction tree……Page 447
10.13.1 Merging separate adjustments……Page 448
10.13.3 Absorption of evidence……Page 450
10.14 Further reading……Page 452
11.2 Covariance matrices and quadratic forms……Page 454
11.3.2 Computing the Moore–Penrose inverse……Page 455
11.3.3 Other properties of generalized inverses……Page 456
11.4 Multiplication laws……Page 457
11.5 Range and null space of a matrix……Page 458
11.7.1 Definiteness for a partitioned real symmetric matrix……Page 459
11.7.2 Generalized inverses for partitioned non-negative definite matrices……Page 460
11.8 Solving linear equations……Page 461
11.9 Eigensolutions to related matrices……Page 462
11.10 Maximizing a ratio of quadratic forms……Page 463
11.11.1 Introduction……Page 464
11.11.3 An alternative algorithm……Page 465
11.11.4 An algorithm for B – A non-negative definite……Page 467
11.12.1 The Helmert matrix……Page 470
11.12.2 Direct products……Page 471
12.1 Introduction……Page 474
12.2.2 Coherence for two collections……Page 475
12.2.3 Coherence for three collections……Page 476
12.3.1 Consistency for a single collection……Page 478
12.3.2 Consistency for a partitioned collection……Page 479
12.4 Adjusted expectation……Page 480
12.5 Adjusted and resolved variance……Page 481
12.6 The resolved variance matrix……Page 482
12.7 Matrix representations of the resolution transform……Page 483
12.7.1 The symmetrized resolution transform matrix……Page 484
12.7.2 The transform for the reverse adjustment……Page 486
12.7.3 Inverses for the resolved variance matrix……Page 487
12.7.4 Canonical quantities……Page 488
12.7.5 Coherence via the resolution transform matrix……Page 489
12.8 Assessing discrepant data……Page 490
12.9 Consistency of observed adjustments……Page 491
12.9.1 Partitioning the discrepancy……Page 492
12.10 The bearing and size of adjustment……Page 495
12.11 Partial adjustments……Page 496
12.11.2 Calculating the partial bearing……Page 498
12.12.1 Notation……Page 499
12.12.4 Pure exchangeable adjustments……Page 500
12.12.5 General exchangeable adjustments……Page 504
12.13.2 Comparison of exchangeable beliefs……Page 506
A Notation……Page 510
B Index of examples……Page 514
C.2 BAYES-LIN……Page 518
References……Page 520
Index……Page 526