An Inductive Logic Programming Approach to Statistical Relational Learning

K. Kersting1586036742, 9781586036744, 9781429455275

In this publication, the author Kristian Kersting has made an assault on one of the hardest integration problems at the heart of Artificial Intelligence research. This involves taking three disparate major areas of research and attempting a fusion among them. The three areas are: Logic Programming, Uncertainty Reasoning and Machine Learning. Every one of these is a major sub-area of research with its own associated international research conferences. Having taken on such a Herculean task, Kersting has produced a series of results which are now at the core of a newly emerging area: Probabilistic Inductive Logic Programming. The new area is closely tied to, though strictly subsumes, a new field known as ‘Statistical Relational Learning’ which has in the last few years gained major prominence in the American Artificial Intelligence research community. Within this book, the author makes several major contributions, including the introduction of a series of definitions which circumscribe the new area formed by extending Inductive Logic Programming to the case in which clauses are annotated with probability values. Also, Kersting investigates the approach of Learning from proofs and the issue of upgrading Fisher Kernels to Relational Fisher Kernels.

---

Table of contents :
Title page……Page 2
Contents……Page 16
Abstract……Page 24
Overture……Page 25
Statistical Relational Learning……Page 26
Our Approach: The ILP Perspective……Page 29
Contributions and Outline of the Thesis……Page 30
Probabilistic Inductive Logic Programming……Page 32
Logic Programming Concepts……Page 33
Inductive Logic Programming (ILP) and its Settings……Page 36
Probabilistic ILP Settings……Page 43
Probabilistic ILP: A Definition and Example Algorithms……Page 48
Conclusions……Page 55
Part I: Probabilistic ILP over Interpretations……Page 58
Bayesian Logic Programs……Page 60
The Propositional Case: Bayesian Networks……Page 61
The First-Order Case……Page 62
Extensions of the Basic Framework……Page 70
The Learning Setting: Probabilistic Learning from Interpretations……Page 77
Scooby – Structural learning of intensional Bayesian logic programs……Page 80
Parameter Estimation……Page 86
Experimental Evaluation……Page 96
Balios – The Engine for Bayesian Logic Programs……Page 100
Future Work……Page 101
Conclusions……Page 102
Related Work……Page 103
Part II: Probabilistic ILP over Time……Page 112
Logical Hidden Markov Models……Page 114
Representation Language……Page 115
Semantics……Page 119
Design Choices……Page 121
Three Basic Inference Problems for Logical HMMs……Page 123
Evaluation……Page 125
Most Likely State Sequences……Page 126
Parameter Estimation……Page 127
Advantages of Logical Hidden Markov Models……Page 129
Real World Applications……Page 132
Learning the Structure of Logical HMMs……Page 139
The Learning Setting: Probabilistic Learning from Proofs……Page 140
A Naive Learning Algorithm……Page 141
sagEM: A Structural Generalized EM……Page 144
Experimental Evaluation……Page 147
Future Work……Page 149
Conclusions……Page 150
Related Work……Page 151
Intermezzo: Exploiting Probabilistic ILP in Discriminative Classifiers……Page 156
Relational Fisher Kernels……Page 158
Kernel Methods and Probabilistic Models……Page 159
Fisher Kernels for Interpretations and Logical Sequences……Page 160
Experimental Evaluation……Page 163
Future Work and Conclusions……Page 169
Related Work……Page 170
Part III: Making Complex Decisions in Relational Domains……Page 172
Markov Decision Programs……Page 174
Markov Decision Processes……Page 175
Representation Language……Page 178
Semantics……Page 181
Solving Markov Decision Programs……Page 182
Abstract Policies……Page 183
Generalized Relational Policy Iteration……Page 185
Model-free Relational TD(lambda)……Page 187
Model-based Relational Value Iteration based on ReBel……Page 193
Future Work……Page 203
Conclusions……Page 204
Related Work……Page 205
Finale……Page 208
Summary……Page 210
Conclusions……Page 211
Future Work……Page 212
Appendix……Page 218
Logical HMM for Unix Command Sequences……Page 220
Tree-based logical HMM for mRNA Sequences……Page 221
Bibliography……Page 224
Symbol Index……Page 245
Index……Page 247