Information theory and statistical learning

Ray J. Solomonoff (auth.), Frank Emmert-Streib, Matthias Dehmer (eds.)0387848150, 9780387848150

Information Theory and Statistical Learning presents theoretical and practical results about information theoretic methods used in the context of statistical learning.

The book will present a comprehensive overview of the large range of different methods that have been developed in a multitude of contexts. Each chapter is written by an expert in the field. The book is intended for an interdisciplinary readership working in machine learning, applied statistics, artificial intelligence, biostatistics, computational biology, bioinformatics, web mining or related disciplines.

Advance Praise for Information Theory and Statistical Learning:

“A new epoch has arrived for information sciences to integrate various disciplines such as information theory, machine learning, statistical inference, data mining, model selection etc. I am enthusiastic about recommending the present book to researchers and students, because it summarizes most of these new emerging subjects and methods, which are otherwise scattered in many places.”

— Shun-ichi Amari, RIKEN Brain Science Institute, Professor-Emeritus at the University of Tokyo

Table of contents :
Front Matter….Pages i-x
Algorithmic Probability: Theory and Applications….Pages 1-23
Model Selection and Testing by the MDL Principle….Pages 25-43
Normalized Information Distance….Pages 45-82
The Application of Data Compression-Based Distances to Biological Sequences….Pages 83-100
MIC: Mutual Information Based Hierarchical Clustering….Pages 101-123
A Hybrid Genetic Algorithm for Feature Selection Based on Mutual Information….Pages 125-152
Information Approach to Blind Source Separation and Deconvolution….Pages 153-182
Causality in Time Series: Its Detection and Quantification by Means of Information Theory….Pages 183-207
Information Theoretic Learning and Kernel Methods….Pages 209-230
Information-Theoretic Causal Power….Pages 231-265
Information Flows in Complex Networks….Pages 267-287
Models of Information Processing in the Sensorimotor Loop….Pages 289-308
Information Divergence Geometry and the Application to Statistical Machine Learning….Pages 309-332
Model Selection and Information Criterion….Pages 333-354
Extreme Physical Information as a Principle of Universal Stability….Pages 355-384
Entropy and Cloning Methods for Combinatorial Optimization, Sampling and Counting Using the Gibbs Sampler….Pages 385-434
Back Matter….Pages 435-439