Information Theory: A Tutorial Introduction, 2nd edition

"A superb introduction."​

Originally developed by Claude Shannon in the 1940s, information theory laid the foundations for the digital revolution, and is now an essential tool in telecommunications, genetics, linguistics, brain sciences, and deep space communication. In Information Theory: A Tutorial Introduction, accessible examples are used to introduce information theory in terms of everyday games like "20 questions" before more advanced topics are explored.

Online MATLAB computer programs provide hands-on experience of information theory in action, and PowerPoint slides give support for teaching. Written in an informal style, with a comprehensive glossary and tutorial appendices, this text is an ideal primer for novices who wish to learn the essential principles and applications of information theory.

Note that this Second Edition is identical to the First Edition, except for the addition of two new chapters: Rate Distortion Theory and Transfer Entropy. Claude Shannon introduced the idea of Rate Distortion Theory in his original book published in 1949 ("The Mathematical Theory of Communication"), and he followed this with a complete theory in a paper published in 1959 ("Coding Theorems for a Discrete Source with a Fidelity Criterion"). More recently, information theory has been applied to the problem of inferring causality, in the form of Transfer Entropy. The underlying theory of both Rate Distortion Theory and Transfer Entropy are explained in the context of recent applications in signal processing and biology.