A Technical Explanation of Technical Explanation
©2005 by Eliezer Yudkowsky.
--------------------------------------------------------------------------------
This essay is meant for a reader who has attained a firm grasp of Bayes' Theorem. An introduction to Bayes' Theorem may be found at An Intuitive Explanation of Bayesian Reasoning. You should easily recognize, and intuitively understand, the concepts "prior probability", "posterior probability", "likelihood ratio", and "odds ratio". This essay is intended as a sequel to the Intuitive Explanation, but you might skip that introduction if you are already thoroughly Bayesian. Where the Intuitive Explanation focused on providing a firm grasp of Bayesian basics, the Technical Explanation builds, on a Bayesian foundation, theses about human rationality and philosophy of science.
The Intuitive Explanation of Bayesian Reasoning promised that mastery of addition, multiplication, and division would be sufficient background, with no subtraction required. To this the Technical Explanation of Technical Explanation adds logarithms. The math is simple, but necessary, and it appears first in the order of exposition. Some pictures may not be drawn with words alone.
As Jaynes (1996) emphasizes, the theorems of Bayesian probability theory are just that, mathematical theorems which follow inevitably from Bayesian axioms. One might naively think that there would be no controversy about mathematical theorems. But when do the theorems apply? How do we use the theorems in real-world problems? The Intuitive Explanation tries to avoid controversy, but the Technical Explanation willfully walks into the whirling helicopter blades. Bluntly, the reasoning in the Technical Explanation does not represent the unanimo ...