Formal Bayesian Theory of Surprise Home Page

For a good summary of the story which does not require any maths background, have a look at two press releases by Eric Mankin here and here. We propose that surprise is a general, information-theoretic concept, which can be derived from first principles and formalized analytically across spatio-temporal scales, sensory modalities, and, more generally, data types and data sources. Two elements are essential for a principled definition of surprise. First, surprise can exist only in the presence of uncertainty, which can arise from intrinsic stochasticity, missing information, or limited computing resources. A world that is purely deterministic and predictable in real-time for a given observer contains no surprises. Second, surprise can only be defined in a relative, subjective, manner and is related to the expectations of the observer, be it a single synapse, neuronal circuit, organism, or computer device. The same data may carry different amounts of surprise for different observers, or

For a good summary of the story which does not require any maths background, have a look at two press releases by Eric Mankin here and here. We propose that surprise is a general, information-theoretic concept, which can be derived from first principles and formalized analytically across spatio-temporal scales, sensory modalities, and, more generally, data types and data sources. Two elements are essential for a principled definition of surprise. First, surprise can exist only in the presence of uncertainty, which can arise from intrinsic stochasticity, missing information, or limited computin