Redundancy - Ubu@50 - 50 Ubus
When a message is transmitted across a channel, in bib⁄Shannon’s terminology, it is beneficial (in terms of bandwidth scarcity) to compress the message into the shortest possible code. This is only feasible because natural language features redundancy to clarify meaning and avoid misunderstanding. Given a fixed context, agreed upon on both sides of the channel, the redundancy can be eliminated. The context implies a distribution of possible messages, from which an optimal encoding can be deduced, as determined by Shannon’s theory of information. Schematically, in a military context, the context in which most information technology originated, the options “launch the missile” and “retreat the troops” can be reduced to a single bit: 0 or 1. It is critical to note that this is context-dependent. Compression is conditional on who is receiving the message, a process that is reified through deterministic encoders and decoders, e.g. h264 for video transmission. Nonetheless, redundancy is not
Redundancy - Ubu@50 - 50 Ubus EDIT_THIS ADD_MIRROR ADD_REFLECTION ADD_SHARD PUBLISH ? S tage: Everything Is Temporary » An Autonomous Archive » Redundancy Redundancy shard⁄R edundancy glassblower⁄ Cristóbal Sciutto Rodríguez in reflections⁄ An Autonomous Archive When a message is transmitted across a channel, in bib⁄ Shannon 's terminology, it is beneficial (in terms of bandwidth scarcity) to compress the message into the shortest possible code. This is only feasible because natural language features redundancy to clarify meaning and avoid misunderstanding. Given a fixed context, agreed upon o
related reading
- A Mathematical Theory of Communicationpeople.math.harvard.edu
- [2206.07867] A visual introduction to information theoryarxiv.org
- Visual Information Theory -- colah's blogcolah.github.io
- book.pdfinference.org.uk
- How Claude Shannon Invented the Future | Quanta Magazinequantamagazine.org
- How Lossless Data Compression Works | Quanta Magazinequantamagazine.org
- Entropy coding - Wikipediaen.wikipedia.org
- Sending Samples Without Bits-Backjoschu.net
- Information entropy (Foundations of information theory: Part 2) - Matthew N. Bernsteinmbernste.github.io
- John Gordon: The Alice and Bob After Dinner Speechurbigenous.net
- Lossless compressionen.wikipedia.org
- [physics/0004057] 1 Introductionar5iv.labs.arxiv.org