Coherence — sparse-plex v2019.02
Finding out the spark of a dictionary 𝐷 is NP-hard since it involves considering combinatorially large number of selections of columns from 𝐷 . In this section we consider the coherence of a dictionary which is computationally tractable and quite useful in characterizing the solutions of sparse approximation problems. The coherence of a dictionary 𝐷 is defined as the maximum absolute inner product between two distinct atoms in the dictionary: If the dictionary consists of two orthonormal bases, then coherence is also known as mutual coherence or proximity. Since the atoms within each orthonormal basis are orthogonal to each other, the coherence is determined only by the inner products of atoms from one basis with another basis. We note that d ω i 𝑑 𝜔 𝑖 is the i 𝑖 -th column of synthesis matrix 𝐷 . Also H 𝐷 𝐻 𝐷 is the Gram matrix for 𝐷 whose elements are nothing but the inner-products of columns of 𝐷 . We note that by definition ‖ d ω ‖ 2
Coherence - sparse-plex v2019.02 --> Docs >> Sparse Signal Models >> Coherence Coherence ¶ Finding out the spark of a dictionary \(\DDD\) is NP-hard since it involves considering combinatorially large number of selections of columns from \(\DDD\) . In this section we consider the coherence of a dictionary which is computationally tractable and quite useful in characterizing the solutions of sparse approximation problems. Definition The coherence of a dictionary \(\DDD\) is defined as the maximum absolute inner product between two distinct atoms in the dictionary: \[\mu = \underset{j \neq k}{\t
related reading
- [2608.27540] Towards a mathematical theory of superpositionarxiv.org
- Singular value decomposition - Wikipediaen.wikipedia.org
- https://www.deeplearningbook.org/contents/linear_algebra.htmldeeplearningbook.org
- Proofs, beliefs and algorithms through the lens of Sum of Squaressumofsquares.org
- 1404.5236 Sum-of-Squares Proofs and the Quest toward Optimal Algorithmsarxiv.org
- MathofSNLnotes2025.pdfpeople.math.ethz.ch
- [2605.01192] Linear-Readout Floors and Threshold Recovery in Computation in Superpositionarxiv.org
- Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
- Restricted isometry property - Wikipediaen.wikipedia.org
- A short survey on almost orthogonal vectors in a few specific large dimensionsarxiv.org
- notes.pdfjingbol.web.illinois.edu
- 1409.6366arxiv.org