Published March 25, 2011
| Version 10330
Journal article
Open
Learning an Overcomplete Dictionary using a Cauchy Mixture Model for Sparse Decay
Authors/Creators
Description
An algorithm for learning an overcomplete dictionary
using a Cauchy mixture model for sparse decomposition of an underdetermined
mixing system is introduced. The mixture density
function is derived from a ratio sample of the observed mixture
signals where 1) there are at least two but not necessarily more
mixture signals observed, 2) the source signals are statistically
independent and 3) the sources are sparse. The basis vectors of the
dictionary are learned via the optimization of the location parameters
of the Cauchy mixture components, which is shown to be more
accurate and robust than the conventional data mining methods
usually employed for this task. Using a well known sparse
decomposition algorithm, we extract three speech signals from two
mixtures based on the estimated dictionary. Further tests with
additive Gaussian noise are used to demonstrate the proposed
algorithm-s robustness to outliers.
Files
10330.pdf
Additional details
References
- M. S. Lewicki and T. J. Sejnowski, "Learning Overcomplete Representations, " Neural Computations, vol. 12, No. 2, pp. 337-365, February 2000.
- M. Zhong, H. Tang, H. Cheng and Y. Tang, "An EM Algorithm for Learning Sparse and Overcomplete Representations, " Neurocomputing, vol. 57, pp. 467-476, 2004.
- M. Aharon, M. Elad and A. Bruckstein, "K-SVD: An algorithm for Designing Overcomplete Dictionaries for Sparse Representations," IEEE Transactions on Signal Processing, vol. 54, No. 11, November 2006.
- K. Kreutz-Delgado, J. F. Murray, B. D. Rao, K. Egan, T. Lee and T. J. Sejnowski, "Dictionary Learning Algorithms for Sparse Representations, " eural Computations, vol. 15, No. 2, pp. 349-396, 2003.
- K. Egan, S. O. Aase and J. H. Hakon-Husoy, "Method of Optimal Directions for Frame Design, "in IEEE International Conference of Acoustic, Speech and Signal Processing, Vol. 5, pp. 2443-2446, 1999.
- I. T. Jollife, Principal Component Analysis, Series: Springer Series in Statistics, 2nd Edition, .
- P. Common, "Independent Component Analysis: A new Concept?, " Signal Processing, vol. 36, pp. 287-314, 1994.
- G. Marsaglia, "Ratios of Normal Variables, " Journal of Statistical Software, vol. 16, No. 4, May 2006.
- W. Feller, An Introduction to Probability Theory and Applications, 2nd Edition, New York: Wiley, pp. 229-235, 1968. [10] P. G. Hoel, Introduction to Mathematical Statistics, 3rd Edition, New York: Wiley, pp. 57-62, 1962. [11] J. W. Harris and H. Stoker, "Maximum Likelihood Method, " in Handbook of Mathematics and Computational Science, New York: Springer-Verlag, pp. 824-835, 1998. [12] A. Koutrouvelis, "Estimation of the Location and Scale of the Cauchy Distribution using the Empirical Characteristic Function, " Biometrica, Vol. 69, pp. 205-213, 1982. [13] F. Nagy, "Parameter Estimation of the Cauchy Distribution in Information Theory Approach, "Journal of Universal Computer Science, Vol. 12, No. 9, pp. 1332-1344, May 2006. [14] G. B. Freue, "The Pitman Estimator of the Cauchy Location Parameter, " Journal of Statistical Planning and Inference, vol. 137, pp. 1900-1913, May 2006. [15] K. M. Hanson and D. R. Wolf, "Estimators for the Cauchy Distribution, " In Maximum Entropy and Bayesian Methods, pp. 255-263, 1996. [16] C. Jutten, G. H. Mohimani and M. B. Zadeh, "Fast Sparse Representations based on the l0 Norm, " In Pro. ICA-07, London, UK, 2007. [17] G. H. Mohimani, M. B. Zaden and C. Jutten, "Complex Valued Sparse Representations based on Smoothed l0 Norm, " In ICASSP-08, 2008. [18] S. S. Chen, D. L. Donoho and M. A. Saunders, "Atomic Decomposition by Basis Pursuit, " SIAM Journal of Computing, Vol. 20, No. 1, pp. 33- 61, 1999. [19] S. G. Mallat and Z. Zhang, "Matching Pursuit with Time-Frequency Dictionaries, " IEEE Transactions on Signal Processing, pp. 3397-3415, December 1993. [20] I. F. Gorodnitsky and B. D. Rao, "Sparse Signal Reconstruction from Limited Data using FOCUSS: A Re-weighted Minimum Norm Algorithm, " IEEE Transactions on Signal Processing, Vol. 45, No. 3, March 1997. [21] J. E. Marsden and J. M. Hoffman, Basic Complex Analysis, 3rd Edition, W. H. Freeman, 1998, ISBN: 978-0716728771.