Sparsity Research Papers - Academia.edu (original) (raw)

In recent years, compressed sensing (CS) has attracted considerable attention in areas of applied mathematics, computer science, and electrical engineering by suggesting that it may be possible to surpass the traditional limits of... more

In recent years, compressed sensing (CS) has attracted considerable attention in areas of applied mathematics, computer science, and electrical engineering by suggesting that it may be possible to surpass the traditional limits of sampling theory. CS builds upon the fundamental fact that we can represent many signals using only a few non-zero coefficients in a suitable basis or dictionary. Nonlinear optimization can then enable recovery of such signals from very few measurements. In this chapter, we provide an up-to-date review of the basic theory underlying CS. After a brief historical overview, we begin with a discussion of sparsity and other low-dimensional signal models. We then treat the central question of how to accurately recover a high-dimensional signal from a small set of measurements and provide performance guarantees for a variety of sparse recovery algorithms. We conclude with a discussion of some extensions of the sparse recovery framework. In subsequent chapters of the book, we will see how the fundamentals presented in this chapter are extended in many exciting directions, including new models for describing structure in both analog and discrete-time signals, new sensing design techniques, more advanced recovery results, and emerging applications.

The general trend to compress signals after they have been completely recovered is no longer the most effective method in signal processing and communication. Since most signals are compressible, we can collect fewer measurements and... more

The general trend to compress signals after they have been completely recovered is no longer the most effective method in signal processing and communication. Since most signals are compressible, we can collect fewer measurements and recover only the necessary information to maximize the efficacy of the sampling and reconstruction system. Compressive Sampling uses the concept of signal sparsity for an n dimensional signal and develops algorithms to reconstruct the signal from m<n measurements. This report describes various conditions on the sparsity S, the measurements m and the restricted isometry property (RIP) on a sensing matrix A; and attempts to find a solution to the Basis Pursuit optimization program for exact S-sparse and near S-sparse vectors under noisy conditions. Also, a strong emphasis is given on random matrices, incoherence of basis and it is proved that random sensing is the key to acquire fewer measurements. Based on the theoretical concepts, the report describes emerging applications along with two hardware architectures that utilize robust sensing design techniques. The content of this report is based on the study of the paper An Introduction to Compressive Sampling by Candes et al. and Wakin et. al. and thus the scope of the report is limited to discrete time signals.

A prerequisite for implementing collaborative filtering recommender systems is the availability of users' preferences data. This data, typically in the form of ratings, is exploited to learn the tastes of the users and to serve them with... more

A prerequisite for implementing collaborative filtering recommender systems is the availability of users' preferences data. This data, typically in the form of ratings, is exploited to learn the tastes of the users and to serve them with personalized recommendations. However, there may be a lack of preference data, especially at the initial stage of the operations of a recommender system, i.e., in the Cold Start phase. In particular, when a new user has not yet rated any item, the system would be incapable of generating relevant recommendations for this user. Or, when a new item is added to the system catalogue and no user has rated it, the system cannot recommend this item to any user. This chapter discusses the cold start problem and provides a comprehensive description of techniques that have been proposed to address this problem. It surveys algorithmic solutions and provides a summary of their performance comparison. Moreover, it lists publicly available resources (e.g., libraries and datasets) and offers a set of practical guidelines that can be adopted by researchers and practitioners.

The increasing growth of the World Wide Web especially in a social network with the multiplicity of items offered (such as products or web pages), it is really difficult for a user to pick up relevant items who is searching for it. On the... more

The increasing growth of the World Wide Web especially in a social network with the multiplicity of items offered (such as products or web pages), it is really difficult for a user to pick up relevant items who is searching for it. On the other hand, the different tastes and behaviors of users is making the probability for finding a neighbor user hard to get, therefore, difficult for automated software systems to discover what is interesting to the user. We have proposed a new approach to adapt to this widespread in e-commerce nowadays and reduce the impact of the multiplicity of items and different views of the users that can quickly produce the recommendations through, exploit the domain knowledge of training data set to create testing data set depending on attributes of one feature that represents the distinctive genres of item as the inputs to a hybrid recommender systems which is aspired to achieve best recommendations by implementing meta-level hybridization techniques that combine of collaborative recommender systems and content-based recommender systems, these operations will reduce from the effects of sparsity, cold start and scalability very common problems with the collaborative recommender systems additional to improve the accuracy of recommendation comparing with the pure collaborative filtering Pearson Correlation approach.

Least squares support vector machines (LSSVMs) have been widely applied for classification and regression with comparable performance with SVMs. The LSSVM model lacks sparsity and is unable to handle large-scale data due to computational... more

Least squares support vector machines (LSSVMs) have been widely applied for classification and regression with comparable performance with SVMs. The LSSVM model lacks sparsity and is unable to handle large-scale data due to computational and memory constraints. A primal fixed-size LSSVM (PFS-LSSVM) introduce sparsity using Nyström approximation with a set of prototype vectors (PVs). The PFS-LSSVM model solves an overdetermined system of linear equations in the primal. However, this solution is not the sparsest. We investigate the sparsity-error tradeoff by introducing a second level of sparsity. This is done by means of L0-norm-based reductions by iteratively sparsifying LSSVM and PFS-LSSVM models. The exact choice of the cardinality for the initial PV set is not important then as the final model is highly sparse. The proposed method overcomes the problem of memory constraints and high computational costs resulting in highly sparse reductions to LSSVM models. The approximations of t...

We apply Non-negative Matrix Factorization (NMF) to the prob- lem of identifying underlying trends in stock market data. NMF is a recent and very successful tool for data analysis including image and audio processing; we use it here to... more

We apply Non-negative Matrix Factorization (NMF) to the prob- lem of identifying underlying trends in stock market data. NMF is a recent and very successful tool for data analysis including image and audio processing; we use it here to decompose a mixture a data, the daily closing prices of the 30 stocks which make up the Dow Jones In- dustrial

1 adsetiawan@students.itb.ac.id, 2 suksmono@stei.itb.ac.id, 3 hgunawan@math.itb.ac.id, 4 tmengko@itb.ac.id Abstrak Teknologi multimedia berkembang terus dan semakin banyak digunakan dalam konten komunikasi. Konten multimedia memungkinkan interaksi yang lebih kaya jika diandingkan dengan konten tekstual. Citra merupakan komponen komunikasi yang banyak digunakan. Pertumbuhan penggunaan citra sebagai konten komunikasi menyebabkan meningkatnya kebutuhan infrastruktur telekomunikasi pita lebar. Pengembangan algoritma pemampatan citra dapat dimanfaatkan untuk mengifisienkan penggunaan lebar pita telekomunikasi. Algoritma pemampatan merugi (lossy) memberikan rasio pemampatan yang lebih tinggi dibandingkan algoritma pemampatan tak merugi (lossless). Algoritma pemampatan merugi memanfaatkan sifat sparsitas dari citra. Citra akan ditransformasikan ke domain basis tertentu yang merepresentasikan citra tersebut lebih sparse (kompresif). Basis yang diusulkan dalam penelitian ini adalah basis latih K-SVD. Penggunaan basis latih ini dapat mentransformasikan citra ke domain yang lebih kompresif. Hal ini disebabkan karena penggunaan pustaka latih lebih mengeksplorasi sifat-sifat dari citra. Metoda pemampatan yang dikembangkan merupakan pengembangan dari kuantisasi vektor yang diperumum, sehingga memungkinkan representasi citra dari kombinasi linear beberapa fungsi basis dengan jumlah basis yang tetap (sparsitas tetap). Metoda sparsitas tetap yang diusulkan akan dibandingkan dengan kuantisasi vektor fuzzy (Scalable Fuzzy Vektor Quantization) dan JPEG pada laju bit yang rendah untuk mengefisienkan penggunaan lebar pita telekomunikasi. Kata kunci : representasi sparse, K-SVD, kuantisasi vektor, laju bit rendah Abstract Multimedia technology continues to evolve and is increasingly used in the content of communication. Multimedia content enables richer interaction compared to textual content. The image is a component of widely used communications. The growth of image usage as communication content led to a growing need for broadband telecommunications infrastructure. The development of image compression algorithms can be used to optimize telecommunications bandwidth usage more efficiently. Lossy compression algorithm gives higher compression ratio than a lossless one. Lossy compression algorithm exploits the nature of image sparsity. The image will be transformed into certain basis domain to represent the image more sparsely. The proposed basis are trained-basis based on K-SVD. The proposed trained-basis capable of transforming the image into a more compressible domain. The reason is that the trained-basis dictionary explore the prior of the image. The developed compression method is based on generalized vector quantization, allowing the representation of image over a linear combination of some basis functions. The proposed method is compared with fuzzy vector quantization (Scalable Fuzzy Vector Quantization) and JPEG at low bit rate to make efficient use of telecommunications bandwidth.

Several authors have shown that it is possible to reconstruct exactly a sparse signal from a fewer linear measurements, this method known as compressed sensing (CS). CS aim to reconstruct signals and images from significantly fewer... more

Several authors have shown that it is possible to reconstruct exactly a sparse signal from a fewer linear measurements, this method known as compressed sensing (CS). CS aim to reconstruct signals and images from significantly fewer measurements. With CS it‟s possible to make an accurate reconstruction from small number of samples (measurements). Doppler ultrasound is an important technique for non-invasively detecting and measuring the velocity of moving structure, and particularly blood, within the body. Doppler ultrasound signal has been reconstructed with CS by using random sampling and non-uniform sampling via ℓ1-norm to generate Doppler sonogram. The result show that the recovered signals with non-uniform sampling are the same as the original signal, there is a loss of very small peaks, when random sampling used for recovering the signals, there is no significant different between the original signal and reconstructed one when we used more than 85 % of the data, when less than ...

This paper jointly addresses the problems of chromatogram baseline correction and noise reduction. The proposed approach is based on modeling the series of chromatogram peaks as sparse with sparse derivatives, and on modeling the baseline... more

This paper jointly addresses the problems of chromatogram baseline correction and noise reduction. The
proposed approach is based on modeling the series of chromatogram peaks as sparse with sparse derivatives, and on modeling the baseline as a low-pass signal. A convex optimization problem is formulated so as to encapsulate these non-parametric models. To account for the positivity of chromatogram peaks, an
asymmetric penalty function is utilized. A robust, computationally ecient, iterative algorithm is developed
that is guaranteed to converge to the unique optimal solution. The approach, termed Baseline Estimation
And Denoising with Sparsity (BEADS), is evaluated

The l1/l2 ratio regularization function has shown good performance for retrieving sparse signals in a number of recent works, in the context of blind deconvolution. Indeed, it benefits from a scale invariance property much desirable... more

The l1/l2 ratio regularization function has shown good performance for retrieving sparse signals in a
number of recent works, in the context of blind deconvolution. Indeed, it benefits from a scale invariance
property much desirable in the blind context. However, the l1/l2 function raises some difficulties
when solving the nonconvex and nonsmooth minimization problems resulting from the use of such
a penalty term in current restoration methods. In this paper, we propose a new penalty based on a
smooth approximation to the l1/l2 function. In addition, we develop a proximal-based algorithm to
solve variational problems involving this function and we derive theoretical convergence results. We
demonstrate the effectiveness of our method through a comparison with a recent alternating optimization
strategy dealing with the exact l1/l2 term, on an application to seismic data blind deconvolution.

This paper jointly addresses the problems of chromatogram baseline correction and noise reduction. The proposed approach is based on modeling the series of chromatogram peaks as sparse with sparse derivatives, and on modeling the baseline... more

This paper jointly addresses the problems of chromatogram baseline correction and noise reduction. The proposed approach is based on modeling the series of chromatogram peaks as sparse with sparse derivatives, and on modeling the baseline as a low-pass signal. A convex optimization problem is formulated so as to encapsulate these non-parametric models. To account for the positivity of chromatogram peaks, an asymmetric penalty function is utilized. A robust, computationally efficient, iterative algorithm is developed that is guaranteed to converge to the unique optimal solution. The approach, termed Baseline Estimation And Denoising with Sparsity (BEADS), is evaluated and compared with two state-of-the-art methods using both simulated and real chromatogram data.

The Common Spatial Pattern (CSP) method is a powerful technique for feature extraction from multichannel neural activity and widely used in brain computer interface (BCI) applications. By linearly combining signals from all channels, it... more

The Common Spatial Pattern (CSP) method is a powerful technique for feature extraction from multichannel neural activity and widely used in brain computer interface (BCI) applications. By linearly combining signals from all channels, it maximizes variance for one condition while minimizing for the other. However, the method overfits the data in presence of dense recordings and limited amount of training data. To overcome this problem we construct a sparse CSP (sCSP) method such that only subset of channels contributes to feature extraction. The sparsity is achieved by a greedy search based generalized eigenvalue decomposition approach with low computational complexity. Our contributions in this study are extension of the greedy search based solution to have multiple sparse filters and its application in a BCI problem. We show that sCSP outperforms traditional CSP in the classification challenge of the multichannel ECoG data set of BCI competition 2005. Furthermore, it achieves nearly similar performance as infeasible exhaustive search and better than that of obtained by LI norm based sparse solution.

A common problem with OnLine Analytical Processing (OLAP) databases is data explosion data size multiplies, when it is loaded from the source data into multidimensional cubes. Data explosion is not an issue for small databases, but can be... more

A common problem with OnLine Analytical Processing (OLAP) databases is data explosion data size multiplies, when it is loaded from the source data into multidimensional cubes. Data explosion is not an issue for small databases, but can be serious problems with large databases. In this paper we discuss the sparsity and data explosion phenomenon in multidimensional data model, which lie at the core of OLAP systems. Our researches over five companies with different branch of business confirm the observations that in reality most of the cubes are extremely sparse. We also consider a different method that relational and multidimensional severs applies to reduce the data explosion and sparsity problems as compression and indexes techniques, partitioning, preliminary aggregations.

Starting with the seminal papers of Reynolds (1987), Vicsek et. al. (1995), Cucker{Smale (2007) there has been a ood of recent works on models of self-alignment and consensus dynamics. Self-organization has been so far the main driving... more

Starting with the seminal papers of Reynolds (1987), Vicsek et. al. (1995), Cucker{Smale
(2007) there has been a
ood of recent works on models of self-alignment and consensus dynamics.
Self-organization has been so far the main driving concept of this research direction. However, the
evidence that in practice self-organization does not necessarily occur (for instance, the achievement
of unanimous consensus in government decisions) leads to the natural question of whether it is
possible to externally in
uence the dynamics in order to promote the formation of certain desired
patterns. Once this fundamental question is posed, one is also faced with the issue of de ning
the best way of obtaining the result, seeking for the most \economical" way to achieve a certain
outcome. Our paper precisely addressed the issue of nding the sparsest control strategy in order
to lead us optimally towards a given outcome, in this case the achievement of a state where the
group will be able by self-organization to reach an alignment consensus. As a consequence we
provide a mathematical justi cation to the general principle according to which \sparse is better":
in order to achieve group consensus, a policy maker not allowed to predict future developments
should decide to control with stronger action the fewest possible leaders rather than trying to act
on more agents with minor strength. We then establish local and global sparse controllability
properties to consensus. Finally, we analyze the sparsity of solutions of the nite time optimal
control problem where the minimization criterion is a combination of the distance from consensus
and of the `1-norm of the control. Such an optimization models the situation where the policy
maker is actually allowed to observe future developments. We show that the lacunarity of sparsity
is related to the codimension of certain manifolds in the space of cotangent vectors.

The richness of natural images makes the quest for optimal representations in image processing and computer vision challenging. The latter observation has not prevented the design of image representations, which trade off between... more

The richness of natural images makes the quest for optimal representations in image processing and computer vision challenging. The latter observation has not prevented the design of image representations, which trade off between efficiency and complexity, while achieving accurate rendering of smooth regions as well as reproducing faithful contours and textures. The most recent ones, proposed in the past decade, share an hybrid heritage highlighting the multiscale and oriented nature of edges and patterns in images. This paper presents a panorama of the aforementioned literature on decompositions in multiscale, multi-orientation bases or dictionaries. They typically exhibit redundancy to improve sparsity in the transformed domain and sometimes its invariance with respect to simple geometric deformations (translation, rotation). Oriented multiscale dictionaries extend traditional wavelet processing and may offer rotation invariance. Highly redundant dictionaries require specific algorithms to simplify the search for an efficient (sparse) representation. We also discuss the extension of multiscale geometric decompositions to non-Euclidean domains such as the sphere or arbitrary meshed surfaces. The etymology of panorama suggests an overview, based on a choice of partially overlapping "pictures". We hope that this paper will contribute to the appreciation and apprehension of a stream of current research directions in image understanding.

We propose a new generalized thresholding algorithm useful for inverse problems with sparsity constraints. The algorithm uses a thresholding function with a parameter p, first mentioned in [1]. When p = 1, the thresholding function is... more

We propose a new generalized thresholding algorithm useful for inverse problems with sparsity constraints. The algorithm uses a thresholding function with a parameter p, first mentioned in [1]. When p = 1, the thresholding function is equivalent to classical soft thresholding. For values of p below 1, the thresholding penalizes small coefficients over a wider range and applies less bias to the larger coefficients, much like hard thresholding but without discontinuities. The functional that the new thresholding minimizes is non-convex for p