Pavan Turaga | Arizona State University (original) (raw)

Papers by Pavan Turaga

Procedings of the Proceedings of the 1st International Workshop on DIFFerential Geometry in Computer Vision for Analysis of Shapes, Images and Trajectories 2015, 2015

Persistent surveillance from camera networks, such as at parking lots, UAVs, etc., often results ... more Persistent surveillance from camera networks, such as at parking lots, UAVs, etc., often results in large amounts of video data, resulting in significant challenges for inference in terms of storage, communication and computation. Compressive cameras have emerged as a potential solution to deal with the data deluge issues in such applications. However, inference tasks such as action recognition require high quality features which implies reconstructing the original video data. Much work in compressive sensing (CS) theory is geared towards solving the reconstruction problem, where state-of-the-art methods are computationally intensive and provide low-quality results at high compression rates. Thus, reconstruction-free methods for inference are much desired. In this paper, we propose reconstruction-free methods for action recognition from compressive cameras at high compression ratios of 100 and above. Recognizing actions directly from CS measurements requires features which are mostly nonlinear and thus not easily applicable. This leads us to search for such properties that are preserved in compressive measurements. To this end, we propose the use of spatio-temporal smashed filters, which are compressive domain versions of pixel-domain matched filters. We conduct experiments on publicly available databases and show that one can obtain recognition rates that are comparable to the oracle method in uncompressed setup, even for high compression ratios.

In this paper, we consider the problem of fast and efficient indexing techniques for sequences ev... more In this paper, we consider the problem of fast and efficient indexing techniques for sequences evolving in non-Euclidean spaces. This problem has several applications in the areas of human activity analysis, where there is a need to perform fast search, and recognition in very high dimensional spaces. The problem is made more challenging when representations such as landmarks, contours, and human skeletons etc. are naturally studied in a non-Euclidean setting where even simple operations are much more computationally intensive than their Euclidean counterparts. We propose a geometry and data adaptive symbolic framework that is shown to enable the deployment of fast and accurate algorithms for activity recognition, dynamic texture recognition, motif discovery. Toward this end, we present generalizations of key concepts of piece-wise aggregation and symbolic approximation for the case of non-Euclidean manifolds. We show that one can replace expensive geodesic computations with much faster symbolic computations with little loss of accuracy in activity recognition and discovery applications. The framework is general enough to work across both Euclidean and non-Euclidean spaces, depending on appropriate feature representations without compromising on the ultra-low bandwidth, high speed and high accuracy. The proposed methods are ideally suited for real-time systems and low complexity scenarios.

2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2015

Spatial-multiplexing cameras have emerged as a promising alternative to classical imaging devices... more Spatial-multiplexing cameras have emerged as a promising alternative to classical imaging devices, often enabling acquisition of 'more for less'. One popular architecture for spatial multiplexing is the single-pixel camera (SPC), which acquires coded measurements of the scene with pseudorandom spatial masks. Significant theoretical developments over the past few years provide a means for reconstruction of the original imagery from coded measurements at sub-Nyquist sampling rates. Yet, accurate reconstruction generally requires high measurement rates and high signal-tonoise ratios. In this paper, we enquire if one can perform high-level visual inference problems (e.g. face recognition or action recognition) from compressive cameras without the need for image reconstruction. This is an interesting question since in many practical scenarios, our goals extend beyond image reconstruction. However, most inference tasks often require non-linear features and it is not clear how to extract such features directly from compressed measurements. In this paper, we show that one can extract nontrivial correlational features directly without reconstruction of the imagery. As a specific example, we consider the problem of face recognition beyond the visible spectrum e.g in the short-wave infra-red region (SWIR) -where pixels are expensive. We base our framework on smashed filters which suggests that inner-products between high-dimensional signals can be computed in the compressive domain to a high degree of accuracy. We collect a new face image dataset of 30 subjects, obtained using an SPC. Using face recognition as an example, we show that one can indeed perform reconstruction-free inference with a very small loss of accuracy at very high compression ratios of 100 and more.

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015

Persistent surveillance from camera networks, such as at parking lots, UAVs, etc., often results ... more Persistent surveillance from camera networks, such as at parking lots, UAVs, etc., often results in large amounts of video data, resulting in significant challenges for inference in terms of storage, communication and computation. Compressive cameras have emerged as a potential solution to deal with the data deluge issues in such applications. However, inference tasks such as action recognition require high quality features which implies reconstructing the original video data. Much work in compressive sensing (CS) theory is geared towards solving the reconstruction problem, where state-of-the-art methods are computationally intensive and provide low-quality results at high compression rates. Thus, reconstruction-free methods for inference are much desired. In this paper, we propose reconstruction-free methods for action recognition from compressive cameras at high compression ratios of 100 and above. Recognizing actions directly from CS measurements requires features which are mostly nonlinear and thus not easily applicable. This leads us to search for such properties that are preserved in compressive measurements. To this end, we propose the use of spatio-temporal smashed filters, which are compressive domain versions of pixel-domain matched filters. We conduct experiments on publicly available databases and show that one can obtain recognition rates that are comparable to the oracle method in uncompressed setup, even for high compression ratios.

Spatio-temporal patterns abound in the real world, and understanding them computationally holds t... more Spatio-temporal patterns abound in the real world, and understanding them computationally holds the promise of enabling a large class of applications such as video surveillance, biometrics, computer graphics and animation. In this dissertation, we study models and algorithms to describe complex spatio-temporal patterns in videos for a wide range of applications.

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015

In this paper, we consider the Précis problem of sampling K representative yet diverse data point... more In this paper, we consider the Précis problem of sampling K representative yet diverse data points from a large dataset. This problem arises frequently in applications such as video and document summarization, exploratory data analysis, and pre-filtering. We formulate a general theory which encompasses not just traditional techniques devised for vector spaces, but also non-Euclidean manifolds, thereby enabling these techniques to shapes, human activities, textures and many other image and video based datasets. We propose intrinsic manifold measures for measuring the quality of a selection of points with respect to their representative power, and their diversity. We then propose efficient algorithms to optimize the cost function using a novel annealing-based iterative alternation algorithm. The proposed formulation is applicable to manifolds of known geometry as well as to manifolds whose geometry needs to be estimated from samples. Experimental results show the strength and generality of the proposed approach.

Spatial-multiplexing cameras have emerged as a promising alternative to classical imaging devices... more Spatial-multiplexing cameras have emerged as a promising alternative to classical imaging devices, often enabling acquisition of 'more for less'. One popular architecture for spatial multiplexing is the single-pixel camera (SPC), which acquires coded measurements of the scene with pseudo-random spatial masks. Significant theoretical developments over the past few years provide a means for reconstruction of the original imagery from coded measurements at sub-Nyquist sampling rates. Yet, accurate reconstruction generally requires high measurement rates and high signal-to-noise ratios. In this paper, we enquire if one can perform high-level visual inference problems (e.g. face recognition or action recognition) from compressive cameras without the need for image reconstruction. This is an interesting question since in many practical scenarios, our goals extend beyond image reconstruction. However, most inference tasks often require non-linear features and it is not clear how to...

International Journal of Computer Vision

In this paper, we consider the problem of fast and efficient indexing techniques for human activi... more In this paper, we consider the problem of fast and efficient indexing techniques for human activity sequences evolving in Euclidean and non-Euclidean spaces. This problem has several applications in the areas of human activity analysis, where there is a need to perform fast search and recognition in large databases. The problem is made more challenging when features such as landmarks, contours, and stick-figures etc. are naturally studied in a non-Euclidean setting where even simple operations are much more computationally intensive than their Euclidean counterparts. We propose a geometry and data adaptive symbolic framework that is shown to enable the deployment of fast and accurate algorithms for activity recognition and motif discovery. Toward this end, we present generalizations of key concepts of piece-wise aggregation and symbolic approximation for the case of non-Euclidean manifolds. We show that one can replace expensive geodesic computations with much faster symbolic comput...

This paper concerns itself with compression strategies for orientation signals, seen as signals e... more This paper concerns itself with compression strategies for orientation signals, seen as signals evolving on the space of quaternions. The compression techniques extend classical signal approximation strategies used in data mining, by explicitly taking into account the quotient-space properties of the quaternion space. The approximation techniques are applied to the case of human gesture recognition from cellphone-based orientation sensors. Results indicate that the proposed approach results in high recognition accuracies, with low storage requirements, with the geometric computations providing added robustness than classical vector-space computations.

Procedings of the Proceedings of the 1st International Workshop on DIFFerential Geometry in Computer Vision for Analysis of Shapes, Images and Trajectories 2015, 2015

2015 Data Compression Conference, 2015

This paper concerns itself with compression strategies for orientation signals, seen as signals e... more This paper concerns itself with compression strategies for orientation signals, seen as signals evolving on the space of quaternions. The compression techniques extend classical signal approximation strategies used in data mining, by explicitly taking into account the quotientspace properties of the quaternion space. The approximation techniques are applied to the case of human gesture recognition from cellphone-based orientation sensors. Results indicate that the proposed approach results in high recognition accuracies, with low storage requirements, with the geometric computations providing added robustness than classical vector-space computations.

International Journal of Computer Vision, 2015

Clustering video sequences in order to infer and extract events from a single video stream is an ... more Clustering video sequences in order to infer and extract events from a single video stream is an extremely impor- tant problem and has significant potential in video index- ing, surveillance, activity discovery and event recognition. Clustering a video sequence into events requires one to si- multaneously recognize event boundaries (event consistent subsequences) and cluster these event subsequences. In or- der

2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014

ABSTRACT The compressive sensing paradigm holds promise for more cost-effective imaging outside o... more ABSTRACT The compressive sensing paradigm holds promise for more cost-effective imaging outside of the visible range, particularly in infrared wavelengths. However, the process of reconstructing compressively sensed images remains computationally expensive. The proof-of-concept tracker described here uses a particle filter with a likelihood update based on a “smashed filter” which estimates correlation directly, avoiding the reconstruction step. This approach leads to increased noise in correlation estimates, but by implementing the track-before-detect concept in the particle filter, tracker convergence may still be achieved with reasonable sensing rates. The tracker has been successfully tested on sequences of moving cars in the PETS2000 dataset.

2004 IEEE Region 10 Conference TENCON 2004., 2004

We present a system for automatic detection and tracking of faces in video sequences. Detection i... more We present a system for automatic detection and tracking of faces in video sequences. Detection is done based on a statistical characterization of skin-color. The position and size of the dominant face are estimated using statistical means of the binary map projections. Tracking is done using a Kalman filter. We propose a novel technique for updating the process noise covariance

2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010

Flow Doppler imaging has become an integral part of an echocardiographic exam. Automated interpre... more Flow Doppler imaging has become an integral part of an echocardiographic exam. Automated interpretation of flow doppler imaging has so far been restricted to obtaining hemodynamic information from velocity-time profiles depicted in these images. In this paper we exploit the shape patterns in Doppler images to infer the similarity in valvular disease labels for purposes of automated clinical decision support. Specifically, we model the similarity in appearance of Doppler images from the same disease class as a constrained non-rigid translation transform of the velocity envelopes embedded in these images. The shape similarity between two Doppler images is then judged by recovering the alignment transform using a variant of dynamic shape warping. Results of similarity retrieval of doppler images for cardiac decision support on a large database of images are presented.

Synthesis Lectures on Image, Video, and Multimedia Processing, 2014

Procedings of the Proceedings of the 1st International Workshop on DIFFerential Geometry in Computer Vision for Analysis of Shapes, Images and Trajectories 2015, 2015

Persistent surveillance from camera networks, such as at parking lots, UAVs, etc., often results ... more Persistent surveillance from camera networks, such as at parking lots, UAVs, etc., often results in large amounts of video data, resulting in significant challenges for inference in terms of storage, communication and computation. Compressive cameras have emerged as a potential solution to deal with the data deluge issues in such applications. However, inference tasks such as action recognition require high quality features which implies reconstructing the original video data. Much work in compressive sensing (CS) theory is geared towards solving the reconstruction problem, where state-of-the-art methods are computationally intensive and provide low-quality results at high compression rates. Thus, reconstruction-free methods for inference are much desired. In this paper, we propose reconstruction-free methods for action recognition from compressive cameras at high compression ratios of 100 and above. Recognizing actions directly from CS measurements requires features which are mostly nonlinear and thus not easily applicable. This leads us to search for such properties that are preserved in compressive measurements. To this end, we propose the use of spatio-temporal smashed filters, which are compressive domain versions of pixel-domain matched filters. We conduct experiments on publicly available databases and show that one can obtain recognition rates that are comparable to the oracle method in uncompressed setup, even for high compression ratios.

In this paper, we consider the problem of fast and efficient indexing techniques for sequences ev... more In this paper, we consider the problem of fast and efficient indexing techniques for sequences evolving in non-Euclidean spaces. This problem has several applications in the areas of human activity analysis, where there is a need to perform fast search, and recognition in very high dimensional spaces. The problem is made more challenging when representations such as landmarks, contours, and human skeletons etc. are naturally studied in a non-Euclidean setting where even simple operations are much more computationally intensive than their Euclidean counterparts. We propose a geometry and data adaptive symbolic framework that is shown to enable the deployment of fast and accurate algorithms for activity recognition, dynamic texture recognition, motif discovery. Toward this end, we present generalizations of key concepts of piece-wise aggregation and symbolic approximation for the case of non-Euclidean manifolds. We show that one can replace expensive geodesic computations with much faster symbolic computations with little loss of accuracy in activity recognition and discovery applications. The framework is general enough to work across both Euclidean and non-Euclidean spaces, depending on appropriate feature representations without compromising on the ultra-low bandwidth, high speed and high accuracy. The proposed methods are ideally suited for real-time systems and low complexity scenarios.

2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2015

Spatial-multiplexing cameras have emerged as a promising alternative to classical imaging devices... more Spatial-multiplexing cameras have emerged as a promising alternative to classical imaging devices, often enabling acquisition of 'more for less'. One popular architecture for spatial multiplexing is the single-pixel camera (SPC), which acquires coded measurements of the scene with pseudorandom spatial masks. Significant theoretical developments over the past few years provide a means for reconstruction of the original imagery from coded measurements at sub-Nyquist sampling rates. Yet, accurate reconstruction generally requires high measurement rates and high signal-tonoise ratios. In this paper, we enquire if one can perform high-level visual inference problems (e.g. face recognition or action recognition) from compressive cameras without the need for image reconstruction. This is an interesting question since in many practical scenarios, our goals extend beyond image reconstruction. However, most inference tasks often require non-linear features and it is not clear how to extract such features directly from compressed measurements. In this paper, we show that one can extract nontrivial correlational features directly without reconstruction of the imagery. As a specific example, we consider the problem of face recognition beyond the visible spectrum e.g in the short-wave infra-red region (SWIR) -where pixels are expensive. We base our framework on smashed filters which suggests that inner-products between high-dimensional signals can be computed in the compressive domain to a high degree of accuracy. We collect a new face image dataset of 30 subjects, obtained using an SPC. Using face recognition as an example, we show that one can indeed perform reconstruction-free inference with a very small loss of accuracy at very high compression ratios of 100 and more.

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015

Persistent surveillance from camera networks, such as at parking lots, UAVs, etc., often results ... more Persistent surveillance from camera networks, such as at parking lots, UAVs, etc., often results in large amounts of video data, resulting in significant challenges for inference in terms of storage, communication and computation. Compressive cameras have emerged as a potential solution to deal with the data deluge issues in such applications. However, inference tasks such as action recognition require high quality features which implies reconstructing the original video data. Much work in compressive sensing (CS) theory is geared towards solving the reconstruction problem, where state-of-the-art methods are computationally intensive and provide low-quality results at high compression rates. Thus, reconstruction-free methods for inference are much desired. In this paper, we propose reconstruction-free methods for action recognition from compressive cameras at high compression ratios of 100 and above. Recognizing actions directly from CS measurements requires features which are mostly nonlinear and thus not easily applicable. This leads us to search for such properties that are preserved in compressive measurements. To this end, we propose the use of spatio-temporal smashed filters, which are compressive domain versions of pixel-domain matched filters. We conduct experiments on publicly available databases and show that one can obtain recognition rates that are comparable to the oracle method in uncompressed setup, even for high compression ratios.

Spatio-temporal patterns abound in the real world, and understanding them computationally holds t... more Spatio-temporal patterns abound in the real world, and understanding them computationally holds the promise of enabling a large class of applications such as video surveillance, biometrics, computer graphics and animation. In this dissertation, we study models and algorithms to describe complex spatio-temporal patterns in videos for a wide range of applications.

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015

In this paper, we consider the Précis problem of sampling K representative yet diverse data point... more In this paper, we consider the Précis problem of sampling K representative yet diverse data points from a large dataset. This problem arises frequently in applications such as video and document summarization, exploratory data analysis, and pre-filtering. We formulate a general theory which encompasses not just traditional techniques devised for vector spaces, but also non-Euclidean manifolds, thereby enabling these techniques to shapes, human activities, textures and many other image and video based datasets. We propose intrinsic manifold measures for measuring the quality of a selection of points with respect to their representative power, and their diversity. We then propose efficient algorithms to optimize the cost function using a novel annealing-based iterative alternation algorithm. The proposed formulation is applicable to manifolds of known geometry as well as to manifolds whose geometry needs to be estimated from samples. Experimental results show the strength and generality of the proposed approach.

Spatial-multiplexing cameras have emerged as a promising alternative to classical imaging devices... more Spatial-multiplexing cameras have emerged as a promising alternative to classical imaging devices, often enabling acquisition of 'more for less'. One popular architecture for spatial multiplexing is the single-pixel camera (SPC), which acquires coded measurements of the scene with pseudo-random spatial masks. Significant theoretical developments over the past few years provide a means for reconstruction of the original imagery from coded measurements at sub-Nyquist sampling rates. Yet, accurate reconstruction generally requires high measurement rates and high signal-to-noise ratios. In this paper, we enquire if one can perform high-level visual inference problems (e.g. face recognition or action recognition) from compressive cameras without the need for image reconstruction. This is an interesting question since in many practical scenarios, our goals extend beyond image reconstruction. However, most inference tasks often require non-linear features and it is not clear how to...

International Journal of Computer Vision

In this paper, we consider the problem of fast and efficient indexing techniques for human activi... more In this paper, we consider the problem of fast and efficient indexing techniques for human activity sequences evolving in Euclidean and non-Euclidean spaces. This problem has several applications in the areas of human activity analysis, where there is a need to perform fast search and recognition in large databases. The problem is made more challenging when features such as landmarks, contours, and stick-figures etc. are naturally studied in a non-Euclidean setting where even simple operations are much more computationally intensive than their Euclidean counterparts. We propose a geometry and data adaptive symbolic framework that is shown to enable the deployment of fast and accurate algorithms for activity recognition and motif discovery. Toward this end, we present generalizations of key concepts of piece-wise aggregation and symbolic approximation for the case of non-Euclidean manifolds. We show that one can replace expensive geodesic computations with much faster symbolic comput...

This paper concerns itself with compression strategies for orientation signals, seen as signals e... more This paper concerns itself with compression strategies for orientation signals, seen as signals evolving on the space of quaternions. The compression techniques extend classical signal approximation strategies used in data mining, by explicitly taking into account the quotient-space properties of the quaternion space. The approximation techniques are applied to the case of human gesture recognition from cellphone-based orientation sensors. Results indicate that the proposed approach results in high recognition accuracies, with low storage requirements, with the geometric computations providing added robustness than classical vector-space computations.

Procedings of the Proceedings of the 1st International Workshop on DIFFerential Geometry in Computer Vision for Analysis of Shapes, Images and Trajectories 2015, 2015

2015 Data Compression Conference, 2015

This paper concerns itself with compression strategies for orientation signals, seen as signals e... more This paper concerns itself with compression strategies for orientation signals, seen as signals evolving on the space of quaternions. The compression techniques extend classical signal approximation strategies used in data mining, by explicitly taking into account the quotientspace properties of the quaternion space. The approximation techniques are applied to the case of human gesture recognition from cellphone-based orientation sensors. Results indicate that the proposed approach results in high recognition accuracies, with low storage requirements, with the geometric computations providing added robustness than classical vector-space computations.

International Journal of Computer Vision, 2015

Clustering video sequences in order to infer and extract events from a single video stream is an ... more Clustering video sequences in order to infer and extract events from a single video stream is an extremely impor- tant problem and has significant potential in video index- ing, surveillance, activity discovery and event recognition. Clustering a video sequence into events requires one to si- multaneously recognize event boundaries (event consistent subsequences) and cluster these event subsequences. In or- der

2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014

ABSTRACT The compressive sensing paradigm holds promise for more cost-effective imaging outside o... more ABSTRACT The compressive sensing paradigm holds promise for more cost-effective imaging outside of the visible range, particularly in infrared wavelengths. However, the process of reconstructing compressively sensed images remains computationally expensive. The proof-of-concept tracker described here uses a particle filter with a likelihood update based on a “smashed filter” which estimates correlation directly, avoiding the reconstruction step. This approach leads to increased noise in correlation estimates, but by implementing the track-before-detect concept in the particle filter, tracker convergence may still be achieved with reasonable sensing rates. The tracker has been successfully tested on sequences of moving cars in the PETS2000 dataset.

2004 IEEE Region 10 Conference TENCON 2004., 2004

We present a system for automatic detection and tracking of faces in video sequences. Detection i... more We present a system for automatic detection and tracking of faces in video sequences. Detection is done based on a statistical characterization of skin-color. The position and size of the dominant face are estimated using statistical means of the binary map projections. Tracking is done using a Kalman filter. We propose a novel technique for updating the process noise covariance

2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010

Flow Doppler imaging has become an integral part of an echocardiographic exam. Automated interpre... more Flow Doppler imaging has become an integral part of an echocardiographic exam. Automated interpretation of flow doppler imaging has so far been restricted to obtaining hemodynamic information from velocity-time profiles depicted in these images. In this paper we exploit the shape patterns in Doppler images to infer the similarity in valvular disease labels for purposes of automated clinical decision support. Specifically, we model the similarity in appearance of Doppler images from the same disease class as a constrained non-rigid translation transform of the velocity envelopes embedded in these images. The shape similarity between two Doppler images is then judged by recovering the alignment transform using a variant of dynamic shape warping. Results of similarity retrieval of doppler images for cardiac decision support on a large database of images are presented.

Synthesis Lectures on Image, Video, and Multimedia Processing, 2014