Principal Component Analysis with External Information on both Subjects and Variables | Psychometrika | Cambridge Core (original) (raw)

Abstract

A method for structural analysis of multivariate data is proposed that combines features of regression analysis and principal component analysis. In this method, the original data are first decomposed into several components according to external information. The components are then subjected to principal component analysis to explore structures within the components. It is shown that this requires the generalized singular value decomposition of a matrix with certain metric matrices. The numerical method based on the QR decomposition is described, which simplifies the computation considerably. The proposed method includes a number of interesting special cases, whose relations to existing methods are discussed. Examples are given to demonstrate practical uses of the method.

References

Bechtel, G. G., Tucker, L. R., & Chang, W. (1971). A scalar product model for the multidimensional scaling of choice. Psychometrika, 36, 369–387.CrossRefGoogle Scholar

Besse, P., && Ramsay, J. O. (1986). Principal components analysis of sampled functions. Psychometrika, 51, 285–311.CrossRefGoogle Scholar

Bloxom, B. (1978). Constrained multidimensional scaling in N spaces. Psychometrika, 43, 397–408.CrossRefGoogle Scholar

Böckenholt, U., & Böckenholt, I. (1990). Canonical analysis of contingency tables with linear constraints. Psychometrika, 55, 633–639.CrossRefGoogle Scholar

Carroll, J. D. (1972). Individual differences and multidimensional scaling. In Shepard, R. N., Romney, A. K., & Nerlove, S. B. (Eds.), Multidimensional scaling, Vol. I (pp. 105–155). New York: Seminar Press.Google Scholar

Carroll, J. D., Pruzansky, S., & Kruskal, J. B. (1980). CANDELINC: A general approach to multidimensional analysis of many-way arrays with linear constraints on parameters. Psychometrika, 45, 3–24.CrossRefGoogle Scholar

Corsten, L. C. A. (1976). Matrix approximation, a key to application of multivariate methods. Invited paper presented at the 9th Biometric Conference, Boston.Google Scholar

Corsten, L. C. A., & Van Eijnsbergen, A. C. (1972). Multiplicative effects in two-way analysis of variance. Statistica Neelandica, 26, 61–68.CrossRefGoogle Scholar

Critchley, F. (1985). Influence in principal component analysis. Biometrika, 72, 627–636.CrossRefGoogle Scholar

de Leeuw, J. (1984). Fixed rank matrix approximation with singular weights matrices. Computational Statistics Quarterly, 1, 3–12.Google Scholar

DeSarbo, W. S., Carroll, J. D., Lehmann, D. R., & O'Shaughnessy, J. (1982). Three-way multivariate conjoint analysis. Marketing Science, 1, 323–350.CrossRefGoogle Scholar

DeSarbo, W. S., & Rao, V. R. (1984). GENFOLD2: A set of models and algorithms for the GENeral UnFOLDing analysis of preference/dominance data. Journal of Classification, 1, 147–186.CrossRefGoogle Scholar

De Soete, G., & Carroll, J. D. (1983). A maximum likelihood method for fitting the wandering vector model. Psychometrika, 48, 553–566.CrossRefGoogle Scholar

Eastment, H. T., & Krzanowski, W. J. (1982). Cross-validatory choice of the number of components from a principal component analysis. Technometrics, 24, 73–77.CrossRefGoogle Scholar

Efron, B. (1979). Bootstrap methods: Another look at the Jackknife. Annals of Statistics, 7, 1–26.CrossRefGoogle Scholar

Escoufier, Y., & Holmes, S. (1988). Décomposition de la variabilité dans les analyses exploratoires: Un exemple d'analyse en composantes principles en presence de variables qualitatives concomittantes [Descomposition of variabilities in exploratory data analysis: An example of principal component analysis in the presence of qualitative concomitant variables.], Montpellier, France: ENSAM-INRA.Google Scholar

Fisher, R. A. (1948). Statistical methods for research workers 10th ed.,, London: Oliver and Boyd.Google Scholar

Gabriel, K. R. (1978). Least squares approximation of matrices by additive and multiplicative models. Journal of Royal Statistical Society, Series B, 40, 186–196.CrossRefGoogle Scholar

Gabriel, K. R., & Zamir, S. (1979). Lower rank approximation of matrices by least squares with any choice of weights. Technometrics, 21, 489–498.CrossRefGoogle Scholar

Gifi, A. (1981). Non-linear multivariate analysis, Leiden: University of Leiden, Department of Data Theory.Google Scholar

Gollob, H. F. (1968). A statistical model which combines features of factor analytic and analysis of variance technique. Psychometrika, 33, 73–115.CrossRefGoogle Scholar

Greenacre, M. J., & Underhill, L. G. (1982). Scaling a data matrix in a low-dimensional euclidean space. In Hawkins, D. M. (Eds.), Topics in applied multivariate analysis (pp. 183–268). Cambridge: Cambridge University Press.CrossRefGoogle Scholar

Hayashi, C. (1952). On the prediction of phenomena from qualitative data and the quantification of qualitative data from the mathematico-statistical point of view. Annals of the Institute of Statistical Mathematics, 2, 69–98.CrossRefGoogle Scholar

Heiser, W. J., & de Leeuw, J. (1981). Multidimensional mapping of preference data. Mathematiqué et sciences humaines, 19, 39–96.Google Scholar

Heiser, W. J., & Meulman, J. (1983). Analyzing rectangular tables by joint and constrained multidimensional scaling. Journal of Econometrics, 22, 139–167.CrossRefGoogle Scholar

Heiser, W. J., & Meulman, J. (1983). Constrained multidimensional scaling, including confirmation. Applied Psychological Measurement, 7, 381–404.CrossRefGoogle Scholar

Israëls, A. Z. (1984). Redundancy analysis for qualitative variables. Psychometrika, 49, 331–346.CrossRefGoogle Scholar

Khatri, C. G. (1966). A note on a MANOVA model applied to problems in growth curves. Annals of the Institute of Statistical Mathematics, 18, 75–86.CrossRefGoogle Scholar

Kruskal, J. B. (1964). Nonmetric multidimensional scaling: A numerical method. Psychometrika, 29, 115–129.CrossRefGoogle Scholar

Meulman, J. (1982). Homogeneity analysis of incomplete data, Leiden: DSWO Press.Google Scholar

Nishisato, S. (1978). Optimal scaling of paired comparison and rank order data: An alternative to Guttman's formulation. Psychometrika, 43, 263–271.CrossRefGoogle Scholar

Nishisato, S. (1980). Analysis of categorical data: Dual scaling and its applications, Toronto: University of Toronto Press.CrossRefGoogle Scholar

Nishisato, S. (1980). Dual scaling of successive categories data. Japanese Psychological Research, 22, 134–143.CrossRefGoogle Scholar

Nishisato, S. (1982). Quantification of qualitative data, Tokyo: Asakurashoten.Google Scholar

Nishisato, S. (1988). Dual scaling: Its development and comparisons with other quantification methods. In Pressmar, H. D., Jager, K. E., Krallmann, H., Schellhaas, H., & Streitferdt, L. (Eds.), Deutsche Geselleschaft für operations research proceedings (pp. 376–389). Berlin: Springer.Google Scholar

Nishisato, S., & Lawrence, D. R. (1981, May). Dual scaling of multidimensional tables, a comparative study. Paper presented at the annual meeting of the Psychometric Society, Chapel Hill, NC.Google Scholar

Nishisato, S., & Lawrence, D. R. (1989). Dual scaling of multiway data matrices: Several variants. In Coppi, R., Bolasco, S. (Eds.), Multiway data analysis (pp. 317–326). Amsterdam: North Holland.Google Scholar

Nishisato, S., & Sheu, W. (1984). A note on dual scaling of successive categories data. Psychometrika, 49, 493–500.CrossRefGoogle Scholar

Okamoto, M. (1972). Four techniques of principal component analysis. Journal of Japanese Statistical Society, 2, 63–69.Google Scholar

Potthoff, R. F., & Roy, S. N. (1964). A generalized multivariate analysis of variance model useful especially for growth curve problems. Biometrika, 51, 313–326.CrossRefGoogle Scholar

Ramsay, J. O. (1978). Confidence regions for multidimensional scaling analysis. Psychometrika, 43, 145–160.CrossRefGoogle Scholar

Ramsay, J. O. (1980). Joint analysis of direct ratings, pairwise preferences and dissimilarities. Psychometrika, 45, 149–165.CrossRefGoogle Scholar

Ramsay, J. O. (1989). Monotone regression splines in actions. Statistical Science, 4, 425–441.Google Scholar

Ramsay, J. O., ten Berge, J., & Styan, G. P. H. (1984). Matrix correlation. Psychometrika, 49, 403–423.CrossRefGoogle Scholar

Rao, C. R. (1964). The use and interpretation of principal component analysis in applied research. Sankhya A, 26, 329–358.Google Scholar

Rao, C. R. (1965). The theory of least squares when the parameters are stochastic and its application to the analysis of growth curves. Biometrika, 52, 447–458.CrossRefGoogle Scholar

Rao, C. R. (1979). Separation theorems for singular values of matrices and their applications in multivariate analysis. Journal of Multivariate Analysis, 9, 362–377.CrossRefGoogle Scholar

Rao, C. R. (1980). Matrix approximations and reduction of dimensionality in multivariate statistical analysis. In Krishnaiah, P. R. (Eds.), Multivariate analysis (pp. 3–22). Amsterdam: North Holland.Google Scholar

Rumelhart, D. L., & Greeno, J. G. (1971). Similarity between stimuli: An experimental test of the Luce and Restle choice models. Journal of Mathematical Psychology, 8, 370–381.CrossRefGoogle Scholar

Sabatier, R., Lebreton, J. D., & Chessel, D. (1989). Multivariate analysis of composition data accompanied by qualitative variables describing a structure. In Coppi, R., Bolasco, S. (Eds.), Multiway data analysis (pp. 341–352). Amsterdam: North-Holland.Google Scholar

Shibayama, T. (1988). Multivariate analysis of test scores with missing values. Unpublished Doctoral Dissertation, University of Tokyo. (in Japanese)Google Scholar

Siotani, M., Hayakawa, T., & Fujikoshi, Y. (1985). Modern multivariate statistical analysis: A graduate course handbook, Columbus, OH: American Sciences Press.Google Scholar

Slater, P. (1960). The analysis of personal preferences. The British Journal of Statistical Psychology, 13, 119–135.CrossRefGoogle Scholar

Sjöberg, L. (1967). Successive categories scaling of paired comparisons. Psychometrika, 32, 297–308.CrossRefGoogle Scholar

Takane, Y. (1980). Maximum likelihood estimation in the generalized case of Thurstone's model of comparative judgment. Japanese Psychological Research, 22, 188–196.CrossRefGoogle Scholar

Takane, Y. (1987). Analysis of covariance structures and binary choice data. Communication and Cognition, 20, 45–62.Google Scholar

Takane, Y., & Shibayama, T. (1988). Three vector models of pairwise preference ratings and their generalizations. In Kashiwagi, S. (Eds.), Proceedings of the 16th Annual Meeting of the Behaviormetric Society (pp. 131–132). Tokyo: Behaviormetric Society of Japan.Google Scholar

Takane, Y., & Shibayama, T. (1988). Dual scaling with external criteria reconsidered. In Kashiwagi, S. (Eds.), Proceedings of the 16th Annual Meeting of the Behaviormetric Society (pp. 133–134). Tokyo: Behaviormetric Society of Japan.Google Scholar

Takane, Y., Yanai, H., & Mayekawa, S. (in press). Relationships among several methods of linearly constrained correspondence analysis. Psychometrika.Google Scholar

Tanaka, Y. (1988). Sensitivity analysis in principal component analysis: Influence on the subspace spanned by principal components. Communications in Statistics—Theory and Methods, 17, 3157–3175.CrossRefGoogle Scholar

ter Braak, C. J. F. (1986). Canonical correspondence analysis: A new eigenvector technique for multivariate direct gradient analysis. Ecology, 67, 1167–1179.CrossRefGoogle Scholar

Tucker, L. R. (1959). Intra-individual and inter-individual multidimensionality. In Gulliksen, H., Messick, S. (Eds.), Psychological scaling (pp. 155–167). New York: Wiley.Google Scholar

van den Wollenberg, A. L. (1977). Redundancy analysis: An alternative for canonical correlation analysis. Psychometrika, 42, 207–219.CrossRefGoogle Scholar

van der Heijden, P. G. M., de Falguerolles, A., & de Leeuw, J. (1989). A combined approach to contingency table analysis using correspondence analysis and log linear analysis. Applied Statistics, 38, 249–292.CrossRefGoogle Scholar

Weinberg, S. L., Carroll, J. D., & Cohen, H. S. (1984). Confidence regions for INDSCAL using the jackknife and bootstrap techniques. Psychometrika, 49, 475–491.CrossRefGoogle Scholar

Wilkinson, J. H. (1965). The algebraic eigenvalue problem, Oxford: Oxford University Press.Google Scholar

Winsberg, S. (1988). Two techniques: Monotone spline transformations for dimension reduction in PCA and easy-to-generate metrics for PCA of sampled functions. In van Rijckevorsel, J. L. A. & de Leeuw, J. (Eds.), Component and correspondence analysis (pp. 115–135). New York: Wiley.Google Scholar

Winsberg, S., & Ramsay, J. O. (1983). Monotone spline transformations for dimension reduction. Psychometrika, 48, 575–595.CrossRefGoogle Scholar

Yanai, H. (1970). Factor analysis with external criteria. Japanese Psychological Research, 12, 143–153.CrossRefGoogle Scholar

Yanai, H. (1974). Unification of various techniques of multivariate analysis by means of generalized coefficients of determination. (G.C.D.). Journal of Behaviormetrics, 1, 45–54.Google Scholar

Yanai, H. (1990). Some generalized forms of least squares g-inverse, minimum norm g-inverse and Moore-Penrose inverse matrices. Computational Statistics and Data Analysis, 10, 251–260.CrossRefGoogle Scholar

Yanai, H., & Takeuchi, K. (1983). Projection matrices, generalized inverse and singular value decomposition, Tokyo: University of Tokyo Press (in Japanese)Google Scholar