Matrix Reference Manual (original) (raw)

Copyright © 1998-2022 Mike Brookes, Imperial College, London, UK
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

To cite this manual use: Brookes, M., "The Matrix Reference Manual", [online] http://www.ee.imperial.ac.uk/hp/staff/dmb/matrix/intro.html, 2020. Please see the site accessibility statement here.

Introduction

This manual contains reference information about linear algebra and the properties of real and complex matrices. The manual is divided into the following sections:

• Main Index: Alphabetical index of all entries.
• Properties : Properties and numbers associated with a matrix such as determinant, rank, inverse, …
• Eigenvalues : Theorems and matrix properties relating to eigenvalues and eigenvectors.
• Special : Properties of matrices that have a special form or structure such as diagonal, traingular, toeplitz, …
• Relations : Relations between matrices such as equivalence, congruence, …
• Decompositions : Decomposing matrices as sums or products of simpler forms.
• Identities : Useful equations relating matrices.
• Equations : Solutions of matrix equations
• Calculus : Differentiating expressions involving matrices whose elements are functions of an independent variable.
• Stochastic : Statistical properties of vectors and matrices whose elements are random numbers.
• Signals : Properties of observation vectors and covariance matrices from stochastic and deterministic signals.
• Examples: 2#2 : Examples of 2#2 matrixes with graphical illustration of their properties.
• Formal Algebra : Formal definitions of algebraic constructs such as groups, fields, vector spaces, …
• GNU Free Documentation License

Format of Manual Entries

The general format of each entry is as follows:

1. Definition of the term
2. Outline of why it is important
3. Geometric Interpretation
   The geometric interpretation of a matrix property or theorem is generally described for 2 or 3 dimensions. Words prefixed by + should be altered appropriately for other dimensions. Thus the word +area should be replaced by volume for 3-D spaces and by hyper-volume for larger spaces.
4. List of properties and theorems:
5. Links to related topics

Notation

The notation is based on the MATLABsoftware package; differences are notes below. All vectors are column vectors unless explicitly written as transposed.

• Matrices are represented as bold upper case (A), column vectors as bold lower case (a) and real or complex scalars as italic lower case (a).
• The matrix A[2#3] has 2 rows and 3 columns while the column vector a[4] has 4 elements.
• A matrix can be specified explicitly by listing its elements and using a semicolon to separate each row. Thus [1 2 3; 4 5 6] is a matrix with 2 rows and 3 columns. This notation can be used to compose large matrices from smaller ones: [A B;C D]. Each row must have the same total number of columns and each matrix within a row must have the same number of rows.
• Operators
  • Operator Precedence:
    * (1) Superscripts, powers and : suffix
    * (2) scalar and matrix multiplication/division
    * (3) ⊗ (Kroneker product)
    * (4) • ÷ (elementwise multiplication/division)
    * (5) Addition/Subtraction
  • A • B, A ÷ B andA•n denote element-by-element multiplication, division and raising to a power
  • A ⊗ B = KRON(A,B) is the Kronecker product of A and B. IfA is m#n and B is p#q then A⊗ B is mp#nq and equals the block matrix [a(1,1)B ... a(1,n)B ; ... ;a(m,1)B ... a(m,n)B].
  • A ⊕ B = DIAG(A,B) is the direct sum of A and B. If A is m#n and B is p#q thenA ⊕ B is m+p#n+q.
  • A: (also written vec(A) ) denotes the large column vector formed by concatenating all the columns of A. IfA is m# n, then A: = [_a_1,1 _a_2,1 … am,1 _a_1,2 _a_2,2 …_am,n_]T.
  • The following superscripts are used:
    * AC denotes the complex conjugate of A.
    * AH denotes the conjugate transpose of A. IfA is real then AH =AT.
    * AR and AI are the real and imaginary parts of A = AR + j AI .
    * AT denotes the transpose of A.
    * A-1, A# and A+ denote respectively the inverse, generalized inverse and pseudoinverse of A.
    * A-_T_=(AT)-1=(A-1)_T_denotes the inverse of the transpose
    * A-_H_=(AH)-1=(A-1)_H_denotes the inverse of the conjugate transpose
• For real matrices only, A>B means that each element ofA is greater than the corresponding element of B. Similar definitions apply to <, >= and <=.
• |A| and ||A||F denote the determinant and Frobenius norm of A.
• ||a|| denotes the euclidean norm of a
• |a| denotes the absolute value of a
• δ_i,j_ , the Kronecker delta function, equals 1 if_i_=j and equals 0 if i_≠_j

Subscripts

• aij or ai,j denotes the element of matrix A in row i of column j. Row and column indices begin at 1.
• A2:5,6:7 denotes the 4#2 submatrix of A consisting of row 2,3,4,5 and columns 6 and 7.
• aj denotes the j'th column of matrixA.
• AX,Y defines a matrix of the same size as X and Y (which must be the same size). Subscripts are taken from corresponding positions in X and Y. [_Different from MATLAB_]

Special Matrices

• The dimensions of the following special matrices are normally deduced from context but are occasionally specified explicitly (e.g.0[_m#n_]):
  • The matrix or vector 0 consists entirely of zeros.
  • The matrix or vector 1 consists entirely of ones.
  • The matrix I denotes the square identity matrix with 1's down the main diagonal and 0's elsewhere.
    * ei is the _i_thcolumn of I.
  • The matrix J denotes the square exchange matrix with 1's along the main anti-diagonal and 0's elsewhere.
• m:n denotes a column vector of length |1+n-m| whose elements go from m to n in steps of +1 or -1 according to whether_mn_. [_Different from MATLAB_]

Functions

Several of the functions listed below have different meanings according to whether their argument is a scalar, vector or matrix. The form of the result is indicated by the function's typeface.

• ABS(A) and abs(a) involves taking the absolute value of each matrix or vector element.
• ADJ(A) is the adjoint of the square matrix A.
• CHOOSE(n,r) is a matrix with n!/(r! (n-r)!) rows, each a different choice of r numbers out of the numbers 1:n. Each row is listed in ascending order.
• CONJ(A), also written AC, is the complex conjugate of A.
• conv(a[_m_],b[_n_])[_m+n_-1]is the convolution of a and b, i.e. a vector whose i'th element is the sum of a(j)b(_i_-j+1) where j goes from 1 to i.
• det(A), also written |A|, is the determinant of the square matrix A.
• diag(A) is the vector consisting of the diagonal elements ofA.
• DIAG(a) is the diagonal matrix whose diagonal elements are the elements of a.
• DIAG(A,B,C), also written A ⊕ B ⊕C, denotes the matrix [A 0 0; 0 B 0; 0 0 C]
• eig(A) is a vector containing the eigenvalues of A. If A is Hermitian, they are sorted into descending order.
• floor(x) is the most positive integer <= x
• INV(A) or A-1 is the inverse of A.
• KRON(A,B) = A ⊗ B is the Kronecker product of A and B. IfA is m#n and B is p#q then A⊗ B is mp#nq and equals the block matrix [a(1,1)B ... a(1,n)B ; ... ;a(m,1)B ... a(m,n)B].
• max(a | "condition") is the maximum of a subject to (an optional) "condition".
• min(a | "condition") is the minimum of a subject to (an optional) "condition".
• PERM(n) is a matrix with n! rows, each a different permutation of the numbers 1:n.
• pet(A) is the permanent ofA.
• prod(a) is the product of the elements of a.
• prod() is the vector formed by multiplying together the elements of each row of A. [_Different from MATLAB_].
• rho(A) is the spectral radius of A.
• rows(A) is the number of rows in the matrix A.
• tr(A) is the trace ofA.
• rank(A) is the dimension of the subspace spanned by the columns ofA.
• sgn(a) equals +1 or -1 according to the sign of a or 0 if_a_=0.
• sgn(a) equals +1 or -1 according to the signature of the permutation needed to sort the elements of a into ascending order.
• sgn(A) is a vector giving the permutation signatures of each row of A. Each entry equals +1 or -1.
• SKEW(a) is the 3#3 skew-symmetric matrix [0 -_a_3 _a_2; _a_3 0 -_a_1; -_a_2 _a_1 0] wherea is a 3-element vector. The vector cross product is given by a× b = SKEW(a) b = -SKEW(b) a. [See skew-symmetric for more properties of SKEW()].
• sum(a) is the sum of the elements of a.
• sum(A) is the vector formed by summing the rows ofA. [_Different from MATLAB_].
• sum(A) is the scalar formed by summing all the elements of A. [_Different from MATLAB_].
• TOE(a[m+_n_-1])[m#n_]is the m#n matrix with constant diagonals whose_i,_j_th element is a_i_-j+n.
• TVEC(m,n) is an orthogonal mn#mn permutation matrix whose_i_,_j_th element is 1 if_j_=1+m(_i_-1)-(_mn_-1)floor((_i_-1)/n) or 0 otherwise [see vectorized transpose].
• *vec(*A), also written*A:*, denotes the large column vector formed by concatenating all the columns of A. If A is m# n, then A: = [_a_1,1 a_2,1 …_am,1 _a_1,2 _a_2,2… _am,n_]T.

Other Web Sites

• Linear Algebra Glossary by John Burkardt [R.5]
• Mathworld by Eric Weisstein:Linear Algebra [R.15]
• The Matrix Cookbook by Kaare Brandt Petersen [R.14]

Acknowledgements

No originality is claimed for any of the material in this reference manual. The following books have in particular been very helpful:

• A Survey of Matrix Theory and Matrix Inequalities by M Marcus & H Minc, Prindle, Weber & Schmidt, 1964 / Dover, 1992 [R.12]
• Matrix Analysis and Topics in Matrix Analysis by R A Horn & C R Johnson, CUP 1990/1994, [R.10,R.11]
• Applied Linear Algebra by B. Noble and J.W.Daniel, Prentice-Hall, 1988 [R.13]
• Finite Dimensional Vector Spaces by P.R.Halmos, D Van Nostrand, 1958 [R.8]
• Generalized Inverses by A.Ben-Israel and T.N.E.Greville, Wiley1974 [R.3]
• Matrix Computations by G.H.Golub & C.F.Van Loan, John Hopkins University Press, 1983 ISBN 0-946536-00-7/05-8 [R.7]
• Matrix Methods in Stability Theory by S.Barnett and C.Storey, Nelson, 1970 [R.2]
• Complex Stochastic Processes: An Introduction to Theory and Application by K. S. Miller, Addison-Wesley, 1974 [R.16]

I would like to thank the following people who have made suggestions or corrections to this website and apologise to anyone whose name I have omitted from this list: Gerard Baron, Mike Fairbank, Carlos Fernandes, Thomas Foregger, John Halleck, Olaf Kaehler, Ben Kennedy, James Ng, Kaare Brandt Petersen, Jacopo Piazzi, Walter Tackett, Martin Zimmermann.

This page is part of The Matrix Reference Manual. Copyright © 1998-2017 Mike Brookes, Imperial College, London, UK. See the file <gfl.html> for copying instructions. Please send any comments or suggestions to "mike.brookes" at "imperial.ac.uk".
Updated: Id:intro.html112912021−01−0518:26:10ZdmbId: intro.html 11291 2021-01-05 18:26:10Z dmb Id:intro.html112912021−01−0518:26:10Zdmb