§ 10. Composition of Linear Transformations and Matrix Multiplication¶
Composition of Linear Transformations¶
Theorem 10.1 : Composite of linear transformations is linear.
Let \(V\), \(W\), and \(Z\) be vector spaces over the same field \(F\), and let \(T: V \rightarrow W\) and \(U: W \rightarrow Z\) be linear. Then \(UT: V \rightarrow Z\) is linear.
Proof
Let \(x, y \in V\) and \(a \in F\). Then
Theorem 10.2 : Properties of the composition of linear transformations in \(\mathcal{L}(V)\)
Let \(V\) be a vector space. Let \(T, U_{1}, U_{2} \in \mathcal{L}(V)\). Then
- (a) \(T\left(U_{1}+U_{2}\right)=TU_{1}+TU_{2}\) and \(\left(U_{1}+U_{2}\right) T=U_{1} T+U_{2} T\)
- (b) \(T\left(U_{1} U_{2}\right)=\left(TU_{1}\right) U_{2}\)
- (c) \(TI=IT=T\)
- (d) \(a\left(U_{1} U_{2}\right)=\left(a U_{1}\right) U_{2}=U_{1}\left(a U_{2}\right)\) for all scalars \(a\).
Proof
(By Theorem 10.1, any composition of linear maps is linear, so all maps appearing below are in \(\mathcal L(V)\).)
-
(a) Distributive laws.
For any \(x\in V\),\[ T(U_1+U_2)(x)=T\bigl(U_1(x)+U_2(x)\bigr)=T(U_1(x))+T(U_2(x))=(TU_1+TU_2)(x), \]hence \(T(U_1+U_2)=TU_1+TU_2\). Also,
\[ (U_1+U_2)T(x)=U_1(T(x))+U_2(T(x))=(U_1T+U_2T)(x), \]so \((U_1+U_2)T=U_1T+U_2T\).
-
(b) Associativity of composition.
For any \(x\in V\),\[ T(U_1U_2)(x)=T\bigl(U_1(U_2(x))\bigr)=(TU_1)\bigl(U_2(x)\bigr)=((TU_1)U_2)(x), \]hence \(T(U_1U_2)=(TU_1)U_2\).
-
(c) Identity transformation.
For any \(x\in V\),\[ (TI)(x)=T(I(x))=T(x)\quad\text{and}\quad (IT)(x)=I(T(x))=T(x), \]hence \(TI=IT=T\).
-
(d) Compatibility with scalar multiplication.
For any \(x\in V\),\[ \bigl(a(U_1U_2)\bigr)(x)=a\,U_1(U_2(x))=(aU_1)(U_2(x))=((aU_1)U_2)(x), \]so \(a(U_1U_2)=(aU_1)U_2\). Also,
\[ \bigl(U_1(aU_2)\bigr)(x)=U_1\bigl(a\,U_2(x)\bigr)=a\,U_1(U_2(x))=\bigl(a(U_1U_2)\bigr)(x), \]so \(a(U_1U_2)=U_1(aU_2)\). Therefore
\[ a(U_1U_2)=(aU_1)U_2=U_1(aU_2). \]
A more general result to Theorem 10.2 holds for linear transformations that have domains unequal to their codomains. (See Exercise 10.8.)
Matrix Multiplication¶
Concept 10.3 : Motivation for matrix multiplication
Let \(T: V \rightarrow W\) and \(U: W \rightarrow Z\) be linear transformations, and let \(A=[U]_{\beta}^{\gamma}\) and \(B=[T]_{\alpha}^{\beta}\), where \(\alpha=\left\{v_{1}, v_{2}, \ldots, v_{n}\right\}, \beta=\left\{w_{1}, w_{2}, \ldots, w_{m}\right\}\), and \(\gamma=\left\{z_{1}, z_{2}, \ldots, z_{p}\right\}\) are ordered bases for \(V, W\), and \(Z\), respectively. We would like to define the product \(A B\) of two matrices so that \(A B=[U T]_{\alpha}^{\gamma}\). Consider the matrix \({[UT]}_{\alpha}^{\gamma}\). For \(1 \leq j \leq n\), we have
where
Definition 10.4 : Product of Matrix
Let \(A\) be an \(m \times n\) matrix and \(B\) be an \(n \times p\) matrix. We define the product of \(A\) and \(B\), denoted \(A B\), to be the \(m \times p\) matrix such that
Concept 10.5 : Properties of product
-
In order for the product \(A B\) to be defined, there are restrictions regarding the relative sizes of \(A\) and \(B\). The following mnemonic device is helpful: " \((m \times n) \cdot(n \times p)=(m \times p)\) "; that is, in order for the product \(A B\) to be defined, the two "inner" dimensions must be equal, and the two "outer" dimensions yield the size of the product.
-
Matrix multiplication is not commutative, it need not be true that \(A B=B A\).
\[ \left(\begin{array}{ll} 1 & 1 \\ 0 & 0 \end{array}\right)\left(\begin{array}{ll} 0 & 1 \\ 1 & 0 \end{array}\right)=\left(\begin{array}{ll} 1 & 1 \\ 0 & 0 \end{array}\right) \quad \text { and } \quad\left(\begin{array}{ll} 0 & 1 \\ 1 & 0 \end{array}\right)\left(\begin{array}{ll} 1 & 1 \\ 0 & 0 \end{array}\right)=\left(\begin{array}{ll} 0 & 0 \\ 1 & 1 \end{array}\right) . \] -
If \(A\) is an \(m \times n\) matrix and \(B\) is an \(n \times p\) matrix, then \((A B)^{t}=B^{t} A^{t}\). Since
\[ (A B)_{i j}^{t}=(A B)_{j i}=\sum_{k=1}^{n} A_{j k} B_{k i} \]and
\[ \left(B^{t} A^{t}\right)_{i j}=\sum_{k=1}^{n}\left(B^{t}\right)_{i k}\left(A^{t}\right)_{k j}=\sum_{k=1}^{n} B_{k i} A_{j k} \]we are finished. Therefore the transpose of a product is the product of the transposes in the opposite order.
-
We see that if
\[ A=\left(\begin{array}{ll} 0 & 0 \\ 1 & 0 \end{array}\right) \]then \(A^{2}=O\) (the zero matrix) even though \(A \neq O\). Thus the cancellation property for multiplication in fields is not valid for matrices. To see why, assume that the cancellation law is valid. Then, from \(A \cdot A=A^{2}=O=A \cdot O\), we would conclude that \(A=O\), which is false.
Theorem 10.6, Corollary 10.7 are immediate consequences of Concept 10.3.
Theorem 10.6 : Matrix of a composite transformation
Let \(V, W\), and \(Z\) be finite-dimensional vector spaces with ordered bases \(\alpha, \beta\), and \(\gamma\), respectively. Let \(T: V \rightarrow W\) and \(U: W \rightarrow Z\) be linear transformations. Then
Corollary 10.7 : Matrix of a composite in \(\mathcal{L}(V)\)
Let \(V\) be a finite-dimensional vector space with an ordered basis \(\beta\). Let \(T, U \in \mathcal{L}(V)\). Then \([UT]_{\beta}=[U]_{\beta}[T]_{\beta}\).
Definition 10.8 : Kronecker delta / Identity matrix
We define the Kronecker delta \(\delta_{i j}\) by \(\delta_{i j}=1\) if \(i=j\) and \(\delta_{i j}=0\) if \(i \neq j\). The \(n \times n\) identity matrix \(I_{n}\) is defined by \(\left(I_{n}\right)_{i j}=\delta_{i j}\).
Theorem 10.9 : Basic properties of matrix multiplication
Let \(A\) be an \(m \times n\) matrix, \(B\) and \(C\) be \(n \times p\) matrices, and \(D\) and \(E\) be \(q \times m\) matrices. Then
- (a) \(A(B+C)=A B+A C\) and \((D+E) A=D A+E A\).
- (b) \(a(A B)=(a A) B=A(a B)\) for any scalar \(a\).
- (c) \(I_{m} A=A=A I_{n}\).
- (d) If \(V\) is an \(n\)-dimensional vector space with an ordered basis \(\beta\), then \([I_V]_{\beta}=I_{n}\).
Proof
-
(a) We have
\[ \begin{aligned} {[A(B+C)]_{i j} } & =\sum_{k=1}^{n} A_{i k}(B+C)_{k j}=\sum_{k=1}^{n} A_{i k}\left(B_{k j}+C_{k j}\right) \\ & =\sum_{k=1}^{n}\left(A_{i k} B_{k j}+A_{i k} C_{k j}\right)=\sum_{k=1}^{n} A_{i k} B_{k j}+\sum_{k=1}^{n} A_{i k} C_{k j} \\ & =(A B)_{i j}+(A C)_{i j}=[A B+A C]_{i j} . \end{aligned} \]So \(A(B+C)=A B+A C\).
-
(b) For each \(i,j\),
\[ \begin{aligned} {[a(AB)]_{ij}} &=a[AB]_{ij} =a\sum_{k=1}^{n}A_{ik}B_{kj} =\sum_{k=1}^{n}(aA_{ik})B_{kj} =[((aA)B)]_{ij}. \end{aligned} \]Hence \(a(AB)=(aA)B\). Also,
\[ \begin{aligned} {[A(aB)]_{ij}} &=\sum_{k=1}^{n}A_{ik}(aB)_{kj} =\sum_{k=1}^{n}A_{ik}(aB_{kj}) =a\sum_{k=1}^{n}A_{ik}B_{kj} =[a(AB)]_{ij}. \end{aligned} \]Therefore \(a(AB)=(aA)B=A(aB)\).
-
(c) We have
\[ \left(I_{m} A\right)_{i j}=\sum_{k=1}^{m}\left(I_{m}\right)_{i k} A_{k j}=\sum_{k=1}^{m} \delta_{i k} A_{k j}=A_{i j} \] -
(d) Let \(\beta=\{v_1,\dots,v_n\}\) be an ordered basis of \(V\). Since \(I_V(v_j)=v_j\), the \(j\)th coordinate column of \([I_V]_\beta\) is \([v_j]_\beta=e_j\), the \(j\)th standard basis vector in \(F^n\). Hence the columns of \([I_V]_\beta\) are \(e_1,\dots,e_n\), so \([I_V]_\beta=I_n\).
Corollary 10.10 : Distributivity over linear combinations
Let \(A\) be an \(m \times n\) matrix, \(B_{1}, B_{2}, \ldots, B_{k}\) be \(n \times p\) matrices, \(C_{1}, C_{2}, \ldots, C_{k}\) be \(q \times m\) matrices, and \(a_{1}, a_{2}, \ldots, a_{k}\) be scalars. Then
and
Proof
Apply Theorem 10.9(a) repeatedly to distribute \(A\) over sums, and apply Theorem 10.9(b) repeatedly to pull scalars out:
Similarly,
Definition 10.11 : Power of Matrix
For an \(n \times n\) matrix \(A\), we define \(A^{1}=A, A^{2}=A A, A^{3}=A^{2} A\), and, in general, \(A^{k}=A^{k-1} A\) for \(k=2,3, \ldots\). We define \(A^{0}=I_{n}\).
Theorem 10.12 : Columns of a product
Let \(A\) be an \(m \times n\) matrix and \(B\) be an \(n \times p\) matrix. For each \(j(1 \leq j \leq p)\) let \(u_{j}\) and \(v_{j}\) denote the \(j\) th columns of \(A B\) and \(B\), respectively. Then
- (a) \(u_{j}=A v_{j}\)
- (b) \(v_{j}=B e_{j}\), where \(e_{j}\) is the \(j\) th standard vector of \(F^{p}\).
Proof
-
(a) We have
\[ u_{j}=\left(\begin{array}{c} (A B)_{1 j} \\ (A B)_{2 j} \\ \vdots \\ (A B)_{m j} \end{array}\right)=\left(\begin{array}{c} \sum_{k=1}^{n} A_{1 k} B_{k j} \\ \sum_{k=1}^{n} A_{2 k} B_{k j} \\ \vdots \\ \sum_{k=1}^{n} A_{m k} B_{k j} \end{array}\right)=A\left(\begin{array}{c} B_{1 j} \\ B_{2 j} \\ \vdots \\ B_{n j} \end{array}\right)=A v_{j} . \] -
(b) \(B = B I_p\) holds from Theorem 10.9(c), and \(j\)th column of \(I_p\) is \(e_j\). It follows from (a) that \(v_{j}=B e_{j}\).
Theorem 10.13 : Columns and rows of a product as linear combinations
Let \(A\) be an \(m\times n\) matrix and \(B\) be an \(n\times p\) matrix.
Let \(v_1,\dots,v_p\) denote the columns of \(B\), and let \(u_1,\dots,u_n\) denote the columns of \(A\). Let \(r_1,\dots,r_m\) denote the rows of \(A\), and let \(s_1,\dots,s_n\) denote the rows of \(B\).
Then
-
(a) For any column vector \(z=(a_1,\dots,a_p)^t\in F^p\),
\[ Bz=\sum_{j=1}^p a_j v_j. \] -
(b) For each \(j\) \((1\le j\le p)\), column \(j\) of \(AB\) is a linear combination of the columns of \(A\):
\[ \operatorname{col}_j(AB)=\sum_{k=1}^n B_{kj}\,u_k. \] -
(c) For any row vector \(w=(b_1,\dots,b_m)\in F^{1\times m}\),
\[ wA=\sum_{i=1}^m b_i r_i. \] -
(d) For each \(i\) \((1\le i\le m)\), row \(i\) of \(AB\) is a linear combination of the rows of \(B\):
\[ \operatorname{row}_i(AB)=\sum_{k=1}^n A_{ik}\,s_k. \]
To summarize, column \(j\) of \(A B\) is a linear combination of the columns of \(A\) with the coefficients in the linear combination being the entries of column \(j\) of \(B\). An analogous result holds for rows; that is, row \(i\) of \(A B\) is a linear combination of the rows of \(B\) with the coefficients in the linear combination being the entries of row \(i\) of \(A\).
Proof
-
(a) Write \(z=\sum_{j=1}^p a_j e_j\). Then by Theorem 10.12(b), \(Be_j=v_j\), and by Theorem 10.9(a),(b),
\[ Bz=B\left(\sum_{j=1}^p a_j e_j\right)=\sum_{j=1}^p a_j(Be_j)=\sum_{j=1}^p a_j v_j. \] -
(b) Let \(e_j\) be the \(j\)th standard vector of \(F^p\). By Theorem 10.12(a),(b),
\[ \operatorname{col}_j(AB)=A(Be_j)=Av_j. \]Writing \(v_j=(B_{1j},\dots,B_{nj})^t=\sum_{k=1}^n B_{kj} e_k\), and applying (a) with \(B\) replaced by \(A\) gives
\[ Av_j=\sum_{k=1}^n B_{kj}\,u_k, \]which is the desired statement.
-
(c) Take transposes and use (a). Since \((wA)^t=A^t w^t\), apply (a) to \(A^t\) and the vector \(w^t=(b_1,\dots,b_m)^t\):
\[ A^t w^t=\sum_{i=1}^m b_i\,\operatorname{col}_i(A^t). \]Transposing, and noting that \(\operatorname{col}_i(A^t)=(\operatorname{row}_i(A))^t\), we obtain
\[ wA=\sum_{i=1}^m b_i\,\operatorname{row}_i(A)=\sum_{i=1}^m b_i r_i. \] -
(d) Apply (b) to transposes. Since \((AB)^t=B^tA^t\), part (b) (with \(B^t\) in place of \(A\) and \(A^t\) in place of \(B\)) says that column \(i\) of \(B^tA^t\) is a linear combination of the columns of \(B^t\) with coefficients from column \(i\) of \(A^t\), i.e.
\[ \operatorname{col}_i\bigl((AB)^t\bigr)=\sum_{k=1}^n A_{ik}\,\operatorname{col}_k(B^t). \]Transposing, \(\operatorname{col}_i((AB)^t)=(\operatorname{row}_i(AB))^t\) and \(\operatorname{col}_k(B^t)=(\operatorname{row}_k(B))^t=s_k^t\), so
\[ \operatorname{row}_i(AB)=\sum_{k=1}^n A_{ik}\,s_k. \]
Theorem 10.14 : Coordinate formula for a linear transformation
Let \(V\) and \(W\) be finite-dimensional vector spaces having ordered bases \(\beta\) and \(\gamma\), respectively, and let \(T: V \rightarrow W\) be linear. Then, for each \(u \in V\), we have
Proof
Fix \(u \in V\), and define the linear transformations \(f: F \rightarrow V\) by \(f(a)=a u\) and \(g: F \rightarrow W\) by \(g(a)=a T(u)\) for all \(a \in F\). Let \(\alpha=\{1\}\) be the standard ordered basis for \(F\). Notice that \(g=T f\). Identifying column vectors as matrices and using Theorem 10.6, we obtain
Left-Multiplication Transformation¶
Definition 10.15 : Left-Multiplication Transformation
Let \(A\) be an \(m \times n\) matrix with entries from a field \(F\). We denote by \(L_{A}\) the mapping \(L_{A}: F^{n} \rightarrow F^{m}\) defined by \(L_{A}(x)=A x\) (the matrix product of \(A\) and \(x\)) for each column vector \(x \in F^{n}\). We call \(L_{A}\) a left-multiplication transformation.
Theorem 10.16 : Properties of left-multiplication transformations
Let \(A\) be an \(m \times n\) matrix with entries from \(F\). Define \(L_{A}: F^{n} \rightarrow F^{m}\) by
Then \(L_{A}\) is linear. Furthermore, if \(B\) is any other \(m \times n\) matrix and \(\beta\) and \(\gamma\) are the standard ordered bases for \(F^{n}\) and \(F^{m}\), respectively, then:
- (a) \([L_{A}]_{\beta}^{\gamma}=A\).
- (b) \(L_{A}=L_{B}\) if and only if \(A=B\).
- (c) \(L_{A+B}=L_{A}+L_{B}\) and \(L_{aA}=a\,L_{A}\) for all \(a\in F\).
- (d) If \(T: F^{n} \rightarrow F^{m}\) is linear, then there exists a unique \(m \times n\) matrix \(C\) such that \(T=L_{C}\). In fact, \(C=[T]_{\beta}^{\gamma}\).
- (e) If \(E\) is an \(n \times p\) matrix, then \(L_{AE}=L_{A}L_{E}\).
- (f) If \(m=n\), then \(L_{I_n}=I_{F^{n}}\).
Proof
The linearity of \(L_{A}\) follows from the properties of matrix multiplication (Theorem 10.9).
-
(a) The \(j\)th column of \([L_{A}]_{\beta}^{\gamma}\) is \(L_{A}(e_j)=Ae_j\). By Theorem 10.12(b), \(Ae_j\) is the \(j\)th column of \(A\). Hence \([L_{A}]_{\beta}^{\gamma}=A\).
-
(b) If \(L_{A}=L_{B}\), then by (a),
\[ A=[L_{A}]_{\beta}^{\gamma}=[L_{B}]_{\beta}^{\gamma}=B. \]Conversely, if \(A=B\), then \(L_{A}(x)=Ax=Bx=L_{B}(x)\) for all \(x\), so \(L_{A}=L_{B}\).
-
(c) For any \(x\in F^{n}\),
\[ L_{A+B}(x)=(A+B)x=Ax+Bx=L_{A}(x)+L_{B}(x), \]and for any \(a\in F\),
\[ L_{aA}(x)=(aA)x=a(Ax)=a\,L_{A}(x), \]using Theorem 10.9(a),(b). Hence \(L_{A+B}=L_{A}+L_{B}\) and \(L_{aA}=a\,L_{A}\).
-
(d) Let \(C=[T]_{\beta}^{\gamma}\). Then for all \(x\in F^{n}\),
\[ T(x)=C[x]_{\beta}=Cx=L_{C}(x), \]so \(T=L_{C}\). If also \(T=L_{D}\), then \(L_{C}=L_{D}\) and by (b) we have \(C=D\). Thus \(C\) is unique.
-
(e) For any \(x\in F^{p}\),
\[ L_{AE}(x)=(AE)x=A(Ex)=L_{A}(L_{E}(x)), \]using associativity of matrix multiplication. Hence \(L_{AE}=L_{A}L_{E}\).
-
(f) If \(m=n\), then for any \(x\in F^{n}\),
\[ L_{I_n}(x)=I_nx=x, \]so \(L_{I_n}=I_{F^{n}}\).
Theorem 10.17 : Associativity of matrix multiplication
Let \(A, B\), and \(C\) be matrices such that \(A(BC)\) is defined. Then \((AB)C\) is also defined and
Proof
It is routine to check from the sizes of the matrices that if \(A(BC)\) is defined, then \((AB)C\) is also defined.
Using Theorem 10.16(e) and the associativity of composition of functions, we have
By Theorem 10.16(b), equal left-multiplication transformations correspond to equal matrices, so \(A(BC)=(AB)C\).
Exercise¶
Exercise 10.8
State and prove a more general result of Theorem 10.2 involving linear transformations with domains unequal to their codomains.
Exercise 10.11
Let \(V\) be a vector space, and let \(T: V \rightarrow V\) be linear. Prove that \(T^{2}=T_{0}\) if and only if \(R(T) \subseteq N(T)\).
Exercise 10.13
Let \(A\) and \(B\) be \(n \times n\) matrices. Recall that the trace of \(A\) is defined by
Prove that \(\operatorname{tr}(A B)=\operatorname{tr}(B A)\) and \(\operatorname{tr}(A)=\operatorname{tr}\left(A^{t}\right)\).
Exercise 10.15
Let \(M\) and \(A\) be matrices for which the product matrix \(M A\) is defined. If the \(j\) th column of \(A\) is a linear combination of a set of columns of \(A\), prove that the \(j\) th column of \(M A\) is a linear combination of the corresponding columns of \(M A\) with the same corresponding coefficients.
Exercise 10.16
Let \(V\) be a finite-dimensional vector space, and let \(T: V \rightarrow V\) be linear.
- (a) If \(\operatorname{rank}(T)=\operatorname{rank}\left(T^{2}\right)\), prove that \(R(T) \cap N(\dot{T})=\{0\}\). Deduce that \(V=R(T) \oplus N(T)\).
- (b) Prove that \(V=R\left(T^{k}\right) \oplus N\left(T^{k}\right)\) for some positive integer \(k\).
Exercise 10.17
Let \(V\) be a vector space. Determine all linear transformations \(T: V \rightarrow V\) such that \(T=T^{2}\). Hint: Note that \(x=T(x)+(x-T(x))\) for every \(x\) in \(V\), and show that \(V=\{y: T(y)=y\} \oplus N(T)\).