Basis Change:
In this blog we will see how the transformation matrix for any linear mapping changes with the change in the basis. Let's assume we have a linear mapping 
with 
& 
, ordered bases for vector spaces 
and 
are  "B=(b_1,...,b_n)")
and  "C=(c_1,...,c_m)")
respectively and we are changing these bases to  "\tilde{B}=(\tilde{b}_1,...,\tilde{b}_n)")
and  "\tilde{C}=(\tilde{c}_1,...,\tilde{c}_m)")
for vector spaces and respectively. Also assume the transformation matrix in case of bases 
& 
is 
and after changing the bases to 
& 
it becomes 
.
In this blog I will be taking subscripts 
, 
, 
and 
to represent vectors in the bases , , and respectively.
 \rightarrow \tilde{B}(\tilde{b}_j) "B(b_i) \rightarrow \tilde{B}(\tilde{b}_j)")
 \rightarrow \tilde{C}(\tilde{c}_l) "C(c_k) \rightarrow \tilde{C}(\tilde{c}_l)")
As we know basis vectors spans the entire vector space, so will span the entire vector space , so will also span the basis vectors of .
So we can say that each new basis vector 
of will be the linear combinations of the basis vectors 
of and we can write it as,  "\tilde{b}_j = \sum(s_{ij} \cdot b_i)")
.
We store these coefficients in column-wise manner in a matrix (say 
), so entries in the 
column of will be the coefficients for , we can represent as, 
.
Similarly assume for and we denote the coefficient matrix by 
, so we can represent it as, 
and  "\tilde{c}_l = \sum_{k=1}^{m}(t_{kl} \cdot c_k)")
.
is linearly mapping vectors from to so we can write,
 = \sum_{l=1}^{m} \tilde{a}_{lj} \cdot \tilde{c}_l = \sum_{l=1}^{m} \tilde{a}_{lj} \sum_{k=1}^{m} t_{kl} \cdot c_k = \sum_{k=1}^{m} \left( \sum_{l=1}^{m} t_{kl} \cdot \tilde{a}_{lj} \right) \cdot c_k "\phi(\tilde{b}_j) = \sum_{l=1}^{m} \tilde{a}_{lj} \cdot \tilde{c}_l = \sum_{l=1}^{m} \tilde{a}_{lj} \sum_{k=1}^{m} t_{kl} \cdot c_k = \sum_{k=1}^{m} \left( \sum_{l=1}^{m} t_{kl} \cdot \tilde{a}_{lj} \right) \cdot c_k")
Also we can write,
 = \phi \left(\sum_{i=1}^{n}s_{ij} b_i \right) = \sum_{i=1}^{n} s_{ij} \phi(b_i) = \sum_{i=1}^{n} s_{ij} \sum_{k=1}^{m} \left(a_{ki}s_{ij} \right) c_k = \sum_{k=1}^{m} \left( \sum_{i=1}^{n} a_{ki} s_{ij} \right) c_k "\phi(\tilde{b}_j) = \phi \left(\sum_{i=1}^{n}s_{ij} b_i \right) = \sum_{i=1}^{n} s_{ij} \phi(b_i) = \sum_{i=1}^{n} s_{ij} \sum_{k=1}^{m} \left(a_{ki}s_{ij} \right) c_k = \sum_{k=1}^{m} \left( \sum_{i=1}^{n} a_{ki} s_{ij} \right) c_k")
Now from above two equations we can write,
So, this way we can compute the transformation matrix .
Matrix approximation with Singular Value Decomposition(SVD):
SVD factors a rectangular matrix "\left( A \in \mathbb{R}^{m \times n} \right)")
into three parts in the following manner:
, here 
and are called left and right-singular vector matrices. Left and right-singular vectors of are eigenvectors of 
and 
respectively. Matrix is a diagonal matrix and the diagonal elements (
) are called singular values and are square root of eigenvalues of or .
We construct a rank-1 matrix 
as,
this way we can approximate as a rank- matrix as:
=\sum_{i=1}^{k} \sigma_{i} u_{i} v_{i}^{T} = \sum_{i=1}^{k} \sigma_{i} A_{i} "\tilde{A}(k)=\sum_{i=1}^{k} \sigma_{i} u_{i} v_{i}^{T} = \sum_{i=1}^{k} \sigma_{i} A_{i}")
Remark: We take singular vectors corresponding to the large singular values.
Now, I will be considering an image of a Pug (say 'I'), we will see how the rank-1 approximation looks like visually and how a set of rank (k = [1,2,3,4,10,20,50,100]) approximation looks.
Images I1 to I4 are the outer product of 
with 
. Rank-1 approximation is same as I1, rank-2 approximation is the sum of two outer products 
or 
, similarly rank-3 approximation is 
or 
.
References:
- mml-book
Comments
Post a Comment