ENGINEERING:LINEAR ALGEBRA FOR DATA ANALYSIS: PART2

Single value decomposition

   A eigenvalue matrix can be decomposed into three matrices ,which represent two rotation of the vector and one scaling. A= P.V.Pinverse

   A square ,symmetrical matrix can also be decomposed by a different method called single value decomposition. Suppose we have a circle with orthogonal radiii as vectors ,v1 and v2. If we stretch the circle into an ellipse these vectors change into new vectors , the major and minor axis.To allow this transformation of vectors the angle between them has to be orthogonal. Also the stretching can be incorporated by a scaling factor when the norm of the vector remains unchanged during the transformation. These properties occur in orthonormal matrices. 

    So the transformation of radii vectors can be represented by A. v= sigma. u. Here u matrix is called Principal Axis and sigma or scaling factor is called single value.

   where v and u are orthonormal matrices whose norm and angles are preserved during rotation. Sigma is a scaling factor whhich represents the magnitude of stretching of the radius.

The above equation can also be written as A= U.sigma.Vt

Thus matrix A is decomposed into three matrices .where U and V are orthonormal matrices and sigma is a scaling matrix with diagonal congaing values and rest 0.

    However A is not a square or symmetrical matrix. It can be converted to one by multiplying by At.

So. A.At=(U.sigma.Vt)t.U.Sigma.Vt

A,At=V.sigma square.Vt

If Vt=Vinverse

Then At.A.V=V.sigma square

Let’s say that At.A is equal to A and V is vector ,then sigma square is a eigenvalue matrix.So we can use it in a form A,v= sigma.v which is a eigenvalue equation. Solving this equation gives us value of matrix. A similar eigenvalue equation can be used to get values of matrix U. Matrix sigma is a diagonal matrix whose diagonal values are the eigenvalues.

Data analysis by SVD

 SVD can be used for finding latent relation ship between data matrices.Suppose we have three  matrices A,Uand V.Acontains data of eight individuals and their rating of eight songs. The eigenvalue equation can be solved to get values for U matrix contains data of songs and their contexts, V matrix represents data of genre and songs. We can calculate matrix sigma as matrix of eigenvalues.

  Since these are large matrices we can make analysis easier by taking first three single values of the matrix or first  three rows and first three columns to constitue new U,V and Sigma matrices. When we now place the data of these matrices in a coordinate axis form we can readily make out clustering into groups. These groups are the hidden relationships between individuals and contexts or individuals and genre of songs. These hidden relationships can be found visually but also by clustering algorithms.

Principal component analysis