ENGINEERING:LINEAR ALGEBRA FOR DATA ANALYSIS PART1

Vectors and Matrix

 Vectors are mathematical objects with two or more components. In data analysis a vector can represent the person with components as height and weight.The components of a vector can be represented as two components in a Cartesian coordinate system ,one along x- axis the other along Y-axis; these will be the x and y components. The original vector can be represented alternatively by its length and the angle it makes with the x- axis.

   The components of vector can be represented as multiples of unit vectors called I and j.The same vector can be represented by a different i* and j* unit vectors. This change  in characterisation of a vector is called change in basis; the vector total magnitude and angle remains unchanged.Change in basis is achieved by projection of components of a vector to the unit vectors of new basis. Projection is thus basically how much the component aligns in the new direction. The projection is achieved by a process called dot product. Let (2 1) be new i* and (1 2) be new j*.If the original vector is (1 1) we have to calculate the projection of both components in new basis i* and j*.

  The projection is calculated by the dot product

  (2 1)    (1)   Where we multiply the new unit vectors on right side to the original vectors on left side.

  (1 2)    (1)

This comes to (2.1+1.1 )  Or (3)

                        (1.1+2.1)       (3)

The logic of this multiplication is that in upper row the original I component of the vector is projected to newi* component and this projection is aligned in its direction so magnitudes are multiplied.Since projection is a scalar quantity both magnitudes of i* are added. Same goes for the y component of the vector which is multiplied and added to j*.

  The overall identity can be taken as an example of multiplication of a matrix of i* and j* to a vector which results in a new vector with a new component.

Determinants and determination of unknown vectors

      Determinant is the area in coordinate space substended by the angle of rotation between original vector and transformed vector by new basis.Change in basis results in change in angle in coordinate space. However if angle is unchanged it can represent scaling or increase or decrease in magnitude of original vector. For areas substeded which are  square in shape the area change is simply  denoted by multiplying the increase in sides of the square. In other cases the formula to calculate the area or determinant is (ad-bc).

  Thus the matrix we have introduced earlier has a determinant of (2.2-1.1). = 3.

  A matrix can have an inverse ,just like inverse of 5 is 1/5. However there are some added conditions for inverse matrix and these are the original matrix can have a determinant only if it is square ; lastly determinant of a matrix should not be zero if it has to have an inverse. The last is so because if determinat after rotation is zero there is a loss of dimension and the reverse transformation cannot be done. To calculate the inverse the formula is adjuvant A/ determinant of A.The adjuvant is the matrix of cofactors of matrix A. Cofactors are for 2x2 matrix are the numbers left behind after removing a line and row of the matrix. Thus for our example the elimination of first line and row leads to remaining number 2. Similarly all four cofactors are calculated and form a matrix.

When a matrix is multiplied by a vector and it results in a new vector it can be represented as an equation.

     A.x= b where A is a matrix of basis acting on vector x to change it to vector b. X is an unknown vector and we need to find it’s values x1 and x2.

Let determinant A be      (-5 2) and vector b be (-2)

                                         (-4 1)                         (-4)

To calculate x2 we need to know matrix A determinant which is the area rotated due to transformation of x2 to new vector (-2). The determinant is +3. We also need to find a determinant or area formed by X2 with unit vector i*. This matrix is formed by i* and X2. (-5   -2)

                                                                                            (-4  -4)

   The determinant of this new matrix is 12. 

       Since area of X2 = det A . X2 we can find X2 = 12/ 3=4. Similarly X1 is calculatedas 2.

Eigenvalue equation and matrix decomposition

  The vector v when transformed by a matrix land on themselves are called eigenvectors.To be a eigenvector has to be thus a scaled version of original vector v.Thus A.v=lamda .v

    Where lambda L is a scaling factor.

  Thus (A-I.L)v=0 where I is equal to 1 or Identity matrix with all diagonal equal to 1 and rest 0.

 The  determinant of this matrix has to be 0 as these are scaled versions.

Thus det(A-L.I)   =  0

Taking a concrete example (-5   2)   - (L    O)   = 0

                                            (-4   1)      (O    L)

  Thus L.L +4L +3=0

Solving this L =-3 and L =-1. These are the eigenvalues.

Putting L=-3 in matrix equation we get

-2v1+ 2v2=0

-4v1+4v2=0

Solving above leads to v1 =v2=1

So for L-3 the eigenvector is (1. 1). 

Similarly for L-1 the eigen vector is (1/2. 1)

We can represent the eigen  value equation as A.P = P.V. 

Where P are the eigen vector matrix (1. 1)      and V is eigen value matrix (-3  0)

                                                           (1/2  1)                                                (0  -1)

                 This equation allows decomposition of matrix A into three matrices 

    A= P.V.Pinverse

ACK:  Before machine learning Vol 1                                          .