There is hardly any theory which is more elementary than linear algebra, inspite of the fact that generations of

Data scientist and  Mathematicians have obscured its simplicity by preposterous calculations with matrices 

            -Jean Dieudonnie French Mathematician

Mathematics requires a small dose, not of genius but of an imaginative freedom which in a larger dose would be insanity

               – August K Rodgers

It’ll be about linear algebra, which as a lot of you know is one of these subjects that’s required knowledge

for just about any technical education, but it’s also I have noticed generally poorly understood by IT professionals

taking it for the first time. An Data Scientist might go through a class and learn how to compute lost of things, like determinant,

matrices multiplication or cross product which use the determinant or eigenvalues, but they might come out

without really understanding why matrix multiplication is defined the way that it is, why cross product has

anything to do with the determinant or what an eigenvalues really represent.

In this blog I have tried to help all those data scientist to re look of Linear algebra in altogether in a different

perspective in terms of Analytical Geometry or 3 Dimensional Spatial Geometry and help them understand how

this can be easily put to use in Computer vision , Image processing, Speech recognition, and Robotics.

So Let’s start with some basics viz; Vector Space.

 

        Fields and Vector Spaces As has been indicated in the preface, our first aim is to re-work

the linear algebra presented in our AAS course, generalising from R to an arbitrary field as domain

from which coefficients are to be taken. Fields are treated in the companion course, Rings and

Arithmetic, but to make this course and these notes self-contained we begin with a definition of

what we mean by a field.

Axioms for Fields 

A field is a set F with distinguished elements 0, 1, with a unary operation −, and with two binary operations +

and × satisfying the axioms below (the ‘axioms of arithmetic’).

Conventionally, for the image of (a, b) under the function + : F × F → F we write a + b; for the image of (a, b)

under the function × : F × F → F we write a b; and x − y means x + (−y), x + y z means x + (y z).

The axioms of arithmetic

(1) a + (b + c) = (a + b) + c        [+ is associative] 

(2) a + b = b + a                   [+ is commutative]           

(3) a +0= a                                

(4) a + (−a) = 0 

(5) a (b c) = (a b) c                 [× is associative]                

(6) ab = ba                               [× is commutative] 

(7) a 1 = a 

(8) a 6 = 0 ⇒ ∃ x ∈ F : a x = 1 

(9) a (b + c) = a b + a c                            [× distributes over +]  

(10) 0 ≠ 1  

 

Note 1: All axioms are understood to have ∀ a, b, . . . ∈ F in front.

Note 2: See my Notes on Rings and Arithmetic for more discussion.

Note 3: Examples are Q, R, C, Z 2 or more generally Z p , the field of integers modulo p for any prime number p.

Vector Spaces

Let F be a field. A vector space over F is a set V with distinguished element 0, with a unary operation −,

with a binary operation +, and with a function × : F × V → V satisfying the axioms below.

Conventionally: for the image of (a, b) under the function + : V × V → V

we write a + b; for a + (−b) we write a − b; and for the image of (α, v)

under the function × : F × V → V we write α v .

Vector space axioms

$$\begin{gathered} \left(1\right) u + (v + w) = (u + v) + w \\ \left(2\right) u + v = v + u \\ \left(3\right) u +0= u \\ \left(4\right)u + (-u) = 0 \left(5\right) \alpha (\beta v) = (\alpha \beta ) v \\ \left(6\right) \alpha (u + v) = \alpha u + \alpha v \\ \left(7\right) (\alpha + \beta ) v = \alpha v + \beta v \\ \left(8\right) 1 v = v \end{gathered}$$

Note: All axioms are understood to have appropriate quantifiers ∀ u, v, . . . ∈ V
and/or ∀ α, β, . . . ∈ F in front.

EXAMPLES:

1. $F^n is a vector space over F ;$
2. the polynomial ring F [x]  is a vector space over F ;
3. $M_{ m\times n} (F ) select: is a vector space over F ;$
4. if K is a field and F a sub-field then K is a vector space over F ;

Exercise 1. Let X be any set, F any field. Define

$$\begin{gathered} F^{ X} to be the set of all functions X \to F with the usual point–wise addition and \\ multiplication by scalars (elements of F ). Show that F^{ X} is a vector space over F \end{gathered}$$

Exactly as for rings, or for vector spaces over R, one can prove important “trivialities”.

Of course, if we could not prove them then we would add further axioms until we

had captured all properties that we require, or at least expect, of the algebra of vectors.

But the fact is that this set of axioms, feeble though it seems, is enough. For example:

PROPOSITION: Let V be a vector space over the field F . For any v ∈ V and any α ∈ F we have

$$\begin{gathered} \left(i\right) 0 v = 0; \\ \left(ii\right) \alpha 0 = 0; \\ \left(iii\right) if \alpha v = 0 then \alpha = 0 or v = 0; \\ \left(iv\right) \alpha (-v) = -(\alpha v) = (-\alpha ) v ; \end{gathered}$$

Proof. For v ∈ V , from Field Axiom (3) and Vector Space Axiom (7) we have
0 v = (0 + 0) v = 0 v + 0 v.

Then adding −(0 v) to both sides of this equation and using Vector Space Axioms (4)
on the left and (1), (4), (3) on the right, we get that 0 = 0 v , as required for (i). The
reader is invited to give a proof of (ii).

For (iii), suppose that α v = 0 and α 6 = 0: our task is to show that v = 0. By Field
Axioms (8) and (6) there exists β ∈ F such that β α = 1. Then

v = 1 v = (β α) v = β (α v) = β 0

by Vector Space Axioms (8) and (5). But β 0 = 0 by (ii), and so v = 0, as required.
Clause (iv), like Clause (ii), is offered as an exercise:

Subspaces

Let V be a vector space over a field F . A subspace of V is a subset U such that
(1) 0 ∈ U and u + v ∈ U whenever u, v ∈ U
(2) if u ∈ U , α ∈ F then α u ∈ U . and −u ∈ U whenever u ∈ U ;

Note 1: Condition (1) says that U is an additive subgroup of V . Condition (2) is closure under

multiplication by scalars.

$Note 2: We write U\ne V to mean that U is a subspace of V .$ $$\begin{gathered} Note 3: Always \left\{0\right\} is a subspace; if U\ne \left\{0\right\} then we say that U is non–zero or non–trivial. \\ Likewise, V is a subspace; if U\ne V then we say that U is a proper subspace. \end{gathered}$$ $$\begin{gathered} Note 4: A subset U of V is a subspace if and only if U\ne \varnothing and U is closed \\ under + (that is, u, v \in U \Rightarrow u + v \in U ) and under multiplication by scalars. \end{gathered}$$

EXAMPLES:

$$\begin{gathered} \left(1\right) Let {\text{\hspace{0.17em}L}}_1 , . . . , L_m be homogeneous linear expressions \sum c_{ij} x_j with \\ coefficients c_{ ij} \in F , and let U :=\left\{\right(x_{ 1} , . . . , x_{ n} ) \in F^{ n} | L_1 = 0, . . . , L_m = 0\}. \\ Then U⩽ F^{ n} \end{gathered}$$ $\left(2\right) Let F^{ \left[n\right]} \left[x\right] := \{f \in F [x] | f = 0 or deg f \le n\}. Then F^{ \left[n\right]} [x] \le F [x].$ $\left(3\right) Upper triangular matrices form a subspace of M_{ n\times n} (F )$

 QUOTIENT SPACES

Suppose that U <= V where V is a vector space over a field F . Define the quotient
space V /U as follows:

set := {x + U | x ∈ V }   [additive cosets]

0 := U

additive inverse: −(x + U ) := (−x) + U

addition: (x + U ) + (y + U ) := (x + y) + U

$multiplication by scalars: \alpha (x + U ) :=\alpha x +\hspace{0.17em}U$

Note: The notion of quotient space is closely analogous with the notion of quotient
of a group by a normal subgroup or of a ring by an ideal. It is not in the Part A syllabus,
nor will it play a large part in this course. Nevertheless, it is an important and useful
construct which is well worth becoming familiar with.

Dimension

Although technically new, the following ideas and results translate so simply from
the case of vector spaces over R to vector spaces over an arbitrary field F that I propose
simply to list headers for revision:
(1) spanning sets;
linear dependence and independence;
bases;
(2) dimension;

$\left(3\right) dim V = d \Rightarrow V\cong F d ;$

(4) any linearly independent set may be extended (usually in many ways) to a basis;
(5) intersection U ∩ W of subspaces; sum U + W ;

(6) dim (U + W ) = dim U + dim W − dim (U ∩ W );

LINEAR TRANSFORMATION

$Let F be a field, V_1, V_2 vector spaces over F.$

$A map T : V_{ 1 }\to V_{ 2} is said to be linear if$

T 0 = 0,
T (−x) = −T x,
T (x + y) = T x + T y,
and T (λ x) = λ (T x)
for all x, y ∈ V and all λ ∈ F .

Note 1: The definition is couched in terms which are intended to emphasise that
what should be required of a linear transformation  is
that it preserves all the ingredients, 0, +, − and multiplication by scalars, that go into

$the making of a vector space. What we use in practice is the fact that T : V_{ 1} \to V{ }_2 is$

linear if and only if T (α x + β y) = α T x + β T y  for all x, y ∈ V and all α, β ∈ F .

Note 2:

$The identity I : V \to V is linear; if T : V{ }_1 \to V_{ 2} and S : V{ }_2 \to V{ }_3 are linear then S ◦ T : V{ }_1 \to V{ }_3 is linear.$

For a linear transformation T : V → W we define the kernel or null-space by Ker T := {x ∈ V | Tx = 0}.

$$\begin{gathered} We prove that Ker T⩽ V, that Im T \le W and we define \\ nullityT:\hspace{0.17em}dim Ker T, \\ rank\hspace{0.17em}T\hspace{0.17em}:= dim Im T. \end{gathered}$$

RANK NULLITY THEOREM:  

nullity T + rank T = dim V

Note: The Rank-Nullity Theorem is a version of the First Isomorphism Theorem
for vector spaces, which states that

$Im T\cong V/Ker T.$

DIRECT SUMS AND PROJECTION OPERATORS 

The vector space V is said to be the direct sum of its subspaces U and W , and we
write V = U ⊕ W , if V = U + W and U ∩ W = {0}.

Lemma: Let U , W be subspaces of V . Then V = U ⊕ W if and only if for every
v ∈ V there exist unique vectors u ∈ U and w ∈ W such that v = u + w.

Proof. Suppose first that for every v ∈ V there exist unique vectors u ∈ U and

w ∈ W such that v = u + w . Certainly then V = U + W and what we need to prove

is that U ∩ W = {0}. So let x ∈ U ∩ W . Then x = x + 0 with x ∈ U and 0 ∈ W .

Equally, x = 0 + x with 0 ∈ U and x ∈ W . But the expression x = u + w with u ∈ U

and w ∈ W is, by assumption, unique, and it follows that x = 0. Thus U ∩ W = {0}, as required.

Now suppose that V = U ⊕ W . If v ∈ V then since V = U + W there certainly exist vectors u ∈ U

and w ∈ W such that v = u + w . The point at issue therefore is: are u, w uniquely determined by v?

Suppose that u + w = u ′ + w ′ , where u, u ′ ∈ U and w, w ′ ∈ W . Then u − u ′ = w ′ − w .

This vector lies both in U and in W .

By assumption, U ∩ W = {0} and so u − u ′ = w ′ − w = 0. Thus u = u ′ and w = w ′ ,

so the decomposition of a vector v as u + w with u ∈ U and w ∈ W is unique, as required.

Note: What we are discussing is sometimes (but rarely) called the “internal” direct

sum to distinguish it from the natural construction which starts from two vector spaces

$V_{ 1} , V_{ 2} over the same field F and constructs a new vector space whose set is the product$ $set V{ }_1 \times V_{ 2} and in which the vector space structure is defined componentwise—compare$

the direct product of groups or of rings. This is (equally rarely) called the “external”

$direct sum. These are two sides of the same coin: the external direct sum of V_{ 1} and V { }_2$ $is the internal direct sum of its subspaces V{ }_1 \times \left\{0\right\} and \left\{0\right\} \times V{ }_2 ; while if V= U \oplus W$

then V is naturally isomorphic with the external direct sum of U and W .

We come now to projection operators. Suppose that V = U ⊕ W .

Define P : V → V as follows.

For v ∈ V write v = u + w where u ∈ U and w ∈ W and then define

P v := u. Strictly P depends on the ordered pair (U, W ) of summands of V , but to

keep things simple we will not build this dependence into the notation.

OBSERVATIONS:

(1) P is well-defined;
(2) P is linear;
(3) Im P = U , Ker P = W ;

$\left(4\right) P^2 = P$

Proofs. That P is well-defined is an immediate consequence of the existence and
uniqueness of the decomposition v = u + w with u ∈ U , v ∈ V .

$$\begin{gathered} To see that P is linear, \\ let v_1 , v_2 \in V and {\alpha }_{ 1} , {\alpha }_{ 2} \in F (where, as always, F is the field of scalars). \end{gathered}$$ $$\begin{gathered} Let v_1 = u_1 + w_1 and v_2 = u_2 + w_2 be the decompositions of v_1 and v_2. \\ Then P v_1 = u_1 and P v_2 = u_2 . \end{gathered}$$ $$\begin{gathered} What about P ({\alpha }_1 v_{ 1} + {\alpha }_2 v_2 )? \\ Well, {\alpha }_1 v_1 + {\alpha }_2 v_2 \end{gathered}$$ $= {\alpha }_1 (u_1 + w_1 ) + {\alpha }_2 (u_2 + w_2 ) = ({\alpha }_1 u_1 + {\alpha }_2{u}_2 ) + ({\alpha }_1{w}_1 + {\alpha }_2{w}_2 )$ $$\begin{gathered} Since {\alpha }_1{u}_1 + {\alpha }_2{u}_2 \in U and {\alpha }_1{w}_1 + {\alpha }_2{w}_2 \in W it follows that \\ P ({\alpha }_1{v}_1 + {\alpha }_2{v}_2 ) = {\alpha }_1{u}_1 + {\alpha }_2{u}_2. \end{gathered}$$ $$\begin{gathered} Therefore P ({\alpha }_1{v}_1 + {\alpha }_2{v}_2 ) = {\alpha }_{1}P\left(v_1\right) + {\alpha }_{2}P\left(v_2\right), \\ that is, P is linear. \end{gathered}$$

For (3) it is clear from the definition that Im P ⊆ U ; but if u ∈ U then u = P u,

and therefore Im P = U . Similarly, it is clear that W ⊆ Ker P ;

but if v ∈ Ker P then

v = 0 + w for some w ∈ W , and therefore Ker P = W .

Finally, if v ∈ V and we write v = u + w with u ∈ U and w ∈ W then

$P^{ 2 }v = P (P v) = P u = u = P v ,$ $and this shows that P^2 = P , as required.$

Terminology: the operator (linear transformation) P is called the projection of V onto U along W.

$$\begin{gathered} \mathrm{Note} 1. \mathrm{Suppose} \mathrm{that} \mathrm{V} \mathrm{is} \mathrm{finite}–\mathrm{dimensional}. \mathrm{Choose} \mathrm{a} \mathrm{basis} {\mathrm{u}}_{ 1} , . . . , \mathrm{u}{ }_{\mathrm{r}} \mathrm{for} \mathrm{U} \mathrm{and} \mathrm{a} \mathrm{basis} {\mathrm{w}}_{ 1} , . . . , \mathrm{w}{ }_{\mathrm{m}} \mathrm{for} \mathrm{W} . \\ \mathrm{Then} \mathrm{the} \mathrm{matrix} \mathrm{of} \mathrm{P} \mathrm{with} \mathrm{respect} \mathrm{to} \mathrm{the} \mathrm{basis} \mathrm{u}{ }_1 , . . . , \mathrm{u}{ }_{\mathrm{r}} , \mathrm{w}{ }_1 , . . . , \mathrm{w}{ }_{\mathrm{m}} \\ \mathrm{of} \mathrm{V} \mathrm{is} \left({\mathbf{I}}_{\mathbf{r}}\mathbf{ }\mathbf{ }\mathbf{0} \\ \mathbf{0}\mathbf{ }\mathbf{ }\mathbf{0}\right) \end{gathered}$$

 

Note 2. If P is the projection onto U along W then I − P , where I : V → V is the identity transformation,

is the projection onto W along U .

Note 3. If P is the projection onto U along W then u ∈ U if and only if P u = u. The fact that if u ∈ U then

P u = u is immediate from the definition of P , while if P u = u then obviously u ∈ Im P = U .

Our next aim is to characterise projection operators algebraically. It turns out that Observation (4) above is the key:

TERMINOLOGY :

$\mathit{A}\mathit{n}\mathbf{ }\mathit{o}\mathit{p}\mathit{e}\mathit{r}\mathit{a}\mathit{t}\mathit{o}\mathit{r}\mathbf{ }\mathit{T}\mathbf{ }\mathit{s}\mathit{u}\mathit{c}\mathit{h}\mathbf{ }\mathit{t}\mathit{h}\mathit{a}\mathit{t}\mathbf{ }{\mathit{T}}^{\mathbf{2}}\mathbf{ }\mathbf{=}\mathbf{ }\mathit{T}\mathbf{ }\mathit{i}\mathit{s}\mathbf{ }\mathit{s}\mathit{a}\mathit{i}\mathit{d}\mathbf{ }\mathit{t}\mathit{o}\mathbf{ }\mathit{b}\mathit{e}\mathbf{ }\mathit{i}\mathit{d}\mathit{e}\mathit{m}\mathit{p}\mathit{o}\mathit{t}\mathit{e}\mathit{n}\mathit{t}\mathbf{.}$

 

THEOREM : A linear transformation is a projection operator if and only if it is idempotent.

Proof: We have seen already that every projection is idempotent, so the problem is to prove that an idempotent

linear transformation is a projection operator. Suppose that P : V → V is linear and idempotent. Define

$\mathit{U}\mathbf{ }\mathbf{:}\mathbf{=}\mathbf{ }\mathit{I}\mathit{m}\mathbf{ }\mathit{P}\mathbf{,}\mathbf{ }\mathit{W}\mathbf{ }\mathbf{:}\mathbf{=}\mathbf{ }\mathit{K}\mathit{e}\mathit{r}\mathbf{ }\mathit{P}\mathbf{.}$ $$\begin{gathered} Our first task is to prove that V = U \oplus W . Let v \in V . Then v = Pv + (v - Pv). \\ Now Pv \in U \left(obviously\right), and P(v - Pv) = Pv - P^{2}v = 0, so v - Pv \in W . \\ Thus V = U + W . Now let x \in U \cap W . Since x \in U there exists y \in V such that \\ x = Py , and \sin ce x \in W also Px = 0. Then x = Py = P^2 y = Px = 0. \\ Thus U \cap W = \left\{0\right\}, and so V = U \oplus W . \end{gathered}$$

To finish the proof we need to convince ourselves that P is the projection onto U along W .

For v ∈ V write v = u + w where u ∈ U and w ∈ W . Since u ∈ U there must exist x ∈ V such that
u = P x. Then

 

${\mathbf{\text{Pv = P(u + w) = Pu + Pw = P}}}^{\mathbf{2}}\mathbf{ }\mathit{x}\mathbf{ }\mathbf{+}\mathbf{ }\mathbf{0}\mathbf{ }\mathbf{=}\mathbf{ }\mathit{P}\mathit{x}\mathbf{ }\mathbf{=}\mathbf{ }\mathit{u}\mathbf{ }\mathbf{,}$

 

and therefore P is indeed the projection onto U along W , as we predicted.

The next result is a theorem which turns out to be very useful in many situations.
It is particularly important in applications of linear algebra in Quantum Mechanics.

 

THEOREM :

$Let P : V \to V be the projection onto U along W and$ $\mathit{l}\mathit{e}\mathit{t}\mathbf{ }\mathit{T}\mathbf{ }\mathbf{:}\mathbf{ }\mathit{V}\mathbf{ }\mathbf{\to }\mathbf{ }\mathit{V}\mathbf{ }\mathit{b}\mathit{e}\mathbf{ }\mathit{a}\mathit{n}\mathit{y}\mathbf{ }\mathit{l}\mathit{i}\mathit{n}\mathit{e}\mathit{a}\mathit{r}\mathbf{ }\mathit{t}\mathit{r}\mathit{a}\mathit{n}\mathit{s}\mathit{f}\mathit{o}\mathit{r}\mathit{m}\mathit{a}\mathit{t}\mathit{i}\mathit{o}\mathit{n}\mathbf{.}\mathbf{ }\mathit{T}\mathit{h}\mathit{e}\mathit{n}\mathbf{ }\mathit{P}\mathbf{ }\mathit{T}\mathbf{ }\mathbf{=}\mathbf{ }\mathit{T}\mathbf{ }\mathit{P}\mathbf{ }$ $\mathit{i}\mathit{f}\mathbf{ }\mathit{a}\mathit{n}\mathit{d}\mathbf{ }\mathit{o}\mathit{n}\mathit{l}\mathit{y}\mathbf{ }\mathit{i}\mathit{f}\mathbf{ }\mathit{U}\mathbf{ }\mathit{a}\mathit{n}\mathit{d}\mathbf{ }\mathit{W}\mathbf{ }\mathit{a}\mathit{r}\mathit{e}\mathbf{ }\mathit{T}\mathbf{ }\mathbf{–}\mathit{i}\mathit{n}\mathit{v}\mathit{a}\mathit{r}\mathit{i}\mathit{a}\mathit{n}\mathit{t}\mathbf{ }\mathbf{(}\mathit{t}\mathit{h}\mathit{a}\mathit{t}\mathbf{ }\mathit{i}\mathit{s}\mathbf{ }\mathit{T}\mathbf{ }\mathit{U}\mathbf{\le }\mathbf{ }\mathit{U}\mathbf{ }\mathit{a}\mathit{n}\mathit{d}\mathbf{ }\mathit{T}\mathbf{ }\mathit{W}\mathbf{ }\mathbf{\le }\mathbf{ }\mathit{W}\mathbf{ }\mathbf{)}\mathbf{.}$

Proof : Suppose first that P T = T P . If u ∈ U , so that P u = u, then T u = T P u = P T u ∈ U , so T U 6 U .

$And if w \in W then P (T w) = T P w = T 0 = 0, and therefore T W \le W.$

       Now suppose conversely that U and W are T -invariant. Let v ∈ V and write

v = u + w with u ∈ U and w ∈ W . Then

P T v = P T (u + w) = P (T u + T w) = T u ,

since T u ∈ U and T w ∈ W . Also,

T P v = T P (u + w) = T u .

Thus P T v = T P v for all v ∈ V and therefore P T = T P , as asserted.

$$\begin{gathered} We turn now to direct sums of more than two subspaces. The vector space V \\ is said to be the direct sum of subspaces U_1,.. . . . , U_k if for every v \in V there \\ exist unique vectors u_i \in U_i for 1 \le i\le k such that v = u_1 + · · · + u_{{}_k} . \\ We write V = U_1\oplus · · · \oplus U_{{}_k} . \end{gathered}$$

Note 1: If k = 2 then this reduces to exactly the same concept as we have just been
studying.

$Moreover, if k > 2 then U{ }_1 \oplus U_{ 2} \oplus · · · \oplus U{ }_k = (· · · ((U{ }_1 \oplus U{ }_2 ) \oplus U{ }_3 ) \oplus · · · \oplus U{ }_k ).$

Note 2: 

$$\begin{gathered} If U_i \le V for 1\le i \le k then V = U_1 \oplus · · · \oplus U_k if and only if V = U_1 + U_2 + · · · + U_k and \\ U_r \cap ( \sum _{i\ne r} U_i ) = \left\{0\right\} for 1 \le r \le k . It is NOT sufficient that \\ {U}_i \cap U_j = \left\{0\right\} whenever i\ne j . Consider, for example, the 2–dimensional space F^2 of pairs (x_1 , x_2 ) with x_1 , x_2 \in F . Its three subspaces \end{gathered}$$

 

$$\begin{gathered} U_{ 1} := \left\{\right(x, 0) | x \in F \}, \\ {U}_{ 2} := \{(0, x) | x \in F \}, \\ {U}_{ 3} := \left\{\right(x, x) | x \in F \} \end{gathered}$$

satisfy

U 1 ∩ U 2 = U 1 ∩ U 3 = U 2 ∩ U 3 = {0}

and yet it is clearly not true that

$F^2$

is their direct sum.

 

$$\begin{gathered} Note 3: If V = U_1 \oplus U_2 \oplus · · · \oplus U_k and B_i is a basis of U_i then B_1 \cup B_2 \cup · · · \cup B_{k } \\ is a basis of V . In particular, dim V = \sum _{i=1}^k dim U_i . \\ Let P_1 , . . . , P_k be linear mappings V \to V such that {P_i}^2 = P_i for all i and P_i{P}_j = 0 \\ whenever i \ne j . If P_1 + · · · + P_k = I then \{P_1 , . . . , P_k \} is known as a partition \\ of the identity on V . \end{gathered}$$

 

EXAMPLE : If P is a projection then {P, I − P } is a partition of the identity.

THEOREM: 

$$\begin{gathered} If V = U_1 \oplus · · · \oplus U_k and P_i is the projection of V onto U_i along {\oplus }_{j\ne i} U_j \\ then \{P_1 , . . . , P_k \} is a partition of the identity on V. Conversely, if \{P_1 , . . . , P_k \} \\ is a partition of the identity on V and U_i := Im P_i then V = U_1 \oplus · · · \oplus U_k . \end{gathered}$$

 

PROOF: 

$$\begin{gathered} Suppose first that V = U_1 \oplus · · · \oplus U_k . Let P_i be the projection of V \\ onto 2 U_i along {\oplus }_{j\ne i} U_j . Certainly {P_i}^2 = P_i , and if i\ne j then Im P_j⩽ Ker P_i \\ so P_i P_j = 0. If v \in V then there are uniquely determined vectors \\ u_i \in U_i for 1 ⩽ i ⩽ k such that v = u_1 + · · · + u_k . Then L P_i v = u_i \\ by definition of what it means for P_i to be projection of V onto U_i along {\oplus }_{j\ne i} U_j . \\ Therefore \\ Iv = v = P_1 v + · · · + P_k v = (P_1 + · · · + P_k ) v . \\ Since this equation holds for all v \in V we have I = P_1 + · · · + P_k . \\ Thus \{P_{ 1} , . . . , P_k \} is a partition of the Identity. \\ To unders\tan d the converse, let \{P_1 , . . . , P_k \} be a partition of the identity \\ on V and let U_i := Im P_i. For v \in V , defining u_i := P_{i}v we have \\ v = Iv = (P_1 + · · · + P_k ) v = P_{1}v + · · · + P_{k}v= u_1 + · · · + u_k . \end{gathered}$$

 

$$\begin{gathered} Suppose that v = w_1 + · · · + w_k where w_i \in U_i for 1 ⩽ i ⩽ k . Then P_i{w}_i = w_i \\ \sin ce P_i is a projection onto U_i. And if j \ne i then P_j w_i = P_j (P_i w_i ) = (P_j{P}_i ) w_i, \\ so P_j{w}_i = 0 \sin ce P_j{P}_i = 0. Therefore \\ P_{i}v = P_i(w_1 + · · · + w_{ k} ) = P_i{w}_1 + · · · + P_i w_k = w_i , that is, w_i = u_i. \\ This shows the uniqueness of the decomposition v = u_1 + · · · + u_k \\ with u_i \in U_i and so V = U_1 \oplus · · · \oplus U_k, and the proof is complete. \end{gathered}$$

 

LINEAR FUNCTIONS AND DUAL SPACES

$$\begin{gathered} A linear functional on the vector space V over the field F is a function f : V \to F \\ such that \\ f ({\alpha }_1{v}_1 + {\alpha }_2{v}_2 ) = {\alpha }_{1}f \left(v_1\right) + {\alpha }_{2}f\left(v_2\right) \\ for all {\alpha }_1 , {\alpha }_2 \in F and all v_1, v_2 \in V . \\ Note: A linear functional, then, is a linear transformation V \to F , where F is \\ construed as a 1–dimensional vector space over itself. \\ Example. If V = F n (column vectors) and y is a 1 \times n row vector then the \\ map v 7\to y v is a linear functional on V . \end{gathered}$$

 

$$\begin{gathered} The dual space V ‘ of V is defined as follows: \\ Set := set of linear functionals on V \\ 0 := zero function [v\mapsto 0 for all v \in V ] \\ (-f )\left(v\right) := -(f (v\left)\right) \\ (f_1 + f_2 )\left(v\right) := f_1 \left(v\right) + f_2\left(v\right) [pointwise addition] \\ (\lambda f )\left(v\right) := \lambda f \left(v\right) [pointwise multiplication by scalars] \end{gathered}$$

 

Note: It is a matter of important routine to check that the vector space axioms are

satisfied. It is also important that, when invited to define the dual space of a vector

space, you specify not only the set, but also the operations which make that set into a vector space.

 

THEOREM :

$$\begin{gathered} Let V be a finite–dimensional vector space over a field F . For every basis v_1 , v_2 , . . . , v_n of V \\ there is a basis f_1 , f_2 , . . . , f_n of V‘ \\ such that f_i (v_j ) =\left\{ 1 if i=j , \\ 0 if i\ne j .\right\} in particular, dim V‘ = dim V \end{gathered}$$ $$\begin{gathered} Proof. Define f i as follows. For v \in V we set f i \left(v\right) := {\alpha }_i where {\alpha }_1 , . . . , {\alpha }_n \in F are \\ such that v = {\alpha }_1{v}_1 + · · · + {\alpha }_n{v}_n. This definition is acceptable because v_1 , . . . , v_n \\ span V and so such scalars {\alpha }_1 , . . . , {\alpha }_n certainly exist; moreover, \sin ce v_1 , . . . , v_n \\ are linearly independent the coefficients {\alpha }_1 , . . . , {\alpha }_n are uniquely determined by v . \\ If w \in V , say w = {\beta }_1{v}_1 + · · · + {\beta }_n{v}_n , and \lambda , \mu \in F then \end{gathered}$$ $$\begin{gathered} f i (\lambda v + \mu w) =f_i\left(\lambda \sum _j{\alpha }_j{v}_j+\mu \sum _j{\beta }_j{v}_j\right) \\ =f_i\left(\lambda \sum _j(\lambda {\alpha }_j+\mu {\beta }_j)v_j\right) \\ = \lambda {\alpha }_i+\mu {\beta }_i \\ =\lambda f_i\left(v\right)+\mu f_i\left(w\right), \end{gathered}$$ $$\begin{gathered} and so f i \in V \prime . We have thus found elements f_{ 1} , . . . , f_{ n} of V \prime \\ such that f_{ i} (v_{ j} ) = \\ \left\{1 if i = j. \\ 0 if i \ne j. \\ \right\} \end{gathered}$$

To finish the proof we must show that they form a basis of V ′ .

To see that they are independent suppose that

$\sum _j\mu j f j = 0,$ $where {\mu }_1 , . . . , {\mu }_n \in F .$ $Evaluate at v_i :$ $$\begin{gathered} 0 = \left(\sum _j{\mu }_j{f}_j\right) \left(v_i\right) = \sum _j{\mu }_j{f}_j\left(v_i\right) ={\mu }_i \\ Thus {\mu }_1 = · · · = {\mu }_n = 0 and so f_1 , . . . , f_n are linearly independent. \\ To see that they span V‘ let f \in V‘ and define g :=\sum _{j}f\left(v_j\right) f_j. Then also g \in V‘ \\ and for 1 ⩽ i ⩽ n we have \\ g(v i ) =\left(\sum _j f\left(v_j\right) f_j\right)\left(v_i\right) = \sum _{j}f\left(v_i\right) f_j\left(v_i\right) = f\left(v_i\right) j \\ Since f and g are linear and agree on a basis of V we have \\ f = g , that is, f =\sum _{j}f\left(v_j\right)f_j. \\ Thus f_1, . . . , f_n is indeed a basis ofV‘ , as the theorem states. \end{gathered}$$

 

$$\begin{gathered} Note. The basis f_1, f_2 , . . . , f_n is known as the dual basis of v_1, v_2 , . . . , v_n . \\ Clearly, it is uniquely determined by this basis of V . \\ EXAMPLE: If V = F^n(n \times 1 column vectors) then we may identify V‘ with the \\ space of 1 \times n row vectors. The canonical basis e_1 , e_2 , . . . , e_n then has \\ dual basis e{‘}_1, e{‘}_2 , . . . , e{‘}_n , the canonical basis of the space of row vectors. \end{gathered}$$

Let V be a vector space over the field F . For a subset X of V the annihilator is defined by

$X^0 := \{f \in V\prime | f\left(x\right) = 0 for all x \in X \} .$ $$\begin{gathered} Note: For any subset X the annihilator X^0 is a subspace. For, if f_1 , f_2 \in X^{◦} and {\alpha }_1 , {\alpha }_2 \in F \\ then for any x \in X({\alpha }_1 f_1 + {\alpha }_2{f}_2 )\left(x\right) = {\alpha }_1 f_1\left(x\right) + {\alpha }_2{f}_2\left(x\right) = 0 + 0 = 0, and so {\alpha }_1{f}_1 + {\alpha }_2{f}_2 \in X^{◦}. \\ Note: X^{◦} = \{f \in V‘ | X \subseteq Ker f \}. \end{gathered}$$

THEOREM

Let V be a finite-dimensional vector space over a field F and let U be
a subspace. Then

$dim U + dim U^{◦} = dim V .$

Proof:  

$$\begin{gathered} Let u_1 , . . . , u_m be a basis for U and extend it to a basis u_1 , . . . , u_m , u_{m+1} , . . . , u_n for V. \\ Thus dim U = m and dim V = n. Let f_1 , . . . , f_n be the dual basis of V‘. We’ll prove that f_{m+1} , . . . , f_n \\ is a basis of U^{◦} . Certainly, if m + 1\le j \le n then f_j (u_i ) = 0 for i \le m and so f_j \in U^{◦} \\ \sin ce u_1 , . . . , u_m span U. Now let f \in U^{◦}. There exist {\alpha }_1 , . . . , {\alpha }_n \in F such that \\ f = \sum _j{\alpha }_j{f}_j . Then \\ f (u i ) = \sum _j{\alpha }_j{f}_j(u_i ) = {\alpha }_i, \\ and so {\alpha }_i = 0 for 1 \le i \le m, that is, f is a linear combination of f_{m+1}, . . . , f_n. \\ Thus f_{m+1}, . . . , f_n span U^{◦} and so they form a basis of it. Therefore \\ dim U^{◦} = n - m, that is, dim U + dim U^{◦} = dim V . \\ For example : Let V be a finite–dimensional vector space over a field F . \\ Show that if U 1 , U 2 are subspaces then (U 1 + U 2 {)}^{◦} = {U_1}^{◦}\cap {U_2}^{◦} \\ and (U_1 \cap U_2 {)}^{◦} = {U_1}^{◦} + U{2}^{◦} . \end{gathered}$$

 

$$\begin{gathered} Response. If f \in {U_1}^{◦} \cap {U_2}^{◦} then f (u_1 + u_2 ) = f\left(u_1\right) + f\left(u_2\right) = 0 + 0 = 0 for any \\ {u}_1\in U_{1}and any u_2 \in U_2. Therefore {U_1}^{◦}\cap {U_2}^{◦} \subseteq (U_1 + U_2 {)}^{◦}. On the other hand, \\ {U}_1 \subseteq U_1 + U_2 and U_2 \subseteq U_1 + U_2 and so if f \in (U_1 + U_2 {)}^{◦} then f \in {U_1}^{◦}\cap {U_2}^{◦}, \\ that is (U_1\cap U_2 {)}^{◦} \subseteq {U_1}^{◦} +{U_2}^{◦}. Therefore in fact (U_1 \cap U_2 {)}^{◦} = {U_1}^{◦} +{U_2}^{◦} . \\ Note that the assumption that V is finite–dimensional is not needed here. \\ For the second part, clearly {U_1}^{◦} \subseteq (U_1 \cap U_2 {)}^{◦} and {U_2}^{◦}\subseteq (U_1 \cap U_2 {)}^{◦} and \\ so {U_1}^{◦} + {U_2}^{◦}\subseteq (U_1 \cap U_2 {)}^{◦}. Now we compare dimensions: \\ dim ({U_1}^{◦} + {U_2}^{◦} ) = dim {U_1}^{◦} + dim {U_2}^{◦}- dim ({U_1}^{◦} \cap {U_2}^{◦}) \\ = dim {U_1}^{◦} + dim {U_2}^{◦} - dim (U_1 + U_2{)}^{◦} [by the above] \\ = (dim V - dim U_1 ) + (dim V - dim U_2 ) - (dim V - dim (U_1 + U_2 \left)\right) \\ = dim V - (dim U_1 + dim U_2 - dim (U{ }_1 + U_2 \left)\right) \\ = dim V - dim (U_1 \cap U_2 ) \\ = dim (U_1 \cap U_2{)}^{◦} . \\ Therefore {U_1}^{◦} + U_2{ }^{◦} = (U_1 \cap U_2 {)}^{◦} , as required. \end{gathered}$$

 

To finish this study of dual spaces we examine the second dual, that is, the dual of
the dual. It turns out that if V is a finite-dimensional vector space then the second dual
V ′′ can be naturally identified with V itself.

THEOREM

$$\begin{gathered} Let V be a vector space over a field F . Define \Phi : V \to V ‘‘ by (\Phi v)(f ) := f \left(v\right) for all v \in V and all f \in V ‘ . \\ Then \Phi is linear and one–one \left[injective\right]. If V is finite–dimensional then \Phi is an isomorphism. \\ Proof. We check linearity as follows. \\ For v_1 , v_2 \in V and \alpha { }_1 , \alpha { }_2 \in F , and for any f \in V‘ , \\ \Phi ({\alpha }_1 v_1 + {\alpha }_2{v}_2)\left(f\right) = f ({\alpha }_1{v}_1 + {\alpha }_2{v}_2 ) \\ = {\alpha }_{1}f\left(v_1\right) + {\alpha }_{2}f\left(v_2\right) \\ = {\alpha }_1\left(\Phi v_1\right)\left(f\right) + {\alpha }_2(\Phi v_2 )\left(f\right) \\ = \left({\alpha }_1\right(\Phi v_1) + {\alpha }_2 (\Phi v_2 \left)\right)(f ), \\ and so \Phi ({\alpha }_1 v_1 + {\alpha }_2 v_2 ){\alpha }_1 (\Phi v_1 ) + {\alpha }_2 (\Phi v_2 ). \\ Now Ker \Phi = \{v \in V | \Phi v = 0\} = \{v \in V | f (v) \\ = 0 for all f \in V \prime \} = \left\{0\right\} and therefore \Phi is injective. \\ If V is finite–dimensional then dim Im \left(\Phi \right) = dim V = dim V \prime = dim V \prime \prime \\ and so \Phi is also surjective, that is, it is an isomorphism. \end{gathered}$$

In the next article I will be writing about 3 dimensional implementation of Dual Spaces and Eigen Values & Eigenvectors.

to be continued.