Foundations of Data Science

Exercise sheet 2

Sebastian Nickels Raffael Gretenkort Nils Frahm

354793 354745 354620

Exercise 1
Code:
import numpy a s np

S = [
[ np . a r r a y ( [ 4 , 2 , 1 ] ) , 1] ,
[ np . a r r a y ([ 1 , 2 , 4]) , 1 ] ,
[ np . a r r a y ( [ 8 , 2, 3]) , 1 ] ,
[ np . a r r a y ( [ 1 , 1, 1 ] ) , 1] ,
[ np . a r r a y ( [ 2 , 2, 5 ] ) , 1] ,
[ np . a r r a y ([ 6 , 2 , 7 ] ) , 1]
]

def n o r m a l i z e ( S ) :
for s in S :
s [ 0 ] = np . append ( s [ 0 ] , 1 )

maximum = max( [ np . l i n a l g . norm ( s [ 0 ] ) f o r s in S ] )

return [ [ s [ 0 ] / maximum , s [ 1 ] ] f o r s in S ]

def p e r c e p t r o n ( S ) :
w = np . a r r a y ( [ 0 , 0 , 0 , 0 ] )

l o o p = True

while l o o p :
loop = False

for s in S :
i f np . s i g n ( np . dot (w, s [ 0 ] ) ) != s [ 1 ] :
w = w + s [1] s [0]
print ( w = , w)

l o o p = True

return w

S = normalize (S)
w = perceptron (S)

1
Weights:

w0 = (0, 0, 0, 0)
w1 = (0.42163702, 0.21081851, 0.10540926, 0.10540926)
w2 = (0.42163702, 0.42163702, 0.42163702, 0)
w = (0.31622777, 0.31622777, 0.52704628, 0.10540926)

Exercise 2
Let S 0 = [([1, 2, 4], 1), ([4, 2, 1], 1), ([8, 2, 3], 1), ([1, 1, 1], 1), ([2, 2, 5], 1), ([6, 2, 7], 1)].
S 0 contains the same examples as S, only the first two examples changed positions. The perceptron
algorithm needs 3 update steps on S while it only needs 1 update step on S 0 . The resulting weight
vector is w = (0.10540926, 0.21081851, 0.42163702, 0.10540926).

Exercise 3
a)
Let bi be the value of the blue node at position i {1, 2, 3} from left to right, let rj be the value of
the blue nodes at position j {1, 2} from left to right.

b1 = sgn(x1 x2 )

b2 = sgn(x2 x1 )

b3 = sgn(2 x2 2 x1 )
= sgn(x2 x1 )

r1 = sgn(3 2 b1 + 3 2 b2 )
= sgn(b1 + b2 )
= sgn(sgn(x1 x2 ) + sgn(x2 x1 ))
= sgn(sgn(x1 x2 ) sgn(x1 x2 ))
= sgn(0)
=0

13 11
r2 = sgn( b1 + b3 )
17 17
= sgn(13 b1 + 11 b3 )
= sgn(13 sgn(x1 x2 ) + 11 sgn(x2 x1 ))
= sgn(13 sgn(x1 x2 ) 11 sgn(x1 x2 ))
= sgn(2 sgn(x1 x2 ))
= sgn(x1 x2 )

y = sgn(3.98 r1 + 7 r2 )
= sgn(7 sgn(x1 x2 ))
= sgn(x1 x2 )

2
b)

We change the weight of the edge from x2 to b2 to 0 and the weight of the edge from b2 to r1 to 3 2

b1 = sgn(x1 x2 )

b2 = sgn(x1 )

b3 = sgn(2 x2 2 x1 )
= sgn(x2 x1 )

r1 = sgn(3 2 b1 3 2 b2 )
= sgn(b1 b2 )
= sgn(sgn(x1 x2 ) sgn(x1 ))
= sgn(sgn(x1 x2 ) + sgn(x1 ))

13 11
r2 = sgn( b1 + b3 )
17 17
= sgn(13 b1 + 11 b3 )
= sgn(13 sgn(x1 x2 ) + 11 sgn(x2 x1 ))
= sgn(13 sgn(x1 x2 ) 11 sgn(x1 x2 ))
= sgn(2 sgn(x1 x2 ))
= sgn(x1 x2 )

y = sgn(3.98 r1 + 7 r2 )
= sgn(3.98 sgn(sgn(x1 x2 ) + sgn(x1 )) + 7 sgn(x1 x2 ))

If x1 6= x2 then the function will evaluate to sgn(x1 x2 ) due to the weights the expression 3.98 r1
will be ignored because r1 {1, 1}. If x1 = x2 then sgn(x1 x2 ) = 0, so y = sgn(3.98 sgn(0 +
sgn(x1 )) + 7 0) = sgn(3.98 sgn(x1 )) = sgn(x1 ), which shows us that the modified ANN computes the
function f .

3
Exercise 4
a)
Code:
import numpy a s np
import random

S = [
( A , np . a r r a y ( [ 2 , 12])) ,
( B , np . a r r a y ( [ 3 , 11])) ,
( C , np . a r r a y ( [ 3 , 8])) ,
( D , np . a r r a y ( [ 5 , 4])) ,
( E , np . a r r a y ( [ 7 , 5])) ,
( F , np . a r r a y ( [ 7 , 3])) ,
( G , np . a r r a y ( [ 1 0 , 8])) ,
( H , np . a r r a y ( [ 1 3 , 8]))
]

k = 3

z = [ s [ 1 ] f o r s in S [ : k ] ]

oldC = [ [ ] f o r i in range ( 0 , k ) ]

while True :
C = [ [ ] f o r i in range ( 0 , k ) ]

print ( z = , z )
print ( C = , oldC )

for s in S :
minimum = np . l i n a l g . norm ( s [ 1 ] z [ 0 ] )
newJ = 0

for j in range ( 1 , k ) :
v a l u e = np . l i n a l g . norm ( s [ 1 ] z [ j ] )

i f v a l u e < minimum :
minimum = v a l u e
newJ = j

C[ newJ ] . append ( s )

for j in range ( 0 , k ) :
i f len (C [ j ] ) > 0 :
z [ j ] = sum ( [ c [ 1 ] f o r c in C [ j ] ] ) / len (C [ j ] )
else :
z[j] = 0

i f oldC == C :
break

oldC = C

4
Notation: zj = (z 1 , z 2 , z 3 ) and Cj = (C 1 , C 2 , C 3 ) in Iteration j N.

Iterations:

z0 = ((2, 12), (3, 11), (3, 8))

C0 = ({}, {}, {})

z1 = ((2, 12), (3, 11), (7.5, 6))

C1 = ({A}, {B}, {C, D, E, F, G, H})

z2 = ((2, 12), (3, 9.5), (8.4, 5.6))

C2 = ({A}, {B, C}, {D, E, F, G, H})

z3 = ((2.5, 11.5), (3, 8), (8.4, 5.6))

C3 = ({A, B}, {C}, {D, E, F, G, H})

b)
Not done.

c)
When z0 = (C, D, E), then the algorithm does two iterations and returns the clusters C = ({A, B, C},
{D, E, F }, {G, H}). This is due to already existing clusters around the means z0 .

d)
Not done.