Version Spaces Notes

I G(VS), S(VS), Invariants

a. G invariant
1. None of the members cover any of the negative examples.
2. None of the members are generalizations or equal to other members i.e redundant
3. Each of the members is a generalization of every member of S
4. None of the members exclude any of the positive examples

b. S invariant
1. None of the members exclude any of the positive examples
2. None of the members are specializations or equal to other members i.e redundant
3. Each of the members is a specialization of every member of G
4. None of the members cover any of the negative example

III VS Algorithm

Procedure [Success,Convergence,S,G]=VersionSpaceAlgorithm(E)
Success = False, Convergence=False
VS = [G,S],G=[U],S=[]
For each example e in E
If example is positive
[G,S] = ProcessPositiveExample(G,S,e)
Else
[G,S] = ProcessNegativeExample(G,S,e)
End If
End For
If S != [] and G != []
Success = True
If S=G
Convergence = True
End If
End If
End Procedure

Procedure [S,G]=ProcessPositiveExample(G,S,e)
%G Invariant 4
G = RemoveAllNonCoveringHypotheses(G,e)
%S Invariant 1
S = GetAllMinimalGeneralizations(S,e)
%S Invariant 2
S = RemoveRedundantGeneralHypothesis(S)
%S Invariant 3
S = RemoveAllNonSpecializingGHypotheses(S,G)
End Procedure
Procedure [S,G]=ProcessNegativeExample(G,S,e)
%S Invariant 4
S = RemoveAllCoveringHypotheses(S,e)**
%G Invariant 1
G = GetAllMinimalSpecializations(G,e)
%G Invariant 2
G = RemoveRedundantSpecificHypothesis(G)
%G Invariant 3
G = RemoveAllNonGeneralizingSHypotheses(G,S)
End Procedure

** If this is called S may become empty and cause the algorithm to terminate with failure,
especially in case of conjunctive concepts.

For conjunctive concepts,

1. S is always a singleton set all differing attributes must be generalized at one go
2. G is usually not a singleton - while constructing G the call GetAllMinimalGeneralizations(G,e)
keeps all hypotheses that do not include negative example e, and return all SINGLE VARIABLE
specializations of those hypotheses that do include the negative example e
To check count number of possible generalizations combinatorially

IV Return value VS {S,G} Interpretation and usage

Convergence: If Success and S=G
Success: All examples are exhausted and S and G are not empty
Failure: Otherwise
If ALL members of S and G agree then a +/- decision can be made
Otherwise it is undecidable
Extra Exercises Solution

Initialization
G = {[?,?,?,?]}
S = {[empty]}

Example 1. [large,friendly,a_lot,spotted_brown_white] +
S = [[empty],[large,friendly,a_lot,spotted_brown_white]

All invariants satisfied

S = [large,friendly,a_lot,spotted_brown_white]
G = {[?,?,?,?]}

Example 2. [small,aggressive,little,black] -
G = { [?,?,?,?],[large,?,?,?],[?,friendly,?,?],[?,?,a_lot,?],[?,?,?,brown]*, [?,?,?,multiple_colors]
G = { [large,?,?,?],[?,friendly,?,?],[?,?,a_lot,?], [?,?,?,multiple_colors]
GI3 is not satisfied unless [?,?,?,brown] is removed, as it does not generalize S
GI2 is satisfied, No redundant hypotheses, None of the members are generalizations of other
members
SI4 is satisfied None of the members of S cover the negative example
S = [large,friendly,a_lot,spotted_brown_white]

Example 3. [large aggressive a_lot spotted_brown_black] +

S = { [large,friendly,a_lot,spotted_brown_white] ,[large,?,a_lot,spotted] }
G={ [large,?,?,?],[?,friendly,?,?],[?,?,a_lot,?], [?,?,?,multiple_colors]
GI4 is not satisfied unless [?,friendly,?,?] is removed, as it does not include all positive examples
G={ [large,?,?,?],[?,?,a_lot,?], [?,?,?,multiple_colors] }
Example 4. [medium,friendly,a_lot,mixed_red_black] -

G={ [large,?,?,?],
[?,?,a_lot,?], [small,?,a_lot,?],[large,?,a_lot,?], [?,aggressive,a_lot,?],[?,?,a_lot,spotted],
[?,?,a_lot,single_color]
[?,?,?,multiple_colors],[?,?,?,spotted],[large,?,?,mixed_red_black],
[small,?,?,mixed_red_black],[?,aggressive,?,mixed_red_black] ,[?,?,little,mixed_red_black]
}

GI3 is not satisfied unless [small,?,a_lot,?],[?,aggressive,a_lot,?],[?,?,a_lot,single_color] ,

[small,?,?,mixed_red_black],[?,aggressive,?,mixed_red_black] ,[?,?,little,mixed_red_black] are
removed, as it does not generalize S
GI2 is not satisfied unless [large,?,a_lot,?],[?,?,a_lot,spotted],[large,?,?,mixed_red_black] are
removed as these are less general than other members of G
SI4 is satisfied None of the members of S cover the negative example
G={ [large,?,?,?],[?,?,?,spotted]}
S = [large,?,a_lot,spotted]

Example 5. [small,friendly,a_lot,spotted_brown_black] +

S = {[large,?,a_lot,spotted] [?,?,a_lot,spotted] }
G={ [large,?,?,?],[?,?,?,spotted]}
GI4 is not satisfied unless [large,?,?,?],is removed, as it does not include all positive examples

G={[?,?,?,spotted]}
S = {[?,?,a_lot,spotted] }

Termination VS

G={[?,?,?,spotted]}
S = {[?,?,a_lot,spotted] }

Using the result

Example [?,?,?,spotted] [?,?,a_lot,spotted] Decision
[medium,friendly,little, Yes No Maybe
spotted_brown_white]
[large,aggressive,a_lot, Yes Yes Yes
spotted_brown_black]
[small,aggressive,a_lot, No No No
mixed_red_black]