Acai05 Berthold FL

ALTANA Chair for Bioinformatics and Information Mining
Overview
Fuzzy Logic
Motivation Degrees of Membership and Fuzzy Sets Linguistic Values and Variables Operators on Fuzzy Sets Fuzzy Implication
Tutorial: Fuzzy Logic

Advanced Course on Knowledge Discovery, Ljubljana, June 2005
Fuzzy Rules
Inference Construction of Fuzzy Rules from Data
Michael R. Berthold Konstanz University Michael.Berthold@uni-konstanz.de
Fuzzy Arithmetic
Fuzzy Numbers Operations on Fuzzy Numbers Extension Principle
Conclusion
22/06/2005 M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.
#2
Fuzzy Logic: Motivation

Modeling of imprecise concepts:
Age, Weight, Height,
Characteristic Functions: Crisp Sets

Classical Sets can be described by a characteristic function:
Modeling of imprecise dependencies (e.g. rules):

If Temperature is low and Oil is cheap then crank up the heating system
1 x A m A ( x ) := 0 x A
Example: A = {x | a x b}
m A ( x ) {0,1}
Origin of Information:
Modeling of Expert Knowledge Representation of information extracted from inherently imprecise data
m A ( x) 1
a
x
#4
#3
22/06/2005
M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.
Characteristic Functions: Fuzzy Sets

Fuzzy Sets are described by a membership function:
~ A ( x ) [0,1]
Linguistic Variables and Values

Associating meaning (semantic) with fuzzy sets results in
Linguistic Variables: the (labeled!) domain of the fuzzy sets Linguistic Values: a (labeled!) collection of fuzzy sets on this domain
Example:
Examples:
Age: young, old
~ A = x is roughly in [a, b]
~ A ( x)
young/old ( x)
1
Size: small, medium, tall
young
30
old
50
x [years]
sml/med/tall ( x)
1
a
small
150
medium
tall
190 x [cm]
#6
x
#5
22/06/2005
22/06/2005
Linguistic Values & Context

what about basketball players?
Types of Membership Functions

Trapezoid: <a,b,c,d> Gaussian: N(m,s) (x) 1 s 0 a b c d x 0 m x
sml/med/tall ( x) 1
small
180
medium
tall
220 x [cm]
(x) 1
and jockeys?
sml/med/tall ( x) 1
Triangular: <a,b,b,d>
Singleton: (a,1) and (b,0.5) (x) 1
small medium
140 160
tall
x [cm]
(x) 1
linguistic values are inherently context dependent!

0
#7
22/06/2005
x
#8
Fuzzy Membership Function: Basic Concepts

Support: elements having non-zero degree of membership Core: set with elements having degree of 1 -Cut: set of elements with degree >= Height: maximum degree of membership
Operators on Fuzzy Sets

Set of old and tall people (Conjunction)
old ( x) = 0.7 tall ( x) = 0.5 old tall ( x) = ?
old tall ( x) = 0.7 old tall ( x) = 0.5
tall
1
old tall ( x) = 1.0
old tall ( x) = 0.2

A 1 0
22/06/2005
core -cut height support

old tall ( x) = 0
0 x
#9
22/06/2005
0
old
#10
Operators on Fuzzy Sets

Set of old or tall people (Disjunction)
Min / Max-Norm
Classical Fuzzy Operators: Min/Max-Norm
Conjunction: Disjunction:
old ( x) = 0.7 tall ( x) = 0.5 old tall ( x) = ?

tall
old tall ( x) = 0.5 old tall ( x) = 0.2 old tall ( x) = 0
0 1
A B ( x) := min{ A ( x), B ( x)} A B ( x) := max{ A ( x), B ( x)}
old tall ( x) = 0.7
old tall ( x) = 1.0
Negation:
A ( x) := 1 A ( x)
A (x) B (x)
1 0
0
old
#11
#12
Product / Bounded-Sum
Classical Fuzzy Operators: Product / Bounded-Sum
Conjunction: Disjunction: Negation:
Lukasiewicz Norm
A B ( x) := A ( x) B ( x) A B ( x) := A ( x) + B ( x) A ( x) B ( x)
A ( x) := 1 A ( x)
A (x) B (x)
1 0
22/06/2005
#13
22/06/2005
#14
T-norms and S-norms
Fuzzy T- and S-Norms

T-Norm S-Norm
S (a, b ) = max{a, b}
S(u,v) = 1 T(1-u,1-v) T(u,v) = 1 S(1-u,1-v) min/max:
using De Morgans Law (AB = (AB))
T (a, b ) = min{a, b}
Min-Max
max(u,v) = 1 min(1-u,1-v) = 1-1-min(-u,-v) = max(u,v)

T (a, b ) = a b S (a, b ) = a + b a b
Product-Sum
22/06/2005
#15
22/06/2005
#16
Fuzzy T- and S-Norms

T-Norm
T (a, b ) = max{0, a + b 1}
Operator - Spectrum
S-Norm
S (a, b ) = min{ , a + b} 1
T-norms
Lukasiewicz Norm drastic product

a , if b = 1 T (a, b ) = b , if a = 1 0 , else a , if b = 0 S (a, b ) = b , if a = 0 1 , else
S-norms (T-conorms)
bounded sum drastic sum
bounded product
min
max
Drastic Product/Sum
drastic product : T(u,v)=
{ {
u v 0 u v 1
if v=1 if u=1 otherwise if v=0 if u=0 otherwise

#18
drastic sum :
S(u,v)=
22/06/2005
#17
22/06/2005
Fuzzy Norms: Issues

Interesting effects:
A A = ?
1
Fuzzy Implication
One possibility:
derive Implication via tautology AB = A(AB) and min/max norm.
A (x)
A (x) AA (x)
x
A B ( x) := max{ A ( x), min{ A ( x), B ( x)}} 1

*v* A AB
A A = ?
1
A (x) A (x)
A A (x)
22/06/2005
#19
22/06/2005
#20
Fuzzy Implication
Or vice versa:
start with Lukasiewicz-Implication:
Imprecise Reasoning
Classic Modus Ponens
A B ( x) := min{ ,1 A ( x) + B ( x)} 1
and derive disjunction and conjunction using AB = AB
A A B B
A A B ?
A B ( x) := min{ , A ( x) + B ( x)} 1
AB = (AB)
Imprecise: Generalized Modus Ponens
A' ( x )
A B
= max{0, A ( x) + B ( x) 1}
22/06/2005
A B ( x) := 1 min{ ,1 A ( x) + 1 B ( x)} 1
B' ( y)
#21
22/06/2005
#22
Joint Constraint (support distribution)

Using the Min/Max Norm: Implication results in a constraint on (x,y) Cartesian Product AB: A B ( x, y ) = min{ A ( x ), B ( y )}
Conditional Constraint (possibility distribution)

Alternatively (using Lukasiewicz Norm): Implication results in relation expressing possibilities:
B ' ( y ) = sup{min{ A' ( x), AB ( x, y )}}

x
Poss( x, y ) = min{ ,1 A ( x) + B ( y )} 1
x
B ' ( y ) = sup{min{ A' ( x),1 A ( x) + B ( x)}}

y
B' ( y)
A B
B '' ( y )
A B
B' ( y)
A B
A' ( x)
x
A'' ( x)
x
#23
22/06/2005
A' ( x)
x
22/06/2005
#24
Joint vs. Conditional Constraint

Joint Constraints:
express positive knowledge (facts are supported)
Fuzzy Rules
Rule: IF <Antecedent> THEN <Consequent> Fuzzy Version 1: Mamdani Rules
Antecedent: Conjunction of fuzzy memberships Consequent: Fuzzy Set
Conditional Constraints:
express negative knowledge (facts are excluded)
22/06/2005
#25
22/06/2005
#26
Example: Mamdani Rule

IF age IS young AND car-power IS high THEN risk IS high
low medium high risk age old normal
Fuzzy Rules
Rule: IF <Antecedent> THEN <Consequent> Fuzzy Version 1: Mamdani Rules
Antecedent: Conjunction of fuzzy memberships Consequent: Fuzzy Set
Fuzzy Version 2: Takagi-Sugeno -(Kang)- Rules

Antecedent: Conjunction of fuzzy memberships Consequent: (usually) real-valued functions of degree 0-2.
young low medium high car-power

#27
22/06/2005
#28
Example: Takagi-Sugeno Rule

IF age IS young AND car-power IS high THEN risk-factor = w0+w1*age+w2*car-power
old age normal w1*age
Fuzzy Rule System (Mamdani)

R1: IF age IS young AND car-power IS high THEN risk IS high R2: IF age IS normal AND car-power IS medium THEN risk IS medium
young high high
R1:
age car-power risk
young risk-factor w0 low medium high car-power

w2*car-power
normal
medium
medium
R2:
age car-power risk
#29
22/06/2005
#30
Fuzzy Rule Systems: Example 1

Step 1: Fuzzification of crisp inputs

young normal old

Step 2a: Inference (e.g. via Min/Max-Norm) R1 = 0.4
low medium high low medium high
R1:
0.8 0.4
R1:
young
normal
old
low
medium
high
low
medium
high
age
young normal old
car-power
low medium high low medium
risk
age
young normal old low
car-power
risk
R = 0.3
2
R2:
0.3
0.7
high
R2:
medium
high
low
medium
high
age
car-power
risk
age
car-power
low medium
risk
high
age
car-power
age
car-power
risk
a
22/06/2005
b
a
#31
22/06/2005
b
#32

Step 2b: Inference (e.g. via Maximum-Norm)

young normal old low medium high

Step 3: Defuzzification

young normal old low medium high
low medium high
low medium high
R1:
R1:
age
car-power
medium high low medium
risk
age
car-power
medium high low medium
risk
R2:
high
R2:
high
age
car-power
low medium
risk
age
car-power
low medium
risk
high
high
age
car-power
risk
age
car-power
risk
a
22/06/2005
b
a
#33
22/06/2005
b
c=?
#34
Defuzzification: Center of Gravity

Center of Gravity:
Defuzzification: Other Methods
y=
j =1 r j =1 r
risk
( y ) dy
risk
( y ) dy
An easier approximation (with some imprecision):
sj
j
y=
22/06/2005
#35
22/06/2005
#36
Fuzzy Inference (Mamdani)

Fuzzy Inference Fuzzification Inference Fuzzy Rules cold warm hot 0.7 0.2 Measurement t
if temp is cold then valve is open
Fuzzy Rule System (Mamdani)

R1: if x is small then y is medium R2: if x is medium then y is large R3: if x is large then y is zero
Defuzzification
large
cold =0.7
if temp is warm then valve is half
open half close 0.7 0.2 v Output
medium
small
warm =0.2
if temp is hot then valve is close
zero x
hot =0.0
small
#37
22/06/2005
medium
large
x
#38
Fuzzy Rule System (Takagi Sugeno)

R1: IF x IS small THEN y = x R2: IF x IS medium THEN y = 5 R3: IF x IS large THEN y = 2*x-5
Construction of Fuzzy Rule Systems

Wang&Mendel Algorithm
y large R1: if x is zero then y is medium R2: if x is small then y is medium R3: if x is medium then y is large R4: if x is large then y is medium
y=
i =1
Ri
r yi ( x )
Ri
y medium 5 small small medium 5 large 10 x

#39
22/06/2005
i =1
x zero x
zero
small
medium
large
x
#40
22/06/2005
Grid Based Algorithms

Exponentially many rules in high dimensional spaces If grid is chosen fine enough: arbitrarily good approximation possible (but at what cost!) Wrong choice of grid: skipping of extrema One extension: Higgins&Goodman Algorithm (see IDA-book)
pre-define threshold for desired approximation error finds best partitioning (=grid) Disadvantages: concentrates on outliers interpretation difficult: granulation solely data driven
Free Fuzzy Rules

No global dependence on granulation:
individual (per rule) membership functions better modeling of local properties
Not all attributes used for all rules:

individual choice of constraints on few attributes per rule better interpretability in high dimensions no exponential explosion of rules with dimensionality
Other Constructive Algorithms (coming up next)

Local membership functions partially predefined membership functions/granulations possible somewhat tolerant against outliers through local caching
#41
22/06/2005
#42
Formation of Free Fuzzy Rules

Algorithm FRL 1. FORALL training examples (x,c) DO
1. IF correct rule of class c exists: COVERED: increase weight +1 adjust core region of rule to cover x 2. ELSE: COMMIT: insert new rule with core=x Support = infinite (i.e. rule is not constrained) 3. SHRINK: Reduce Support of all Rules of conflicting class that cover x.

Algorithmus FRL Steps: COVERED: easy COMMIT: easy. SHRINK: heuristic one solution: volume-based. Demo.
2. Until no more changes occured.

#43
22/06/2005
#44

Observations: FRL finds set of rules completely describing the data (as long as training data is conflict free) Each rule is a partial hypothesis for a subset of training data
core: most specific hypothesis covering subset of training data support: (one of the) most general hypotheses covering subset of data. support is_more_general_than core.
Other Types of Fuzzy Rule Learning Methods

Constructive:
Finding fuzzy rules by growing from singletons ( FRL shrinks from most general until it fits to data)
GRID:
merge grid cells (or rows/columns) if no points covered or same class predicted
Adaptive:
Initialize rules randomly (or via expert knowledge) and optimize rule parameters (location, sometime also number of membership functions) iteratively (gradient descent, heuristic hill climbing algorithms).
Core and Support regions can also be seen as:

smallest area with highest degree of confidence (we have evidence) largest area without conflict (we have not seen any counter-examples)
Neuro-Fuzzy (later):
inject fuzzy rules into neural network and use NN training algorithms.
22/06/2005
#45
22/06/2005
#46
Fuzzy Numbers: Motivation

Often it is desirable to preprocess imprecise inputs
add or multiply fuzzy numbers apply function (normalization, ) to fuzzy numbers
Fuzzy Numbers
Classic (crisp) Numbers: aR
Characteristic Function ma(x){0,1}
ma
Fuzzy Controller can be extended to process imprecise inputs via use of fuzzy numbers. Fuzzy Numbers: Model Imprecision in e.g. measurement.
Fuzzy Number (about a):

membership function a(x)[0,1]
a Membership functions of fuzzy numbers are (usually): normalized monotone (left/right) singular x
#47
22/06/2005
#48
Operations on Fuzzy Numbers

Addition of two crisp Numbers:
ma/b 1

Addition of two Singletons:
a/b 1
a+b
a+b
1 z = a + b ma +b ( z ) = else 0
ma +b ( z ) = max{ma ( x ) mb ( y ) | x + y = z}
x , yR
= and?...
a +b ( z ) =
min{ a (a ), b (b)} z = a + b 0 else

x , yR
= and
a +b ( z ) = max{min{ a (x ), b ( y )}| x + y = z}
#49
#50

Addition of two (fuzzy) intervals:
a/b 1

And finally: Addition of two Fuzzy Numbers:
a/b 1
a+b
x , yR
= and
x , yR
what about other operators, for example multiplication?
22/06/2005
#51
22/06/2005
#52

Multiplication of two Fuzzy Numbers:
a/b 1 y

Crisp Functions applied to Fuzzy Numbers:
f ( a ) ( y ) = max{ a ( x) | y = f ( x)}
xR
y = f (x )
a b x
a*b ( z ) = max{min{ a (x ), b ( y )}| x * y = z}

x , yR
f(a) what about arbitrary functions? a a
x
#53
22/06/2005
#54

Crisp (non monotone) function applied to Fuzzy Numbers:

Based on Extension Principle:
f ( a ) ( y ) = max{ a ( x) | y = f ( x)}
y
xR
f ( a ) ( y ) = max {min{ a ( x1 ),..., a ( xn )}| y = f ( x1 ,..., xn )}

x1 ,..., xn R
1 n
y = f (x )
Computable in Practice:
via series of -cuts via polynomial representation of left and right side of fuzzy numbers (closed under addition and multiplication a.o.) via granulation on variables (grid based)
f(a) a a
22/06/2005
x
#55
22/06/2005
#56
Imprecise Functions (Fuzzy Graphs)

Real-valued Function: infinite set of crisp points Fuzzy Graph: finite set of imprecise points
IF x IS small THEN y IS medium IF x IS medium THEN y IS small IF x IS large THEN y IS large
y large medium
Imprecise Functions (Fuzzy Graphs)

Each fuzzy point describes an imprecise relation:
r (x , y ) IS
A B
A fuzzy graph is a collection of fuzzy points:
r (x , y ) IS
A1 B1 OR ... OR A r Br
Reasoning using fuzzy graphs:
small small
22/06/2005
r x IS A r ~ f IS U j=1 A j B j y IS B
medium
large x
#57
#58
Fuzzy Graphs
Approximation through cylindrical extension:
Fuzzy Graphs
Learning of Fuzzy Graphs from Data: Demo
Influence of granulation of target variable Influence of outliers: large number of fuzzy points avoidance via outlier model
~ B = proj y (A I ) f
1 1 A X
22/06/2005
#59
22/06/2005
#60
10
Wrap-Up / Conclusions
Fuzzy Methods
model imprecise knowledge (fuzzy rules) draw imprecise conclusions
Main Concepts:
Fuzzy Set and Degree of Membership Fuzzy Set Operators Extension Principle: allow to extend classical functions to fuzzy numbers
Algorithms (mostly heuristic) to

find fuzzy rules in data build fuzzy approximators (fuzzy graphs/rule systems)
Fuzzy Controllers in wide spread use
22/06/2005
#61
11

Acai05 Berthold FL

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Acai05 Berthold FL

Enviado por

Direitos autorais:

Formatos disponíveis

ALTANA Chair for Bioinformatics and Information Mining

Tutorial: Fuzzy Logic

Michael R. Berthold Konstanz University Michael.Berthold@uni-konstanz.de

Fuzzy Logic: Motivation

Characteristic Functions: Crisp Sets

Modeling of imprecise dependencies (e.g. rules):

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

Characteristic Functions: Fuzzy Sets

Linguistic Variables and Values

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

Linguistic Values & Context

Types of Membership Functions

Singleton: (a,1) and (b,0.5) (x) 1

linguistic values are inherently context dependent!

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

Fuzzy Membership Function: Basic Concepts

Operators on Fuzzy Sets

old ( x) = 0.7 tall ( x) = 0.5 old tall ( x) = ?

old tall ( x) = 0.7 old tall ( x) = 0.5

old tall ( x) = 1.0

old tall ( x) = 0.2

core -cut height support

Operators on Fuzzy Sets

old ( x) = 0.7 tall ( x) = 0.5 old tall ( x) = ?

A B ( x) := min{ A ( x), B ( x)} A B ( x) := max{ A ( x), B ( x)}

old tall ( x) = 0.7

old tall ( x) = 1.0

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

T-norms and S-norms

Fuzzy T- and S-Norms

S(u,v) = 1 T(1-u,1-v) T(u,v) = 1 S(1-u,1-v) min/max:

using De Morgans Law (AB = (AB))

max(u,v) = 1 min(1-u,1-v) = 1-1-min(-u,-v) = max(u,v)

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

Fuzzy T- and S-Norms

Lukasiewicz Norm drastic product

drastic product : T(u,v)=

if v=1 if u=1 otherwise if v=0 if u=0 otherwise

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

Fuzzy Norms: Issues

A B ( x) := max{ A ( x), min{ A ( x), B ( x)}} 1

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

Imprecise: Generalized Modus Ponens

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

Joint Constraint (support distribution)

Conditional Constraint (possibility distribution)

B ' ( y ) = sup{min{ A' ( x), AB ( x, y )}}

B ' ( y ) = sup{min{ A' ( x),1 A ( x) + B ( x)}}

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

Joint vs. Conditional Constraint

M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005.

ab ( z ) = max{min{ a (x ), b ( y )}| x y = z}