CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

3 The Fundamentals:Algorithms, the Integers,

and Matrices
3.1 Algorithms
Definition 1 An algorithm is a finite set of instructions for performing a
computation or solving a problem.

Algorithm 1 Finding the maximal element in a finite sequence

procedure max(a1, a2, …, an: integers) n≥1
max := a1; max = max(a1)
for i := 2 to n do
if max < ai → max := ai fi max = max(a1, …, ai)
od max = max(a1, …, an)

3/31/10 Kwang-Moo Choe 1

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

max, i := a1, 2; max = max(a1) (n≥1)

do i ≤ n →
if max < ai → max = ai
| max ≥ ai → skip
fi;
i := i+1 max = max(a1, …, ai-1)
od
max = max(a1, …, ai-1) ∧ (i=n+1) = max(a1, …, an)

3/31/10 Kwang-Moo Choe 2

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Algorithm 2 The linear search algorithm

i := 1; do (i ≤ n ∧ x ≠ ai) → i := i+1 od
[(i≤ n) ⇒ ((1≤∀k<i: x ≠ ak) ∧ (x=ai))] ∨⋅ [(i=n+1) ⇒ (1≤∀k≤n: x ≠ ak)]

if i ≤ n → loc := i | i=n+1 → loc = 0 fi

[(loc≤n)⇒((1≤∀k<loc: x≠ak) ∧ x=aloc)] ∨⋅ [(loc=0)⇒(1≤∀k≤n: x≠ak)]

Another implementation for the linear search

i, loc := 1, 0; (n ≥ 0)
do ((i ≤ n) ∧ (loc = 0)) → if x = ai → loc := i;
| x ≠ ai → i := i+1
od fi
[(loc≤n)⇒((1≤∀k<loc: x≠ak) ∧ x=aloc)] ∨⋅ [(loc=0)⇒(1≤∀k≤n: x≠ak)]

3/31/10 Kwang-Moo Choe 3

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Algorithm 3 The binary search algorithm

procedure binary search(x: integer; a1, a2, …, an: increasing integers)
i, j, loc := 1, n, 0;
do ((i ≤ j) ∧ (loc = 0)) → m := ⎣(i+j)/2⎦ ;
if x > am → i := m+1
| x = am → loc := m
| x < am → j := m−1
fi
od
(1≤∀k<n: ak≤ak+1) ∧ (n ≥ 0) is the given condition(constant)
[(loc≤n) ⇒ (x = aloc)] ∨⋅ [(loc=0) ⇒ (1≤∀k≤n: x ≠ ak)]

3/31/10 Kwang-Moo Choe 4

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

i, j := 1, n; loc := 0;
do (i ≤ j) → m := ⎣(i+j)/2⎦ ;
if x > am → i := m+1
| x = am → loc := m; i, j := m, m-1;
| x < am → j := m−1
fi
od
[(loc≤n) ⇒ (x = aloc)] ∨⋅ [(loc=0) ⇒ (1≤∀k≤n: x ≠ ak)]

3/31/10 Kwang-Moo Choe 5

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

procedure bubble sort(a1, a2, …, an: distinct integers with n ≥ 2)

for i := 1 to n-1 do
for j := 1 to n-i do →
if aj ≥ aj+1 → aj, aj+1 := aj+1, aj
| aj ≤ aj+1 → skip
fi
od
od
i=1 an = max(a1, …, an) (n-1) iterations
i=2 an-1 = max(a1, …, an-1) (n-2) iterations
…
i an-i+1 = max(a1, …, an-i+1) (n-i) iterations
…
i = n-1 a2 = max(a1, a2) (1) iterations
1≤∀k<i<n: ak ≤ ak+1.

3/31/10 Kwang-Moo Choe 6

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

procedure insertion sort(a1, a2, …, an: distinct integers with n ≥ 2)

for j := 2 to n do
i := 1; do (aj > ai) → i := i+1 od
m := aj;
for k:= 0 to j - i -1 do aj-k := aj-k-1 od;
ai := m
od

3/31/10 Kwang-Moo Choe 7

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Dijkstra’s mini language

S ::= x1, …, xn := E1, …, En | S; S | skip | abort
| if B1 → SL1 ‘|’ … ‘|’ Bn → SLn fi
| do B1 → SL1 ‘|’ … ‘|’ Bn → SLn od

if x ≥ y → m := x if x > y → m := x if x ≥ y → m := x
| x < y → m := y | x ≤ y → m := y | x ≤ y → m := y
fi fi fi
(m=x ∨ m=y) ∧ m≥x ∧ m<y
(m=x ∨ m=y) ∧ m>x ∧ m≥y
(m=x ∨ m=y) ∧ m≥x ∧ m≥y

3/31/10 Kwang-Moo Choe 8

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

do B1 → SL1 ‘|’ … ‘|’ Bn → SLn od

initialize P Let’s assume that loop invariance is P.

P
P P ∧ ¬BB and BB = B1 ∨ B2 ∨ … ∨ Bn.
BB?
∴ ¬BB = ¬(B1 ∨ B2 ∨ … ∨ Bn)
P ∧ BB
= ¬B1 ∧ ¬B2 ∧ … ¬Bn.
loop body

?P
recover P
P
Loop invariance P
Loop invariance P should be initialized before the loop,
Loop invariance P must be recovered in the loop, if destroied, and
Loop invariance P is still valid after(termination of) the loop.
P ∧ ¬BB is the final condition that you want after the loop!

3/31/10 Kwang-Moo Choe 9

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

a, b, c, d := A, B, C, D;
do a > b → a, b := b, a
| b > c → b, c := c, b
| c > d → c, d := d, c
od

¬BB = (a ≤ b) ∧ (b ≤ c) ∧ (c ≤ d)
= a ≤ b ≤ c ≤ d.
What is the loop invariance of the above do-od loop?
P = a, b, c, and d are permutation of A, B, C, and D.
Loop invariance P should be strong enough to get final result
with and ¬BB;
whereas weak enough
to be easily initialized and be still valid inside the loop.
P ∧ ¬BB is the final condition that you want.

3/31/10 Kwang-Moo Choe 10

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Greedy algorithm
Optimization problem
to find a solution to the given problem that
either minimizes or maximizes the value of some parameters
(global) optimal
to select the best choise considering all sequences of steps
local optimal(greddy algorithm)
to select the best choise at each step

Algorithm 6 Greedy Change-Making Algorithm

procedure change(n1, n2, …, nr: number of coins are r and
c1 > c2 > … > cr is the value for each coini;
n: positive integer is the amount that you want to make change)
n1, n2, …, nr := 0, 0, …, 0; i := 1;
do (i ≤ r) → do (n ≥ ci) → ni := ni + 1; n := n - ci od; i:= i+1 od

3/31/10 Kwang-Moo Choe 11

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

The Halting Problem

Unsolvable problems
Halting problem
where program will be halt or not(infinite loop)
Alan Turing, 1936
No algorithm that computes it!
Three cases
i) (totally) solvable(computable; recursive)
always terminate
algorithms
ii) partially solvable(partially computable;recursively enumerable)
algorithm(program) exits
but may halts or loops forever
iii) unsolvable(uncomputable; Not recursively enumerable)
No algorithm(program) exists

3/31/10 Kwang-Moo Choe 12

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

proof Assume a procedure H(P: program, I: input data) exists where

If P runs with I and halts → print “halt” as output.
| P with I loops forever → print “loops forever” as output.
Program also can be a input data(compiler)
Condider procedure K(P) as follows
procedure K(P: program)
if H(P, P) prints “halts” → loops forevar
| H(P, P) prints “loops forever” → halts
fi
Now consider K(K)
if H(K, K) print “loops forever” → K(K) halts
| H(K, K) print “halts” → K(K) loops forever
Contradiction
No algorithm for halting program!

3/31/10 Kwang-Moo Choe 13

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Three cases for problem(programs) (Turing-Church’s thesis)

i) Always terminate algorithm
totally solvable
ii) Terminate or loop foever program
partially solvable
programmble
iii) Program does not exits non-programable
halting program
Russel’s Paradox
i) ⊂ ii) ii) ∩ iii) = ∅.

3/31/10 Kwang-Moo Choe 14

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

3.2 The Growth of Functions

Big-O Notation
Definition 1 Let f: N → R or R → R.
We say f(x) is O(g(x)), if ∃c, k ∈ R .∋. |f(x)| ≤ c|g(x)| ∀x > k.
We also write f(x) = O(g(x)) or even f(x) ∈ O(g(x)).

The constant c and k are called the witnesses of ... f(x) is O(g(x)).
Assume c and k are smallest witnesses, Then ∀c’, k’ .∋. c < c’, x < x’,
c’ and k’ are also witnesses

g(x) is faster or equl(not slower) glowing than f(x)

when x becomes large enough
and ignoring multiplying some constant c.

Example 1, 2, 3, 4

3/31/10 Kwang-Moo Choe 15

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Theorem 1 Let f(x) = anxn + an-1xn-1 + … + a1x + a0. Then f(x) is O(xn).
proof easy with witness k=1 and c = Σ |ai|.

Example 6 n!
n! ≤ nn. n! is O(nn) with witness k=1 and c=1.
log n! ≤ log nn = n log n

1, log n, n, nlog n, n2, …, 2n, n!

Definition Let g: N or R → R.
O(g) = {f: N or R → R| ∃c, k > 0 ∀x > k: |f(x)| ≤ c|g(x)|}
If f ∈ O(g), f glows at most order of g.
O(1) ⊂ O(log n) ⊂ O(n) ⊂ O(nlog n) ⊂ O(n2) ⊂ O(n3)⊂ … O(2n) ⊂ O(n!)

3/31/10 Kwang-Moo Choe 16

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Theorem 2, 3 Let f1 ∈ O(g1), f2(x) ∈ O(g2). Then

f1+f2 ∈ O(g1+g2) = O(max(|g1|, |g2|) and
f1f2 ∈ O(g1g2).
Colollary 1 Let f1, f2 ∈ O(g). Then f1+f2 ∈ O(g).

Big-Omega and Big-Theta Notation

Definition 2 Ω(g) = {f: N or R → R| ∃c, k > 0 ∀x > k: |f(x)| ≥ c|g(x)|}
If f ∈ Ω(g), f glows at least order of g.
Definition 3 Θ(g) = {f:N→R| ∃c1,c2,k >0, ∀x>k: c1|g(x)|≤|f(x)|≤ c2|g(x)|}
If f ∈ Ω(g), f glows (exactly) same order of g.
f ∈ Θ(g), iff f ∈ O(g) ∧ f ∈ Ω(g).

Theorem 4 Let f(x) = anxn + an-1xn-1 + … + a1x + a0 with an ≠ 0. Then

f(x) ∈ Θ(xn).

3/31/10 Kwang-Moo Choe 17

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

3.3 Complexity of Algorithms

Computational complexity
time complexity
space complexity

Worst-case analysis
Average-case analysis

Θ(1) constant complexity

Θ(log n) logarithmic complexity
Θ(n) linear complexity
Θ(nk) polinomial complexity
Θ(kn) exponential complexity
Θ(n!) factorial complexity

3/31/10 Kwang-Moo Choe 18

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

(totally) solvable
tractable
polynomial
intractable
exponential
class P
polynomial, tractable

class NP(P ⊆ NP)

Nondeterministc polynomial exponetial(upper bound)
lower bound? tractable or intractable
NP-complete problems
subset of class NP
If any of these problems can be solved in polynomial time,
all the problems in class in NP also in class P(P = NP).

3/31/10 Kwang-Moo Choe 19

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

3.4 Integer and Division

Definition 1 Let a, b ∈ Z ∧ a ≠ 0. We say a divies b, if ∃c ∈ Z .∋. b = ac.
We say a is a factor( 약수 ) of b, and b is a multiple( 배수 ) of a.
a | b denotes a divides b, and a /| b to denote a does not divide b.

Theorem 1 ∀a, b, c ∈ Z.
i) (a | b) ∧ (a | c) ⇒ a | (b+c).
ii) (a | b) ⇒ a | bc.
iii) (a | b) ∧ (b | c) ⇒ a | c.

Corollary 1 ∀a, b, c ∈ Z.(Extension of Thm 1. i) and ii))

(a | b) ∧ (a | c) ⇒ a | (mb+nc) ∀m, n ∈ Z.

3/31/10 Kwang-Moo Choe 20

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Theorem 2 The Division Algorithm

∀a ∈ Z, ∀d ∈ Z+ ∃1q, ∃1r ∈ Z: 0 ≤ r < d .∋. a = dq + r.
Definition 2 d is called the divisor, a is called dividened,
q is called the quotient, and r is called the remainder.
q = a div d, r = a mod d.

Modular Arithmetic
Definition 3 Let a, b ∈ Z, m ∈ Z+. Then a is congruent to b modulo m,
written a ≡ b (mod m) or b ∈ [a]mod m, if m | (a - b).
Theorem 3 a ≡ b (mod m), iff a mod m = b mod m.
or (a - b) mod m = 0.
Theorem 4 a ≡ b (mod m), iff ∃k ∈ Z: a = b + km.
Theorem 5 If a ≡ b (mod m) ∧ c ≡ d (mod m), then
a + c ≡ b + d (mod m) ac ≡ bd (mod m).

3/31/10 Kwang-Moo Choe 21

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

3.5 Primes and Greatest Common Divisors

Definition 1 p ∈ Z+ is prime, ⇔ p > 1 ∧ ¬∃a ∈ Z+: (1 < a < p ∧ a | p).
Otherwise composite.

Theorem 1 The Fundamental Theorem of Arithmetic

Every positive integer has a unique representation
as the product of nondecreasing series of zero or more primes.

Theorem 2 If n is composite integer, then n has a prime divisor less than

or equal to SQRT(n).

Theorem 3 There are infinitely many primes.

proof proof by contradiction
Assume there are only finitelt many primes, p1, p2, … pn.
Let Q = p1p2…pn + 1.

3/31/10 Kwang-Moo Choe 22

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Q is prime or product of two or more primes.

If pj | Q, pj | Q - p1p2…pn = pj | 1.
∴ ¬∃j: 1 < j < n, pj | Q.
∴ There exist other prime not in the list p1, p2, … pn or Q is a prime.
∴ There are infinitely many primes.

Conjectures and Open Problems about Primes

Goldbach’s conjecture
Every even integer n, n > 2, is the sum of two prime numbers.
2 ⋅ 1017.
The Twin Prime conjecture
primes that differs two
16,8699,8733,9975 ⋅ 217,1960 ± 1 are primes with 5,1779 digits.

3/31/10 Kwang-Moo Choe 23

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Graetest common divisors and Least common multiples

Definition 2 Let a, b ∈ Z and not both zero.
d = gcd(a, b) = max(d: d | a ∧ d | b) ⇔,
d | a ∧ d | b ∧ ∀e ∈ Z, (e | a ∧ e | b) → d ≥ e.

Definition 3 The integers a and b are relatively prime(coprime)

if gcd(a, b) = 1.

Definition 4 The integers a1, a2, … an are pairwise relatively prime,

if gcd(ai, aj) = 1 whenever 1 ≤ i < j ≤n.

Definition 5 Let a, b ∈ Z and not both zero.

d = lcm(a, b) = min(m: a | m ∧ b | m) ⇔,
a | m ∧ b | m ∧ ∀n ∈ Z, (n | a ∧ n | b) → m ≤ n.

3/31/10 Kwang-Moo Choe 24

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Let a = p1a1p2a2 … pnan and b = p1b1p2b2 … pnbn.

gcd(a, b) = p1min(a1, b1)p2min(a2, b2)…pnmin(an, bn).
lcm(a, b) = p1max(a1, b1)p2max(a2, b2)…pnmax(an, bn).

Theorem 5 Let a and b be positive integer.

ab = gcd(a, b) lcm(a, b)

3/31/10 Kwang-Moo Choe 25

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

3.6 Integer and Algorithms

Theorem 1 Let b ∈ Z+ , b > 1. Then ∀n ∈ Z+,

n = akbk + ak-1bk-1 + … + a1b + a0.
k ≥ 0, 0 ≤ a0, a1, …, ak < b, ak ≠ 0.
unique repesentation
base expansion of n
written n = (akak-1 … a1a0)b.

Lemma 1 Let a = bq + r, a, b, q, r ∈ Z. Then gcd(a, b) = gcd(b, r)

proof ∀d, d | a ∧ d | b, then d | a - bq = r.(Corollary 1)
∀d, d | b ∧ d | r, then d | bq + r = a.

3/31/10 Kwang-Moo Choe 26

CS204 Discrete Mathematics 3 The Fundamentals: Algorithms, the Integers, and Matrices

Algorithm 6 Euclid algorithm

procedure gcd(a, b: positive integer)
do b ≠ 0 → r := a mod b; a := b; b := r od;
return a

procedure gcd(a, b: positive integer) Euclid’s algorithm

do a > b → a := a − b
| a < b → b := b − a
od;
return a

3.7 Application of Number Theory

3.8 Matrices

3/31/10 Kwang-Moo Choe 27