Machine Learning

Introducción

Sergio Yovine

Universidad ORT Uruguay

yovine@ort.edu.uy

26 de agosto de 2019
Content

I Data analytics and machine learning

I A bit of historical background

I What is machine learning (informally)

I Types of machine learning

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 2 / 17

Data analytics

I Data analytics is about asking/answering interesting questions from

data
I How well movie incomes correlate with viewer ratings or awards?
I How has the amount of cursing changed over time in writing/speaking?
I What is the impact of Chikunguya in Uruguyan children under 5?

I Predictive data analytics is about building/using models to make

predictions from data
I What will be the hotel demand in Piriápolis next summer?
I Will a client buy the premium package?
I Is a white spot in an MRI a malign tumor?

I Machine learning from data is a tool for predictive data analytics

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 3 / 17

Machine learning
I Machine learning is a branch of artificial intelligence

I Artificial intelligence is about building intelligent artifacts

I Learning is a key aspect of intelligence

I Learning involves
I representation
I memorization
I a great deal of inference/generalization

I In this context, a learner is an algorithm that

I Computes some function approximatively
I Error in data
I Insufficient data
I Is able to improve its performance with new evidence

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 4 / 17

Machine Learning today (Q1 XXI)
I Human behavior
I Advertisement (Google ads)
I Financial profile analysis
I Recommendations (purchases, jobs, ...)

I Biology and genetics http://goo.gl/HSioMP

I Genomics (gene finding)

I Systems biology (prot-prot. interact.)

I Statistical genetics (analysis of pedigree)

I Computer vision http://goo.gl/FiKEDd

I Face detection
I Altitude estimation in UAV
I Image denoising

I Language analysis
I Handwriting recognition

I Natural language processing

I Speech recognition

I ...
Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 5 / 17
Machine Learning roots: Learning machines

1950 Alan Turing.

Computing machinery and intelligence.
Mind, 59, 433-460.
http://goo.gl/HLsXuf

We have thus divided our problem into two parts. The child programme
and the education process. ... We cannot expect to find a good child
machine at the first attempt. One must experiment with teaching one
such machine and see how well it learns. One can then try another and
see if it is better or worse. ... It is probably wise to include a random
element in a learning machine.

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 6 / 17

Machine Learning roots: Brain models (Neuron Nets)
1943 W. McCulloch, W. Pitts.
A Logical Calculus of Ideas Immanent in Nervous Activity.
Bulletin of Mathematical Biophysics 5 (4): 115-133.
pdf

1958 F. Rosenblatt.
The Perceptron: A Probabilistic Model For Information
Storage And Organization In The Brain.
Psychological Review 65 (6): 386-408.
http://goo.gl/Qw0vld
The perceptron is a brain model, not an invention for pattern
recognition ... The Mark I perceptron learned to recognize letters by
being zapped when it made a mistake!
1969 M. Minsky, S. Papert.
Perceptrons: An Introduction to Computational Geometry.
pdf
... any good can come of experiments which pay no attention to
limiting factors ... as the small model is scaled up
Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 7 / 17
Machine Learning: End of XXth - Beginning of XX1st
1997 IBM Big Blue supercomputer defeats G. Kasparov.
J. Latson, Time, Feb. 17, 2015. http://goo.gl/Z44mqP
K. Finley,Wired, Sep. 28, 2012. http://goo.gl/lWG2qC

... he was again riled by a move the computer made that was so
surprising, so un-machine-like ... the computer chose a move at random
... the move ... was not a feature, but a bug.

1998 S. Brin, L. Page

Anatomy of a Large-Scale Hypertextual Web Search Engine.
http://goo.gl/EvHxLd

... information that can be inferred about a document ... starting to be

explored such as relevance feedback and ... user context (like the user’s
location)

2006 Hinton, G. E., Osindero, S. and Teh, Y.

A fast learning algorithm for deep belief nets.
Neural Computation, 18, pp 1527-1554.
http://goo.gl/mIVIKO

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 8 / 17

What do we actually mean by Machine Learning?
The field of Machine Learning seeks to:
I Build systems that automatically improve with experience
I Find the fundamental laws that govern all learning processes

What does it mean for a machine to learn?

A machine learns w.r.t. a particular task T, performance metric P, and
type of experience E, if the system reliably improves its performance P
at task T, following experience E.

What is Machine Learning mostly used for in practice?

I Explanation: Look for hidden patterns in raw data

I Understanding: Model the brain
I Prediction: Make a decision for unknown data

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 9 / 17

Examples of machine learning problems
I Learn addition
I T : add two natural numbers in binary form
I P : percentage correct answers
I E : send answers to a math teacher

I MRI tumor recognition

I T : recognize a tumor in an MRI
I P : percentage of correct answers
I E : dataset of correctly classified images

I Learn to play Pacman

I T : play Pacman
I P : percentage of games won
I E : play the game

I Other examples?

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 10 / 17

Types of learning

I Participants and their role

I Active: learn by asking questions to a teacher

I Example: binary number addition

I Passive: learn from data with no one no ask to

I Example: tumor recognition

I Reinforcement: learn by exploring/exploiting guided by rewards

I Example: play Pacman

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 11 / 17

Types of learning

I When it happens

I Online
I Incremental: online with limited memory
I Example: play Pacman

I Offline (or batch)

I Example: tumor recognition

I Combined offline/online
I Example: games, robots

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 12 / 17

Types of learning

I Property of the available data and question to be answered

I Supervised: data with answers (labeled data) is given to the learner

Data is a subset of X1 × . . . Xn × Y
X1 . . . Xn are called predictor variables
Y is the variable to be predicted (the answer/label)
I Example: tumor recognition

I Unsupervised: learner has to figure out the answer to the question

Data is a subset of X1 × . . . Xn
Use a distance (similarity) function d(x, x 0 ) to group data
I Example: Market segmentation, recommendation, word similarity

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 13 / 17

Types of learning

I Kind of answers (or outputs/labels)

I Classification: outputs are finitely many discrete values or classes

I Example: tumor recognition
I Unbalenced data: a few classes appear more often than the others
I Semi-supervised is not the same as unbalanced

Regression: outputs are real numbers

I Example: drug dosage prediction

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 14 / 17

Types of learning
Roughly, machine learning is about building a hypothesis (or model) that
answers a given question for any input

I Exact
I Build a hypothesis which is always true

I Statistical
I Learn from samples
I Error (loss): probability of the hypothesis to be false
I In practice: Minimize the empirical error (best effort)

I Probably Approximately Correct (PAC)

I A theory to reason about learnability
I Build a hypothesis whose error is less than a given bound  with a
confidence greater than 1 − δ

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 15 / 17

This course

I Statistical
I Passive
I Offline
I Supervised
I Classification and regression

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 16 / 17

Bibliografı́a

I T. Mitchell. Machine Learning. McGraw-Hill, 1997. Chap. 1, Pg 1-5.

I S. Shalev-Schwartz, S. Ben-David. Understanding Machine Learning:

From Theory to Algorithms. Cambridge University Press, 2014. Chap.
1.1-1.4, Pg. 19-24.

Sergio Yovine Machine Learning - Introducción 26 de agosto de 2019 17 / 17