MULTIMEDIA DEVICES

Magister Teknologi Informasi

FT - UGM

Agendas
2

Human Perception
Human Computer Interface
Trend in Multimedia devices Technology

Multimedia MTI UGM

10/20/2014

Human Perception
3

Human or biological information processing

uses senses which include sensors for
receiving (but also for producing) signals which
encode information
Perception is a process of understanding
sensory information

Multimedia MTI UGM

10/20/2014

What are the Human perceptual Senses?

4

That should be easy:

SENSE:
INPUT
___
Visual
Eyes
Acoustical
Ears
Haptic
Touch/Warm
Smell
Nose
Taste
Tongue

OUTPUT____
Vocal Tract
Motoric
Skin/Body???
-

Notice that: usually multiple senses operate at the same

time.There is operation across different senses, for
example motoric output can be visual, signal, like
hand waving.
Multimedia MTI UGM

10/20/2014

What are the Human perceptual Senses?

5

But senses mean little without processing and

command center wchich is brain.
Brain + Senses = Perceptual system
Human senses operate in perfectly coordinated
way, they form completely integrated system
This system can receive and processes
multimodal signals from external world and
produce multimedia output seemingly with no
effort
Multimedia MTI UGM

10/20/2014

What are the Human perceptual Senses?

6

How this is done in the brain we dont know exactly and

it would be good to know this:
The more we think about it the more we conclude that
humans are fantastic multimedia system
We use many encoding levels of information from basic
to very abstract, for example :

Sound
Speech-language
Music
Gesture
Reading
writing
Multimedia MTI UGM

10/20/2014

What are the Human perceptual Senses?

7

There are good reason to think that human sensory

system is optimized to produce and receive
multimodal information but optimality is done on
many levels:
We can easily produce and receive single media
information eg. When talking on the phone
But we can also easily deal with multiple media
information eg having direct conversation and
looking to face expression
Multimedia MTI UGM

10/20/2014

What is Multimedia from the Point Of

Human Perception?
8

Multimedia means using multimodal

information to achieve optimal stimulation
or perceptual system
Often stimulation also depends on
external conditions
Example of Cinema : conditions are
created for optimal stimulation: theater,
darkness, screen and play action + music

Multimedia MTI UGM

10/20/2014

What Means of Optimal Stimulation?

9

It is described as
Concentration,
Full

Involvement
Living experience
Cinema

illustrates this. But also

playing Computer Games induces
very high stimulation.
Multimedia MTI UGM

10/20/2014

Multimedia technology aims to provide (some)

matching of information devices to human
perception system
MULTIMEDIA
SYSTEM:
INTERFACE,
PROCESSING,
DELIVERY OF
INFORMATION

NETWORK

Human Computer Interface

11

Human Computer Interface.

Multimedia interfaces and applications.
Vision based interfaces
Systems

using vision based interfaces.

Multimedia MTI UGM

10/20/2014

Human Computer Interface

12

Different meanings of HCI

The

study of people, computer technology and the

ways they interactive with each other.
The design, evaluation and implementation of
computer systems for human use.
The research to study how to make computer
systems more useable.

Multimedia MTI UGM

10/20/2014

Aspects of HCI
13

Design
Intuition,

experiences and design principles.

Human factors
Experiments

testing the perception, memory and

cognition models of human being.

Devices
Physical

devices such as mouse, keyboards and other

fancy HCI hardware.

Software and systems

Such

as windows GUI, shell commands in Unix etc.

Multimedia MTI UGM

10/20/2014

HCI Studies
14

Computers

People

What are people good at?

What is the limitation of human abilities?
Input, output, memory, perception and cognition model.
No average answer. Must accommodate diversities.

Usability

What are they good at and bad at?

Input, output, processing, networking, etc.

More than the size of widgets, where are the buttons and syntax
of commands.

Context

Environment, physical constraints, cognition constraints, task, etc

Multimedia MTI UGM

10/20/2014

The Quality of HCI

15

There are fives factors that affect the

design and implementation of HCI
Time

to learn.
Speed of performance.
Rate of errors.
Retention over time.
Subjective satisfaction
Multimedia MTI UGM

10/20/2014

Multimedia Interfaces

Multimedia interface
combines different sorts of
media modalities to achieve
more natural and more
efficient human computer
communication.

Speech, audio, touch, video input

(patterns, posture, gesture) etc.

Put that there is an example

multimedia HCI that combines
speech, gestures and
graphics.

Advantages of Multimedia Interface

17

More alternative methods.

The flexibility of combining different schemes.
More efficient for specific tasks.
Greater precision of information.
Help handicapped people to use computers.

Multimedia MTI UGM

10/20/2014

Vision Based Interfaces

Vision based interfaces uses computer vision

techniques to achieve more effective HCI.
Vision based interfaces are based on techniques
such as:

Face detection
Head tracking
Gaze
Expression
Lip reading

Hand tracking,
Hand gesture,
Arm gesture
Hand Pointing

Posture, body gesture and

action recognition

Example Systems
19

Motion
Capturing

Multimedia MTI UGM

10/20/2014

Example Systems
20

The MIT Kidsroom

Posture recognition

Action
detection
Multimedia MTI UGM

10/20/2014

Our Changing World

21

Changing Computers
Changing Lives
Changing Societies

Multimedia MTI UGM

10/20/2014

Changing Computers
22

Four Computing Eras

GUIs to Gestures
VDUs to Smart Fabrics
Handsets to the World in our Hands
Simple Robots to Autonomous MachinesThat Learn
Hard Disks to Digital Footprints
Shrink-Wrapped to Mash-Ups

Multimedia MTI UGM

10/20/2014

Four Computing Eras

23

Multimedia MTI UGM

10/20/2014

Four Computing Eras (ctd.)

24

Multimedia MTI UGM

10/20/2014

GUIs to Gesture
25

The Reactable: a multitouch

interface for playing
music. Performers can
simultaneously interact
with it by moving and
rotating physical objects
on its surface. Reactable
was developed by Sergi
Jord and colleagues at
the Universitat Pompeu
Fabra, Barcelona. Icelandic
songstress Bjrk used one
on her 2007 tour.
Multimedia MTI UGM

10/20/2014

GUIs to Gesture
26

The HotHand device:

a ring worn by electric
guitar players that uses
motion sensors and a
wireless transmitter to
create different kinds of
sound effects by various
hand gestures

Multimedia MTI UGM

10/20/2014

VDUs to Smart Fabrics

27

Animated Textiles
developed by Studio
subTela at the Hexagram
Institute, Montreal, Canada.
These two jackets synch
up when the wearers hold
hands, and the message
scrolls from the back of
one person to the other.
Multimedia MTI UGM

10/20/2014

Handsets to the World in our Hands

28

Talk to the hand

From the first mobile
phone brick to the
latest Apple iPhone: as
the size reduces, the
potential expands.

Multimedia MTI UGM

10/20/2014

Simple Robots to Autonomous Machines That Learn

29

Emotional kitty: a robot

hardware platform called

iCAT uses a set of logical

rules to convey emotional
states as it makes decisions,
with the goal of improving
human-robot interaction.

It looks confused if its in

trouble, smiles if it gets

something right

Multimedia MTI UGM

10/20/2014

Simple Robots to Autonomous Machines That Learn

30

The Rovio robotic

webcam is wirelessly
connected to the Internet.
It roams around the home
providing an audio and
video link to keep an eye
on family or pets when
youre out.

Multimedia MTI UGM

10/20/2014

Hard Disks to Digital Footprints

31

Hard Disk

Flash Memory Devices

Sony Microvault
Tiny Drive

Thumb Drive

Multimedia MTI UGM

10/20/2014

Shrink-Wrapped to Mash-Ups
32

It used to be that only the most highly skilled software

developers could write applications, and only
professionalcontent producers could provide us with
digital data or content that we could use. In the early
days of the PC, we all bought our software in shrinkwrapped boxes, spending more money every few years
for a new updated version of our operating system,
word processor, or spreadsheet application.
Today New data sources are available to us all the
time. We are all fast becoming content producers,
publishers and developers as much as we are
consumers
Multimedia MTI UGM

10/20/2014

Changing Lives
33

Learning Differently
Ubi-learning in the
Ambient Wood
(University of Sussex):
a boy using a digitally
augmented probe
tool that shows realtime
measurements
of light and moisture
on an accompanying
mobile device.
Multimedia MTI UGM

10/20/2014

Changing Lives
34

Audiovoxs Digital
Message Center is
designed to be attached
to the refrigerator, letting
families scribble digital
notes and leave audio
and video messages for
each other

Multimedia MTI UGM

10/20/2014

Changing Societies
35

A man in Cape Town,

South Africa, selling mobile
phones. In 2007, 77% of
Africans had a mobile
phone, while only 11% had
computer access.

Multimedia MTI UGM

10/20/2014

Learning Activity of C-Generation

Teacher

Task, short Materials,

discussion

Students

Blog
Presentation and
Full Materials

Videos

Material Enrichment

Discussion

Youtube

Slideshare

Ziddu (File
Sharing)
Online Journals
Google Books

Facebook

Display Technology
37

Multimedia MTI UGM

10/20/2014

Flexible Display
38

Multimedia MTI UGM

10/20/2014

3 D Printer
39

Multimedia MTI UGM

10/20/2014

40

Multimedia MTI UGM

10/20/2014

Exercise
41

Make a paper and presentation about the recent

progress of Multimedia Devices:
Camera

Scanners
Display
Printer

Multimedia MTI UGM

10/20/2014