Haptic User Interfaces

Fall 2007

Contents

TACTILE SENSING & FEEDBACK

Jukka Raisamo
Multimodal Interaction Research Group Tampere Unit for Computer-Human Interaction Department of Computer Sciences University of Tampere, Finland
1

Tactile sensing in detail Tactile feedback Feedback technologies & displays

Tactile sensing
Theres two different types of receptors responsible for tactile sensing found in the skin

Tactile sensing

free nerve endings encapsulated nerve endings, i.e., mechanoreceptors

Most tactile information is delivered via mechanoreceptors but, e.g., hair receptors also affect the sensations
Bent hair Skin Indented skin

Bent hair RA receptor

Indented skin RA receptor

Sustained pressure SA receptor

Mechanoreceptors 1/3

Mechanoreceptors 2/3

Thresholds of different receptors overlap

perceptual qualities of touch are determined by the combined inputs from different types of receptors operating range for the perception of vibration about 0.04 to 500 Hz frequencies over 500 Hz are felt more as textures, not vibration skin surface temperature affects perceiving tactile sensations
5

Mechanoreceptors are sensitive to mechanical pressure or deformation of the skin

four types: Meissners corpuscles, Pacinian corpuscles, Merkels disks and Ruffini endings differ in size, receptive fields, rate of adaptation, location in the skin, and physiological properties
4

Jukka Raisamo

Haptic User Interfaces

Fall 2007

Mechanoreceptors 3/3
Receptor Merkels disks Ruffini endings Meissners corpuscles Pacinian corpuscles Rate of Stimulus adaptation frequency SA-I 030 Hz SA-II RA-I PC (RA-II) 015 Hz 1060 Hz Receptive Location field 23 mm High >10 mm 35 mm Deep High Deep Function Pressure; edges and intensity Directional skin stretch, tension Local skin deformation, low frequency vibratory sensations Unlocalized high frequency vibration; tool use

Hairy vs. hairless skin

Hairy skin is generally less sensitive to vibration compared to glabrous skin
there seems to be no PC receptors in the hairy skin, however, they are present in the deeper underlying tissue surrounding joints and bones

80400 Hz >20 mm

Mechanoreceptors are generally specialized to certain stimuli

contact forces are detected by Merkels discs and Ruffini endings vibration primarily stimulates the Meissners corpuscles and Pacinian corpuscles
6

Hairy skin has poorer absolute threshold for both vibration & pressure
still about the same capacity for discriminating vibrotactile frequencies

Tactile dimensions

Tactile acuity for vibration

Vibration primarily stimulates the Pacinian corpuscles and Meissners corpuscles

Tactile acuity (vibration & pressure) Spatial acuity Temporal acuity

pacinian channel (high frequency, from about 60Hz) non-pacinian channel (low frequency, below 60Hz)

Human thresholds for detecting vibration:

sensitivity for mechanical vibration increases above 100 Hz and decreases above 320 Hz (250 Hz being optimum)

The spatial acuity and pattern perception is better for skin deformation compared to vibrotactile stimuli
8 9

Tactile acuity for pressure

Tactile acuity
Threshold responses for pressure (bars) and vibration (dots) for 15 body sites
human body is highly sensitive for vibration thresholds correlate with the density of cutaneous mechanoreceptors

Sensitivity for pressure is largely dependant on the area of stimulation

discrimination has higher resolution at those parts of the body with a low threshold (e.g. fingertips)

Discrimination is not constant throughout the entire intensity scale, as with vision and auditory senses
amplitude indentation discrimination is low at low intensities

10

11

Jukka Raisamo

Haptic User Interfaces

Fall 2007

Deterioration of tactile acuity

There appears to be no significant reduction in vibrotactile detection at the fingertips in older subjects.
reflects either the high receptor density of the area, or the functional importance of vibrotactile sensibility of the fingertips (or some combination of both of these factors)

Spatial acuity 1/3

Fingertips are the most sensitive part of the human hand in texture & vibrotactile perception
corresponding to the largest density of PC receptors the more spatially distant two stimuli are, the more difficult it is to discriminate them

Tactile texture perception is mediated by vibrational cues for fine textures, and by spatial cues for coarse textures
discrimination of spatial information is considerably more accurate than their temporal interval when using hand, exploration of spatially varying surfaces is done with the entire fingertip (increased sensitivity by active touch)
13

Pressure sensitivity reduces as a function of age Training can be used to improve sensimotor performance
12

Spatial acuity 2/3

Threshold = the point at which an effect begins to be produced
detection threshold (the smallest detectable level of stimulus; a.k.a absolute threshold) difference threshold (the smallest detectable difference between stimuli; a.k.a just noticeable difference (jnd))

Spatial acuity 3/3

Spatial dimension for touch
2-point discrimination (1 mm at fingertips, 3070 mm in the back) localization texture detection (depends on the surface) grating discrimination (detectable distance between two gratings) pressure sensitivity temporal discrimination

Successful method to reduce the detection threshold is either to increase the duration of the tactile stimulation, or the interval of two consecutive stimuli Why do people do better with gratings than twopoint discrimination?
active vs. passive touch
14

15

Spatial acuity for pressure 1/2

Spatial acuity for two-point thresholds (bars) and errors of localization (dots) for 14 body sites
smallest resolution in facial area & hands differences due to both task demands & neural activity

Spatial acuity for pressure 2/2

Variation in pressure threshold over the body

smallest in facial area fingers have about the same acuity as trunk the right side seems to have slightly better acuity on average

16

17

Jukka Raisamo

Haptic User Interfaces

Fall 2007

Temporal acuity

Thermotactile interactions
Eventhough being separate modalities, temperature and touch have interactions
thermal adaptation

Resolution of temporal frequency discrimination is finer at lower frequencies Thresholds for tactile sensations are lowered with increased duration and interval

cooling degrades tactile sensitivity warming sometimes enhances

thermal intensification
cold objects feel heavier warm objects feel heavier but less than cold ones

thermal sharpening
the warmer or colder the two points are, the easier they are to discriminate

Thermal cues are very important in the identification of textures

18 19

Touch is not an absolute sense

Several factors affect the sensitivity
age individual differences, habits attention, fatigue, mood, stress diseases, disabilities training ... scalability is important factor for tactile interfaces
20 21

Tactile feedback technologies

Methods for tactile stimulation

Types of skin sensory stimulation:
skin deformation vibration electric stimulation skin stretch friction (micro skin-stretch) heat

Tactile actuators
Some technologies used in tactile interfaces
vibrating motors linear motors solenoids piezoelectric actuators pneumatic systems shape memory alloys electrorheological fluids thermoelectric elements
23

Possible actuator configurations:

single element multiple elements (array)
22

Jukka Raisamo

Haptic User Interfaces

Fall 2007

Vibrating motors
How they work:

Actuators: vibrating motors

provides relatively small-amplitude vibration (linear or rotary) applies motion either directly to the skin or through mediating structure used singly or in arrays

Most common types

DC-motors with eccentric rotating mass voice coils
24 25

Vibrating motors: eccentric rotating mass

DC-motor rotates an offcenter spinning mass
inexpensive & exsisting technology poor resolution: it takes time to start and stop

Vibrating motors: voice coils

Voice coil basics
current driven through the movable coil created magnetic field interacts with the field of the permanent magnet (one-way movement) vibrations created by switching the current on/off

Frequency control only (amplitude = freq2)

amplitude fixed by the size & the weight of the rotating mass

Used in various devices

mobile phones, pagers, gaming devices, etc.
26

Both frequency and amplitude can be controlled somewhat independently

however, the motor has always peak at certain frequencies (e.g. 250 Hz)
27

Vibrating motors: overview

Advantages:
simple, existing technology relatively inexpensive easily powered and controlled quite small power consumption

Example: vibrotactile devices

Logitech iFeel mouse & Kensington Orbit 3D trackball These ones use the TouchSense technology by Immersion Corporation
(http://www.immersion.com)

Disadvantages:
not very expressive feedback vibration can be irritating sometimes hard to miniaturize efficiently
28

Have a small rotating DCmotor inside the device which applies the vibration through the structure
29

Jukka Raisamo

Haptic User Interfaces

Fall 2007

Linear motors: pin displays

How they work:
pins in an array are actuated independently the actuated pins contact the surface of the skin

Actuators: linear motors

Advantages:
simple, readily available continuously positionable versatile: static pressure, vibration; shapes or force display relatively fast

Disadvantages:
very difficult to pack tightly relatively high cost (lots of motors/device)
30 31

Example: tactile array

Mimics complex tactile sensations Exeter arrays
stimulate the fingertips each pin has piezoelectric actuator Array 1: 100 pins over 1 cm2, frequency range 25-400 Hz Array 2: 24 pins with 2 mm spacing, 25-500Hz

Example: tactile arrays in a mouse

Allows the user to scan the of an image

the pins rise and fall dynamically delivering a tactile stimuli to the fingertips can be used to code patterns and colours into tactile data

VTMouse (2001)
three 4x8 matrix (32 pins) put in the place of the buttons

VTPlayer (2003)
two 4x4 matrix with 16 pins (http://www.virtouch2.com/)

32

33

Solenoids
Multi-modal mouse by Akamatsu & MacKenzie (1996)
solenoid driven pin under the right index finger that rises and falls

Actuators: solenoids

Haptic Pen by Lee et al. (2004)

solenoid shakes the pen by moving up and down in top of the pen
34 35

Jukka Raisamo

Haptic User Interfaces

Fall 2007

Example: solenoids in Braille displays

Braille = tactile language for sensory substitution Traditionally Braille displays use solenoids to push up the pins (nowadays mostly piezoelectric actuators are used) Solenoids have poor power consumption
36 37

Actuators: piezoelectric actuators

Piezoelectric actuators 1/2

How they work:
single or multilayer ceramic elements an element expands/bends when voltage is applied multiple layers can be used to amplify the effect

Piezoelectric actuators 2/2

Electromechanical device that converts electrical energy into mechanical motion Typically very compact as only few components are used in a complete system
actuator itself can be very small

Properties:
very large forces but small motions one element typically around 0.2-1.0 mm thick resolution for frequencies ~0.01 Hz
38

39

Example: STReSS & Virtual Braille Display

Example: Tactile Handheld Miniature Bidirectional (THMB)

THMB is an improved version of VBD miniaturized to fit inside a PDA-size case The handheld device comprises an LCD screen that allows combining tactile and visual feedback THMB stimulates the user's thumb and is mounted on a vertical slider so that it can be dragged up and down along the left side of the case
(http://www.laterotactile.com/)
41

2D tactile display with an array of miniature actuators

stimulate the fingertip at about 1 cm2 in area
(Hayward et al.)

elements can be bended in two directions to increase the forces applied to the fingertip

40

Jukka Raisamo

Haptic User Interfaces

Fall 2007

Piezoelectric actuators: overview

Advantages:
small in size potentially inexpensive in large volumes high frequency and static modes very fast response time low power consumption

Actuators: pneumatic systems

Disadvantages:
dynamics: small displacements require accurate amplification high voltage
42 43

Pneumatic systems
Two possible output modes based on skin indentation (and vibration)
suction air-pressure

Pneumatic systems: suction

Draws air from a suction hole creating an illusion that the skin is pushed Very low spatial resolution (only appropriate for the palm)
two basic patterns of stimulation (large holes and small holes)

How it works:
technologies: fillable air-pockets, air jets, suction holes vibratory rates: typically 20-300 Hz static pressure with sealed pockets
44

Need for regulation of air pressure (=lots of equipment)

45

Pneumatic systems: air-pressure

Pneumatic systems: overview

Advantages:
tubing make it possibly to take the bulky part away from point of application pressure can be more appropriate for some applications than pins or vibrating motors can mimic skin-slip (with multiple adjacent inflated pockets)

DataGlove with pneumatics (Sato et al., 1991)

Teletact II (Stone, 1992)

Disadvantages:
requires bulky parts (air compressor or motor-driven pistons) not really portable can be very noisy difficult to display sharp edges or discontinuities
47

DataGlove
bandwidth of 5 Hz, amplitude & frequency modulated

Teletact II
29+1 air pockets (40 tubes to control the air-pressure) object slippage (fingers) + force feedback (palm)
46

Jukka Raisamo

Haptic User Interfaces

Fall 2007

Shape-memory alloys
Metals that "remembers" their geometry

Actuators: shape-memory alloys

restores its original geometry when heated usually temperature change of about 10C is necessary to initiate the phase change

How it works:
expands (and heats up) when current runs through it contracts when cools down stimulates the skin when vibrates (expandcontract cycles)
48 49

Shape memory alloys

Wearable Tactile Displays (MIT Touchlab) Tactile Display based on Shape Memory Alloy

Actuators: electrorheological fluids

Tactile Display based on Elastomer Actuators

50

51

Electrorheological fluids

Electrorheological fluids: overview

Advantages:
low power consumption no moving parts controlled electrically very compact performance improves as size decreases

Liquid which viscosity changes into semi-solid when electric current is applied
change in viscosity feels as more resistive surface usually packed in 2-3mm bubbles can change from liquid to gel, and back, within milliseconds

Disadvantages:
high voltage required cant control force, only viscosity sharp edges and discontinuities difficult to render
53

The change in viscosity is proportional to the applied current

52

Jukka Raisamo

Haptic User Interfaces

Fall 2007

Skin-stretch
Two main methods:
rotational skin stretch lateral skin stretch

Tactile displays: skin stretch

What happens:
forces are applied to skin for displacement contact forces are perceived as stretching of the skin

Applying skin stretch is being investigated as an alternative method to vibrotactile feedback

54 55

Friction: skin-slip display

Micro skin-stretch
motor driven smooth cylinder strapped against finger when rotates, stimulates the mechanoreceptors
(Chen and Marcus, 1994)

Tactile displays: electrotactile stimulation

Felt as a sensation of slip

grasp simulations: causes the user to increase grip force often used to append force feedback displays

56

57

Electrotactile stimulation
Electrical stimulation is not widely accepted to consumer use
often sudden bursts give an "invasive" impression square waves can be easily felt as too strong stimuli and they keep tickling the nerves the sensitivity to electrical stimulation varies greatly between and within individuals (e.g., sweating & pressure affect the sensation)

Example: SmartTouch

Tactile display to present realistic skin sensation

a thin electrotactile display and a sensor mounted

Two layers
top layer: 4x4 array of stimulating electrodes bottom layer: optical sensors

Visual information is captured by the sensors and displayed through electrical stimulation
e.g. the black stripes are perceived as bumps

Used mostly in research prototypes and for rehabilitation purposes

58

(http://www.star.t.u-tokyo.ac.jp/projects/smarttouch/)

59

Jukka Raisamo

10

Haptic User Interfaces

Fall 2007

Example: Electric mouse

Array of small electrodes placed to fit fingertip
64 electrodes, 1mm in diameter

Example: Bioforce by Mad Catz (2001)

A game controller that delivers mild cramps to the user An electrical shock is delivered by wired pads attached to the forearm 3x1.5V batteries provide 16 mA shocks
similar to the shocks used for years by physical therapists

Pressure sensor located under the electrodes to measure finger pressure.

electrical current is controlled as a function of pressure creates more stable vibratory sensations compared to traditional displays
(http://www.star.t.u-tokyo.ac.jp/projects/tactile-display/)
60

61

Jukka Raisamo

11