Escolar Documentos
Profissional Documentos
Cultura Documentos
Bull et al.
[54] APPARATUS FOR NON-INVASIVE
3.
[73] Assignee:
4,335,276
[11]
145]
Charlottesville, Va.
[22] Filed:
[51]
[52]
[58]
[56]
[57]
2/1947
3,281,534
10/1966
3,383,466
179/] SC
. . .. . . . .. . ..
ABSTRACT
. . . . . . . ..
128/630
Brady ....... ..
179/1 SC
179/1 SC
3,752,929
179/1 SC
3,483,941 12/1969
8/1973
Fletcher .... ..
179/1 SC
3,855,416 12/1974
179/1 SP
Fuller .... ..
3,881,059
4/1975
Stewart
3,906,936
9/1975
Habal
4,015,088
4,061,041 12/1977
.. 179/1 SP
. . . . . . .. ..... . .. .
. . . ..
179/1 SC
Fletcher et a1.
4,074,069
4,187,396
2/1980
128/ 724
..... .. 73/646
179/1 SC
OTHER PUBLICATIONS
F ______ _ZNZIIIEPEE~PROCES___S_IE_____II
Am 1 HP
,4
EMS is
ms
@_l_FILTER ATEIJCONV _ ?i
MIC
'
'
HP
I FILYER
u/v
22 001w
'
01v
as
_
L06
[- ___1H4;OE'?05I15L1|
r
40
T
4
a
|
1
MU) _ 3/ _ V9
52
1
I
LP 2
28 FILTER
AUX 1
~05
(34) :
AM 56
l_
A66 A10
i
___
l
{
L- ----------- ___-J
PS
,56
DISPLAY
II,
gisliqic i2
cavmaLER
1
D/A
______________ __
{_
l
e
5
'
1/0
44
CPU _
7
m i
Emmi :
58
CONTROL
I
|
L
INTERHUPT
TIMERS
\48
MICROCOMPUTER
i
|
_J'
Sheet 1 of 17
NORMAL
4,335,276
H YPEWASAL
QENASAL
OFNIREASODNEXLC
SUBJECTS
H611
$5
$6
$7
U.S. Pateznt
Sheet 3 of 17
4,335,276
Ey
?
_
E_
__
1324
2634 MS
1; +1234
> TAKEDATA
A: _ 234
Trim
FIG. 3
US. Patent
Sheet 4 of 17
4,335,276
WM
m?lI
'lllllllllllkvmh
FIG. 4
US. Patent
Sheet 5 of 17
COMMAND
paocessoa
' FIG. 5A
"''-"
4,335,276
COLLATERAL
PROCESSOR
MAIN DISPLAY
DATA
A U
co ISITION
& SPEECH
PLAYBACK
GET
KEYBOARD
comma
FI6. 58
VALID
COMgfAND
PRINT ERROR
MESSAGE
TRANSFORM
TAKER
LOOK UP
COMMAND IN
JUMP TABLE
8 GO THERE
SPEECH
TAKER
U.S., Patent
NA/N LOOP
4,335,276
Sheet 6 of 17
TRANSFORM
CURSOR
ROUTINE
_QL_, ggaggggq
4I
I
CURSOR
,5
LOOPS
5
5
MOVEMENT
FIG . 5D
I
I
SYNCHRONOUS
SPEECH
PLAYBACK
LOOP
INTERRUPTS
(~ MAIN LOOP )
FIG. 6A
CLEAR SPEECH
\
TURN ON TIMERS
a ENABLE
lNTERRUPTS
RECORD
FULL FLAG
,
SET
INTERRUPT
VECTORS
WAIT FOR
INTERRUPT
TRANsFORN
REAOY FLAG
sPEEcR
STORAGE RECORD
FULL?
REAL-TIME
D'SPLAY
PLOT TRANSFORM
POINT ON SCREEN
INTERRUPTS
EXIT TO MAIN
DISPLAY a
$P5gU?gBACK
US. Patent
Sheet 7 of 17
4,335,276
INTERRIIPT
FROM SPEECH
k
TINIR
SET UP MULW
PLEXER 8 ACOUIRE
A RAW SPEECH
vALuE FROM AN)
YES
S'LENT
SILENT
M55??? FLAG
INTERVAL ~
C AR SILENT
INLTEERVAL FLAG
SET SILENT
INTERvAL
FLAG
INCRENENT
SPEECH STOR
ACE POINTER
INCRENENT SPEECH
STORACE POINTER a
STORE
RAw SPEECH
SET MEMORY
VALVE
LOCATION = PF
7
INCRENENT SPEECH
STORACE POINTER
ENABLE
MEMORY LOCATION
V
c'?i?smoi
SPEECH STORAGE
LOCATION
'
RETURN
FIG- 6B
'NTERRUPTS
wHERE INTERRuPTEO
BY SPEECH TAKER
U.S. Patent
Sheet 8 of 17
4,335,276
TRANSFORM
INTERRUPT
ENABLE
INTERRUPTS
2. TRANSFORM CALCULATIONS,
I
WAIT FOR SPEECH INTERRUPT TO MAINTAIN SYN
CHRONY WITH SPEECH
INCREMENT
TRANSFORM
k STORE TRANSFORM
STORAGE POINTER
VALUE 0F F
SET UP MULTIPLEXER
8 ACOUIRE
TRANSFORM VALUE(S)
I
PERFORM TRANSFORM
CALCULATIONS
FIG. 6C
(IF ANY)
INCREMENT
TRANSFORM
STORAGE POINTER
STORE
TRANSFORM
VALUE
SET
* TRANSFORM-READY
FLAG
>
RETURN
Mm!
Patent
Sheet 9 of 17
4,3352?
INTERRUPT FROM
RANSFORM TIMER '
SET UP MULTIPLEXER
8 ACOUIRE MIC
LOGARITHMIC RMS
VALUE
FIG. 60
SET UP MULTIPLEXER
a ACOUIRE ACCL
LOGARITHMIC RMS
VALUE
MIC
RMS VALUE
RMS VALUE
< THREIPSHOLD
< THREJSHOL
CCL
AT
LgITFFELRLENOE
5ET TRANSFORM
OF MIC HACCL
VALUE = FF FF
LOGARITHMS
SET TRANSFORM
READY FLAG IN
MAIN LOOP
INCREMENT
TRANSFORM
STORAGE POINTER
STORE
TRANSFORM
RETURN
VALUE
U.S. Patent
Sheet 10 of 17
4,335,276
REAL-TIME
DISPLAY
V
GET
TRANSFORM
POINTER
GET
TRANSFORM
VALUE
MORE
POINTS ON
SCREEN
CLEAR SCREEN
?
T
MAP TRANSFORM
VALUE INTO Y-AXIS
INITIAL/2E
X -AXIS COUNTER
INCREMENT X-AXIS
COUNTER a LOOK
UP COORDINATE
F I 6. 6 E
PLOT TRANSFORM
POINT ON SCREEN
CLEAR
TRANSFORM-READY
FLAG
>
U.S. Patent
Jun_:15',1982
Sheet 11 of 17
4,335,27
OUTPUT AVERAGE OF
( MAIN' Loop
W VALUES
ALL TRANSFORM
r0 SCREEN
'
-' SET
'
SET cunson
INVTEEQZLQT
HALF FLAG
1!
CALL
TRANSFORM
CURSOR LOOP
CLEAR SCREEN
V
SET UP POINTERS.
TRANSFORM
STORAGE, E TC.
"
FORMATa
DRAW PAGE
0~ SCREEN
!
PLOTSCREEN-
CL EAR SCREEN
ffjfegs ppm
FORMATS
FULL OF TRANS -
DRAW PAGE
FORM POINTS
0N SCREEN
1!
AVERAGE ALL VA LUES IN TRANSFORM
STORAGE RECORD
60 TO
A
GO TO
A
US. Patent
Sheet 12 of 17
4,335,276
TRANSFORM
CURSOR LOOP
PLOT CURSOR
AT CURRENT
POSITION
CURSOR
CURSOR
TOGGLI; HALT
roam; LEFT
C:
1'
RIGHT
"
FIG. 7B
N0
U050 Patent
4,335,276
Sheet 13 of 17
CURSOR
LEFT FLAG
s51 ?
CLEAR CURSOR -
LEFT FLAG
RESYNCH SPEECH 8
TRANSFORM POINT
ERS AFTER RIGHT/NO
MOVEMENT
'
CLEAR CURSOR
RIGHT FLAG
INITIALIZE
SPEECH-SYNCH
COUNTER; SET CUR
SOR- HALT FLAG
FIG. 7C
I
UPDATE CURSOR
POSITION DISPLAY
I IN msec)
I
UPDATE CURRENT
TRANSFORM
VALUE DISPLAY
EXIT TO
COMMAND
PROCESSOR
A = ENTRY POINT TO
TRANSFORM - CURSOR LOOP
U.S . Patent
4,335,276
Sheet 14 of 17
WAIT FOR
TRANSFORM
INTERRUPT
ENABLE
INTERRIIPTG
GLEAR
CURSOR-HALT
FLAG
WAIT FOR
SPEECH
1N TERRUPT
RESYNCH SPEECH 8
TRANSFORM POINTERS AFTER RIGHT
MOVEMENT
INITIAL/Z5
SPEECH-SYNCH
COUNTER
GLEAR CURSOR-
DECREMENT
TRANSFORM
RIGHT FLAG
sToRAGE
POINTER
INITIAL/2E
SET CURS0R_
COUNTER
TURN ON TIMERS
a ENABLE
INTERRuPTs
I 60 TO
A = ENTRY
POINT T0
TURN OFF TINERG
a MA BLE
FI 6 . 7D
INTERRUPTS
TRANSFORM -
CURSOR LOOP
U.S. Patent
4,335 ,276
Sheet 15 of 17
WA IT FOR
TRANSFORM
IN TERRUPT
v
ENABLE
INTERRUPTS
CLEAR
CURSOR-HALT
& DISABLE
FLAG
WAIT FOR
. SPEECH
[NTERRUPTS
INTERRUPT
RESYNCH SPEECH
I
a
TRANSFORM POINT-
,
5PC;,_$mcH
COUNTER
I T AL/ZE
INCREMENT
CLEAR CURSOR
TRANSFORM
LEFT FLAG
STORAGE
POINTER
INITIALIZE
SPEECH-SYNCH
SE7- CURS0R_
LEFT FLAG
COUNTER
I
TURN ON TIMERS
8 ENABLE
INTERRUPTS
E
A
A = ENTRY
POINT TO
FIG. 7E
TURN OF TIMERS
8 DISABLE
IN TERRUP TS
CURSOR LOOP
US. Patent
Sheet 16 of 17
SPEECH
IN TERRUPT
INCREMENT
SPEECH -SYNCH
COUNTER
FIG. 7F(iI
TOFIG. TFIII)
NEW
SILENT
INTERVAL .9
YES
DECREMENT
SILENT INTERVAL
PLAYBACK
COUNTER
OUTPUT RAW
SPEECH VALUE
TO DA C
I
INCREMENT
SPEECH STORAGE
POINTER
ENABLE
INTERRUPTS
4,335,276
U.S. Patent
Sheet 17 of 17
4,335,276
FROM
COPY VALUE INTO
SILENT INTERVAL
PLAYBACK
COUNTER
T
DECREMENT
SILENT INTERVAL
PLAYBACK
COUNTER
DECREMEN T
SPEECH STORAGE
POINTER
V
ENABLE
INTERRUPTF
RETURN
FIG. 7F (ii)
F16. 7F ( i 1
4,335,276
the measure.
4,335,276
resonance.
ratios.
A ratio of the two linear RMS signals is formed by
means of a divider circuit in hardware. The digital pro
control panel.
phoneme.
the screen.
20
Another object of this invention is to provide a novel 25 stored concurrently with the ratios formed from the
sampled RMS channels in such a manner that the rela
apparatus which permits identi?cation of the phonemic
tive relationship in time between the ratios and the
content of speech associated with speci?c sections of
digitized audio signal is preserved. A moving cursor can
static graphic displays of measures of other transforms
be advanced across the graphic plot synchronously
of speech such as intensity and pitch over time.
These and other objects are achieved according to 30 with the replayed audio signal, permitting identi?cation
of the phonemic content associated with a given seg
the invention by providing a new and improved appara
ment of the plot. This is accomplished by means of a
tus for non-invasive measurement and display of nasal
toggle with three positions: cursor right, cursor left, and
ization in human speech including the following sec
halt. A binary code corresponding to each position of
tions: two transducers (an accelerometer and a direc
tional microphone), an analog proprocessing section, an 35 the toggle is sent to the digital controller, which directs
the movement of the cursor accordingly.
analog-to-digital converter, a digital data processor, a
When the cursor is halted, the instantaneous value of
display section, and a control panel.
the ratio and the time in milliseconds at that point in the
The accelerometer is mounted on the external nasal
utterance are displayed in the lower and upper right
wall for measurement of nasal wall vibration, while
hand corners of the screen respectively. Thus the abso
airborne sound consisting of combined nasal and oral
lute value of a ratio formed at a speci?c vtime in the
output is transduced by the directional microphone.
The microphone is mounted on a headset to maintain a
the two RMS signals. An AGC circuit on the output of 55 in perceptually continuous speech is coded, rather than
storing each sample with a value of zero as a separate
the raw speech channel can be enabled to improve the
byte. During playback of the digitized speech signal, a
?delity of transient consonants such as voiceless /th/
series of zeros is then sent to the digital-to-analog con
which have an inherently low relative intensity level.
verter for the duration coded at that point in the stored
The two RMS signals are provided in two forms:
of microprocessors, controls the multiplexing and ana 65 ment of the nasal wall in some individuals. When the
speed of this movement exceeds the frequency of the 30
log-to-digital conversion of the respective signals. The
Hz ?lter on the output of the accelerometer, an artifact
digital processor forms a ratio of accelerometer output
consisting of a sharp spurious peak in the graphic dis
over microphone output for each successive pair of