Você está na página 1de 42

CONTENTS

1. Introduction 3
2. A Smart Camera 6
2.1 Overview of smart camera 7
2.2 Smart camera V/s standard smart vision system 8
2.3 2.3.1 PC based vision system advantages 9
2.3.2 Smart cameras advantages 9
2.! Smart camera arc"itect#re 1$
2.!.1 %mage sensor basics 11
2.!.2 &na'og to digita' conversion e'ectronics 12
2.!.3 %mage (rocessing e'ectronics 12
2.) %mage (rocessing a'gorit"ms 13
2.).1 Static grey sca'e t"res"o'ding 1!
2.).2 *mbedded image (rocessing a'gorit"m 1)
2.+ Smart camera system 1+
3. Embedded systems 17
3.1 %ntrod#ction to embedded systems 18
3.2 *,am('es of embedded systems 19
3.3 C"aracteristics of embedded systems 21
4. Smart cameras as embedded systems 22
!.1 Overview of SC*S 23
!.2 -etection and recognition a'gorit"ms 23
1
!.3 .e/#irements 28
!.! Com(onents 28
!.) 0estbed arc"itect#re 28
!.+ *,(eriments and o(timi1ations 29
!.7 &'gorit"mic c"anges 3$
!.8 Contro'2to2data transmission 31
5. A!ications o" smart cameras 32
).1 3ie'ds of a(('ication 33
).2 &(('ication of smart camera networ4s 3!
).2.1 Vis#a' "#'' reconstr#ction 3!
6. Conc!usion # some "uture trends 36
+.1 O(timi1ation res#'ts 5 conc'#sion 37
+.2 6oo4ing &"ead 38
+.2.1 3#t#re a(('ications 38
7ib'iogra("y !1
2
Introduction
A smart camera er"orms rea!$time ana!ysis to reco%ni&e scenic e!ements. Smart
cameras are use"u! in a 'ariety o" scenarios( sur'ei!!ance) medicine) etc. 8e "ave b#i't a rea'2
time system for recogni1ing gest#res. O#r smart camera #ses nove' a'gorit"ms to recogni1e
gest#res based on 'ow2'eve' ana'ysis of body (arts as we'' as "idden 9ar4ov mode's for t"e
moves t"at com(rise t"e gest#res. 0"ese a'gorit"ms r#n on a 0rimedia (rocessor. O#r system can
recogni1e gest#res at t"e rate of 2$ frames/second. 0"e camera can a'so f#se t"e res#'ts of
m#'ti('e cameras.
A *smart camera* is basica!!y a 'ideo camera cou!ed to a comuter 'ision system
in a tiny ac+a%e. 0"is comm#nication begins stating t"e main differences between smart
cameras and standard smart vision systems. Smart camera arc"itect#re is described w"ereby a
combination of an on2board micro(rocessor and P6-s a''ow for t"e embedding of image
(rocessing a'gorit"ms in t"e camera. & static t"res"o'ding a'gorit"m is (resented w"ic"
demonstrates t"e abi'ity to trac4 non2#niformity in t"e ins(ection target. & m#'ti camera
ins(ection system a(('ication is (resented w"ere a ma,im#m of twenty smart cameras may be
networ4ed toget"er to a sing'e "ost com(#ter. 3ina''y: a (rediction is made of tec"no'ogica' and
a(('icationa' f#t#re evo'#tion on smart cameras.
0"e smart camera ; a w"o'e vision system contained in one neat "o#sing ; can be #sed
anyw"ere: in any ind#stry w"ere image (rocessing can be a(('ied. Com(anies no 'onger need a
cabinet in w"ic" to 4ee( a'' t"eir com(#ting e/#i(ment< t"e com(#ter is "o#sed wit"in t"e smart
camera. %n t"e ("armace#tica' ind#stry and in c'ean rooms ; w"en not even d#st is a''owed ; t"is
can be a big advantage. & sing'e s/#are meter of s(ace can be com(arative'y very e,(ensive ; if
t"ere is no need for a com(onent rac4 or cabinet: sim('y a smart camera: and t"en t"is co#'d save
a 'ot of money. %n (artic#'ar: t"ere wo#'d not be t"e #s#a' cab'ing invo'ved for ot"er vision
systems: and set2#( is sim('e. 6ater in t"is comm#nication are stated some advantages of #sing
smart cameras or PC2based systems in vision a(('ications.
%n #s#a' vision systems scenarios: on'y a sma'' fraction of a (ict#re frame wi'' be t"e
region of interest =.O%>. %n fact: often no vis#a' image of t"e .O% is even re/#ired. 0"e ob?ect of
a vision system: after a'': is to ma4e a decision< @%s t"ere a b'ob@A @8"ere is t"e b'ob@A @%s t"is a
defect@A
8"at if a'' t"at (i,e' (re2(rocessing and decision2ma4ing co#'d be done wit"in t"e
cameraA %f a'' t"e (rocessing were done inside t"e camera: t"e b'ob ana'ysis of a gigabit image
mig"t res#'t in on'y a few "#ndred bytes of data w"ic" need to be sent somew"ere. S#c"
com(act (ac4ets of data co#'d be easi'y transmitted direct'y to a mac"ine contro' wit"o#t even
(assing t"ro#g" a PC.
3
%nformation s"o#'d be (rocessed w"ere t"e information occ#rs ; i.e. t"e brain s"o#'d be
be"ind t"e eyesB
0"e answer to t"e (rob'em stated ear'ier is a stand2a'one: smart camera.0o i''#strates t"is:
a smart cameras embedded system arc"itect#re is s"own a'ong wit" an e,am('e of a "ardware
embedded image (rocessing a'gorit"m< Static Gray Scale Thresholding.
9any g'oba' com(anies inc'#ding C'a,o and Ciba2Ceigy: and motor com(anies s#c" as
3ord and Vo'4swagen are #sers of smart cameras.
& sma'' e,am('e of a vision system for web ins(ection is s"own w"ere twenty smart
cameras are connected toget"er.
9any c"anges are yet to come concerning smart cameras as in tec"no'ogy as we'' as in
t"ere f#t#re a(('ications. &n overview: concerning t"ese iss#es: is made c'osing t"is
comm#nication. 8e "ave to recogni1e "owever t"at smart cameras wi'' not be t"e answer for a''
vision a(('ications.
4
S,A-T CA,E-A
2.1 O.E-.IE/ O0 S,A-T CA,E-A
& smart camera is an integrated mac"ine vision system w"ic": in addition to image
ca(t#re circ#itry: inc'#des a (rocessor: w"ic" can e,tract information from images wit"o#t need
for an e,terna' (rocessing #nit: and interface devices #sed to ma4e res#'ts avai'ab'e to ot"er
devices.
& smart camera or @inte''igent camera@ is a se'f2contained: standa'one vision system wit"
b#i't2in image sensor in t"e "o#sing of an ind#stria' video camera. %t contains a'' necessary
comm#nication interfaces: e.g. *t"ernet: as we'' as ind#stry2(roof 2!V %/O 'ines for connection
to a P6C: act#ators: re'ays or (ne#matic va'ves. %t is not necessari'y 'arger t"an an ind#stria' or
s#rvei''ance camera.
0"is arc"itect#re "as t"e advantage of a more com(act vo'#me com(ared to PC2based
vision systems and often ac"ieves 'ower cost: at t"e e,(ense of a somew"at sim('er =or missing
a'toget"er> #ser interface.
&'t"o#g" often #sed for sim('er a(('ications: modern smart cameras can riva' PCs in
terms of (rocessing (ower and f#nctiona'ities. Smart cameras "ave been mar4eted since t"e mid
8$s: b#t on'y in recent years "ave t"ey reac"ed wides(read #se: once tec"no'ogy a''owed t"eir
si1e to be red#ced w"i'e t"eir (rocessing (ower "as reac"ed severa' t"o#sand 9%PS =devices
wit" 1CD1 (rocessors and #( to 8$$$9%PS are avai'ab'e as of end of 2$$+>.
Daving a dedicated (rocessor in eac" #nit: smart cameras are es(ecia''y s#ited for
a(('ications w"ere severa' cameras m#st o(erate inde(endent'y and often async"rono#s'y: or
w"en distrib#ted vision is re/#ired =m#'ti('e ins(ection or s#rvei''ance (oints a'ong a (rod#ction
'ine or wit"in an assemb'y mac"ine>.
Fig 2.1 Early smart camera (ca. 1985, in red) with an 8MHz Z8 c!m"ared t! a m!dern de#ice
$eat%ring &e'as (nstr%ments) *+, -1.Hz
5
& smart camera #s#a''y consists of severa' =b#t not necessari'y a''> of t"e fo''owing com(onents<
%mage sensor =matri, or 'inear: CC-2 or C9OS>
%mage digiti1ation circ#itry
%mage memory
(rocessor =often a -SP or s#itab'y (owerf#' (rocessor>
(rogram2 and data memory =.&9: nonvo'ati'e 36&SD>
Comm#nication interface =.S232: *t"ernet>
%/O 'ines =often o(t iso'ated>
6ens "o'der or b#i't in 'ens =#s#a''y C: CS or 92mo#nt>
7#i't in i''#mination device =#s#a''y 6*->
P#r(ose deve'o(ed rea'2time o(erating system =3or e,am('e VC.0)
& video o#t(#t =e.g. VC& or SVC&> may be an o(tion for a Smart Camera.
2.2 Smart Cameras 's. Standard Smart .ision Systems
0"e /#estion often comes #( as to w"at is t"e most a((ro(riate a((roac" to ta4e in im('ementing
a vision system 2 #sing a smart camera or #sing some sort of PC2based a((roac". 0"ere is no
/#estion t"at as t"e micro(rocessors: -SPs and 3PC&s are getting faster and: t"erefore: more
ca(ab'e: smart cameras are getting smarter. 0"erefore: t"ey are a c"a''enge to more EEtraditiona'EE
a((roac"es to vision systems. Significant'y: "owever: EEtraditiona'EE a((roac"es are a'so ta4ing
advantage of t"e advances and so: too: are faster and smarter.
0raditiona' a((roac"es #s#a''y mean a PC2based im('ementation. 0"is co#'d be eit"er #sing a
camera wit" t"e ca(abi'ity to interface direct'y to t"e PC =%*** 139!/3ire wire: Camera 6in4:
6V-S: FS7: etc.>: or a system based on a frame grabber or ot"er inte''igent image (rocessing
board or vision engine t"at ('#gs into t"e PC. %n t"is 'atter case: more conventiona' ana'og
cameras are #sed as t"e in(#t device.
6
& smart camera: on t"e ot"er "and: is a se'f2contained #nit. %t inc'#des t"e imager as we'' as t"e
EEinte''igenceEE and re'ated %/O ca(abi'ities. 7eca#se t"is format resemb'es t"e format of many
inte''igent sensors: t"ese (rod#cts are often referred to as EEvision sensors.EE Dowever: a vision
sensor often "as a 'imited and fi,ed (erformance enve'o(e: w"i'e a smart camera "as more
f'e,ibi'ity or too's: in"erent'y ca(ab'e of being (rogrammed to "and'e many imaging a'gorit"ms
and a(('ication f#nctions. & PC2based vision system is genera''y recogni1ed as "aving t"e
greatest f'e,ibi'ity and: t"erefore: ca(ab'e of "and'ing a wider range of a(('ications.
2.3.1 1C$2ased .ision Systems Ad'anta%es
0"e PC2based vision systems advantages inc'#de<
13 0!e4ibi!ity ;
0"e PC offers greater f'e,ibi'ity in t"e n#mber of o(tions t"at can be se'ected. 3or
e,am('e one can #se a 'ine scan vers#s an area scan camera wit" t"e PC. One can #se
t"ird (arty software (ac4ages wit" t"e PC a((roac" =smart cameras tend to be sing'e
so#rce software>.
23 1o5er ;
PCEs tend to offer greater (ower and s(eed d#e in 'arge (art to t"e s(eed of t"e %nte'
(rocessors #sed interna''y. 0"is (ower in t#rn means t"at PCEs are #sed to "and'e t"e
EEto#g"erEE a(('ications in vision systems.
2.3.2 Smart Cameras Ad'anta%es
0"e smart cameras advantages inc'#de<
13 Cost 6
Smart cameras are genera''y 'ess e,(ensive to (#rc"ase and set #( t"an t"e PC based so'#tion:
since t"ey inc'#de t"e camera: 'enses: 'ig"ting =sometimes>: cab'ing and (rocessing.
23 Sim!icity 6
7
Software too's avai'ab'e wit" smart cameras are of t"e (oint2and2c'ic4 variety and are easier to
#se t"an t"ose avai'ab'e on PCEs. &'gorit"ms come (re2(ac4aged and do not need to be
deve'o(ed: t"#s ma4ing t"e smart camera /#ic4er to set#( and #se.
33 Inte%ration 6
Civen t"eir #nified (ac4aging: smart cameras are easier to integrate into t"e man#fact#ring
environment.
43 -e!iabi!ity 6
8it" fewer moving com(onents =fans: "ard drives> and 'ower tem(erat#res: smart cameras are
more re'iab'e t"an PCEs.
%n genera' t"e (erformance of t"e smart camera wi'' contin#e to increase. 0"is wi'' mean t"at t"e
smart camera wi'' be #sed for more diffic#'t a(('ications: s'ow'y dis('acing t"e PC a((roac".
0i% 2.2. Standard Smart .ision System
8
0i% 2.3. Smart Camera System.
2.4 Smart Camera Arc7itecture
0"e smart camera (resented in t"is comm#nication red#ces t"e amo#nt of data generated to t"e
data of interest by ma4ing #se of embedded image (rocessing a'gorit"ms. 0"e data of interest
mig"t be: for e,am('e: defective areas of t"e (rod#ct being ins(ected. 9#'ti('e cameras can
ro#te t"eir data to a sing'e frame grabber and com(#ter d#e to t"e red#ction of data stream: t"#s
dramatica''y red#cing system cost and increasing ins(ection bandwidt" ca(abi'ity. 0"is smart
camera a'so ma4es #se of an on2board micro(rocessor for comm#nication wit" t"e ins(ection
systems "ost com(#ter and for interna' contro' f#nctions.
0"e fo''owing b'oc4 diagram i''#strates t"e camera arc"itect#re.
0i% 2.4. Smart Camera Arc7itecture 2!oc+ 8ia%ram
& detai'ed e,('anation of t"e camera arc"itect#re fo''ows: starting wit" t"e image sensor.
9
2.4.1 Ima%e Sensor 2asics
%n t"is smart camera: a CC- =C"arge Co#('ed -evice> image sensor converts ("otons ='ig"t>
into e'ectrons =c"arge>. 8"en ("otons "it an image sensor: t"e sensor acc#m#'ates e'ectrons.
0"is is ca''ed c"arge integration. 0"e brig"ter yo#r 'ig"t so#rce: t"e more ("otons avai'ab'e for
t"e sensor to integrate: and t"e sma''er t"e amo#nt of time re/#ired to co''ect a given amo#nt of
'ig"t energy.
3ina''y: t"e sensor transfers its aggregate c"arge to reado#t registers: w"ic" feed eac" (i,e's
c"arge from t"e image sensor into an o#t(#t node t"at converts t"e c"arges into vo'tages. &fter
t"is transfer and conversion: t"e vo'tages are am('ified to become t"e cameras ana'og o#t(#t.
2.4.2 Ana!o% to 8i%ita! Con'ersion E!ectronics
0"e ana'og o#t(#t of t"e CC- is converted to a digita' o#t(#t for f#rt"er (rocessing. 0"e camera
(resented "ere s#b2divides t"e CC- ana'og o#t(#t into eig"t c"anne's of 2)+ (i,e' e'ements
eac". &na'og to digita' conversion is (erformed at a 2$ 9D1 data rate for eac" c"anne' t"#s
yie'ding an effective camera data rate of 1+$ 9D1. 0"e digita' data is t"en (assed a'ong to t"e
image (rocessing e'ectronics for (rocessing and ana'ysis.
2.4.3 Ima%e 1rocessin% E!ectronics
%mage (rocessing is (erformed by embedded a'gorit"ms on a (er c"anne' basis. 0"e fo''owing
b'oc4 diagram i''#strates t"e basic (rocessing arc"itect#re for eac" c"anne'.
10
0i% 2.5. Ima%e 1rocessin% Arc7itecture 2!oc+ 8ia%ram
0"e (rocessing a'gorit"m is embedded in t"e (rocessing P6-. 0"e micro(rocessor "as a
bidirectiona' (at" being ab'e to random'y access t"e a'gorit"m (arameters: as we'' as (rogram a
new a'gorit"m into t"e P6- as re/#ired by t"e #ser. .aw (i,e' data and associated timing and
contro' signa's are a'so connected to in(#t (ins into t"e (rocessing P6-:.
3or storage and s#bse/#ent reado#t: a'gorit"m (rocessed data is o#t(#t a'ong wit" a write contro'
signa' to 3%3O memory. 8<1 m#'ti('e,ing of t"e data is ac"ieved by #sing 3%3O memory.
.eado#t contro' is accom('is"ed by t"e micro(rocessor/3%3O reado#t contro' card w"ose
arc"itect#re is s"own in t"e fig#re 2.+.
0i% 2.6. ,icrorocessor90I0O -eadout Contro! Circuit 2oard 2!oc+ 8ia%ram
11
0"e 9icro(rocessor/3%3O reado#t contro' circ#it board acts as t"e master contro''er for t"e smart
camera. 36&SD memory is #sed to store micro(rocessor code and P6- a'gorit"m code. %n
system (rogrammabi'ity is ac"ieved beca#se a'' P6- devices in t"e image (rocessing section of
t"e camera are S.&9 based.
2.5 Ima%e 1rocessin% A!%orit7ms
9any ty(es of image (rocessing a'gorit"ms can be embedded wit"in t"e camera: since t"e video
(rocessing mod#'es are com('ete'y in2system (rogrammab'e. &s an e,am('e: a static grey sca'e
t"res"o'ding a'gorit"m is (resented be'ow.
2.5.1 Static :rey Sca!e T7res7o!din%
%n static t"res"o'ding: an #((er and a 'ower bo#nd are estab'is"ed aro#nd w"at is considered a
norma' va'#e. %mage data t"at fa''s wit"in t"e bo#ndary window is considered norma' non2
interesting data. %mage data t"at fa''s eit"er above or be'ow t"e bo#ndary window is considered
data of interest. Considering we are dea'ing wit" an 82bit digita' camera: t"e norma': #((er and
'ower bo#ndary va'#es are seen to be digita' n#mbers =-G> on a sca'e of $ to 2)) HCray sca'eI.
%magine t"at a (rod#ct is being ins(ected for defects and t"e grey sca'e 'eve' of non2defective
(rod#ct is 8) -G: and t"e #((er and 'ower bo#ndary va'#es "ave been set to J/2 1) K-G. &''
image data t"at fe'' wit"in t"e bo#nds of 7$ -G to 1$$ -G wo#'d be considered non2interesting
and wo#'d not be transmitted o#t of t"e camera.
%mage data t"at fe'' be'ow 7$ -G and above 1$$ -G wo#'d be considered interesting and wo#'d
be transmitted o#t of t"e camera. S#bstantia' data red#ction is ac"ieved since on'y some of t"e
data wi'' fa'' o#tside of t"e estab'is"ed bo#ndaries.
%t is im(ortant to note t"at a'' of t"e Ldata of interest is transmitted o#t of t"e camera and t"#s
data red#ction is ac"ieved w"ere a'' of t"e grey sca'e information is (reserved. 0"is ty(e of
a'gorit"m is i''#strated by t"e image s"own in fig#re 2.7.
12
0i% 2.7. An E4am!e o" Static T7res7o!din%
3or 'ater dis('ay and ana'ysis eac" (i,e' m#st be given an address s#c" t"at an image can be
reconstr#cted by t"e frame grabber since an a'gorit"m of t"is ty(e (rod#ces noncontin#o#s data.
0"e static t"res"o'ding a'gorit"m re/#ires t"ree (arameters as fo''ows an #((er bo#nd: a 'ower
bo#nd: and a centre va'#e. 0"e determination of t"e centre va'#e is essentia' to t"is ty(e of
a'gorit"m: and t"e acce(tab'e band between t"e #((er and 'ower bo#nd. 0"e static t"res"o'ding
a'gorit"m is e,(ressed as fo''ows<
%3 =P%M*6 C.&N %S O =C*G0*. J FPP*.>> O. =P%M*6 C.&N %S P =C*G0*. ; 6O8*.> >
0D*G
0.&GS9%0 P%M*6
*6S*
%CGO.* P%M*6
2.5.2 Embedded Ima%e 1rocessin% A!%orit7ms
0"e a'gorit"ms are embedded in "ardware wit" a P6-/microcontro''er combination and o(erate
at a 2$9D1 data rate (er c"anne'. 0"e effective (rocessing rate is !$9D1 beca#se eac" image
(rocessing P6- can (rocess two c"anne's of image data. 8it" dedicated -SP contro''ers s#c"
data (rocessing rates co#'d be diffic#'t to ac"ieve. 9icrocontro''er a'so can direct'y contro' t"e
a'gorit"m wit"o#t "ost com(#ter intervention: since it "as access to t"e image data.
13
2.6 Smart Camera System
& vision system for web ins(ection is (resented be'ow w"ere a ma,im#m of twenty 2$!8 (i,e'
"ig" sensitivity 'ine scan smart cameras are networ4ed toget"er to a sing'e "ost com(#ter and
frame grabber. & b'oc4 diagram of t"e system is s"own in fig#re 2.8
14
0i% 2.;. System 2!oc+ 8ia%ram
.
0"e system s"own consists of #( to twenty 2$!8 (i,e' "ig" sensitivity 'ine scan smart cameras
"o#sed wit"in a camera enc'os#re mo#nted above t"e web. 0ransmissive i''#mination is
(rovided since i''#mination so#rce is mo#nted beneat" t"e web. .o#ted t"ro#g" two cabinets are
t"e data: contro': and (ower 'ines to/from t"e cameras. 0"e system ma4es #se of fibre o(tic
tec"no'ogy for transmission of data and contro' signa's t"#s a''owing t"e ins(ector station to be
'ocated remote'y at a distance of #( to 1$$m.
15

C<A1TE- 3(

EMBEDDED SYSTEMS

Today embedded technology produces Intelligent,
Multitasking, Compact electronic products with networking
facilities.
3.1 Introduction to embedded systems(
&n embedded system is a s(ecia'2(#r(ose com(#ter system designed to (erform one or a few
dedicated f#nctions: often wit" rea'2time com(#ting constraints. %t is #s#a''y embedded as (art of
a com('ete device inc'#ding "ardware and mec"anica' (arts. %n contrast: a genera'2(#r(ose
16
com(#ter: s#c" as a (ersona' com(#ter: can do many different tas4s de(ending on (rogramming.
*mbedded systems contro' many of t"e common devices in #se today.
Since t"e embedded system is dedicated to s(ecific tas4s: design engineers can o(timi1e it:
red#cing t"e si1e and cost of t"e (rod#ct: or increasing t"e re'iabi'ity and (erformance. Some
embedded systems are mass2(rod#ced: benefiting from economies of sca'e.
P"ysica''y: embedded systems range from (ortab'e devices s#c" as digita' watc"es and 9P!
('ayers: to 'arge stationary insta''ations 'i4e traffic 'ig"ts: factory contro''ers: or t"e systems
contro''ing n#c'ear (ower ('ants. Com('e,ity varies from 'ow: wit" a sing'e microcontro''er
c"i(: to very "ig" wit" m#'ti('e #nits: (eri("era's and networ4s mo#nted inside a 'arge c"assis or
enc'os#re.
%n genera': @embedded system@ is not an e,act'y defined term: as many systems "ave some
e'ement of (rogrammabi'ity. 3or e,am('e: Dand"e'd com(#ters s"are some e'ements wit"
embedded systems Q s#c" as t"e o(erating systems and micro(rocessors w"ic" (ower t"em Q
b#t are not tr#'y embedded systems: beca#se t"ey a''ow different a(('ications to be 'oaded and
(eri("era's to be connected.
Embedded comutin% systems
R Com(#ting systems embedded wit"in e'ectronic devices
R Dard to define. Gear'y any com(#ting system ot"er t"an a des4to( com(#ter
R 7i''ions of #nits (rod#ced year'y: vers#s mi''ions of des4to( #nits
R Per"a(s )$ (er "o#se"o'd and (er a#tomobi'e.
17
Fig 3.1 n !mbedded "ystem
3.2 E4am!es o" embedded systems
*mbedded systems s(an a'' as(ects of modern 'ife and t"ere are many e,am('es of t"eir #se.
0e'ecomm#nications systems em('oy n#mero#s embedded systems from te'e("one switc"es for
t"e networ4 to mobi'e ("ones at t"e end2#ser. Com(#ter networ4ing #ses dedicated ro#ters and
networ4 bridges to ro#te data.
Cons#mer e'ectronics inc'#de (ersona' digita' assistants =P-&s>: m(3 ('ayers: mobi'e ("ones:
videogame conso'es: digita' cameras: -V- ('ayers: CPS receivers: and (rinters. 9any
"o#se"o'd a(('iances: s#c" as microwave ovens: was"ing mac"ines and dis"was"ers: are
inc'#ding embedded systems to (rovide f'e,ibi'ity: efficiency and feat#res. &dvanced DV&C
systems #se networ4ed t"ermostats to more acc#rate'y and efficient'y contro' tem(erat#re t"at
can c"ange by time of day and season. Dome a#tomation #ses wired2 and wire'ess2networ4ing
t"at can be #sed to contro' 'ig"ts: c'imate: sec#rity: a#dio/vis#a': s#rvei''ance: etc.: a'' of w"ic"
#se embedded devices for sensing and contro''ing.
18
0rans(ortation systems from f'ig"t to a#tomobi'es increasing'y #se embedded systems. Gew
air('anes contain advanced avionics s#c" as inertia' g#idance systems and CPS receivers t"at
a'so "ave considerab'e safety re/#irements. Vario#s e'ectric motors Q br#s"'ess -C motors:
ind#ction motors and -C motors Q are #sing e'ectric/e'ectronic motor contro''ers. &#tomobi'es:
e'ectric ve"ic'es: and "ybrid ve"ic'es are increasing'y #sing embedded systems to ma,imi1e
efficiency and red#ce (o''#tion. Ot"er a#tomotive safety systems s#c" as anti2'oc4 bra4ing
system =&7S>: *'ectronic Stabi'ity Contro' =*SC/*SP>: traction contro' =0CS> and a#tomatic
fo#r2w"ee' drive.
9edica' e/#i(ment is contin#ing to advance wit" more embedded systems for vita' signs
monitoring: e'ectronic stet"osco(es for am('ifying so#nds: and vario#s medica' imaging =P*0:
SP*C0: C0: 9.%> for non2invasive interna' ins(ections.
%n addition to common'y described embedded systems based on sma'' com(#ters: a new c'ass of
miniat#re wire'ess devices ca''ed motes are /#ic4'y gaining (o(#'arity as t"e fie'd of wire'ess
sensor networ4ing rises. 8ire'ess sensor networ4ing: 8SG: ma4es #se of miniat#ri1ation made
(ossib'e by advanced %C design to co#('e f#'' wire'ess s#bsystems to so("isticated sensor:
enab'ing (eo('e and com(anies to meas#re a myriad of t"ings in t"e ("ysica' wor'd and act on
t"is information t"ro#g" %0 monitoring and contro' systems. 0"ese motes are com('ete'y se'f
contained: and wi'' ty(ica''y r#n off a battery so#rce for many years before t"e batteries need to
be c"anged or c"arged. &n *mbedded System is a combination of Dardware and Software t"at
may "ave some mec"anica' com(onents to (erform s(ecific tas4s.
3ig 3.2 PC *nginesE &6%M.1C 9ini2%0M embedded board wit" an ,8+ &9- Ceode 6M 8$$ toget"er wit" Com(act
3'as": miniPC% and PC% s'ots: 222(in %-* interface: a#dio: FS7 and 2)+97 .&9
19
3.3 C7aracteristics o" embedded systems(
1. 8e'ice ro%rammabi!ity or mana%eab!e$
0"e f#nctioning of a s(ecifica''y designed "ardware (art on a smart device can easi'y be
c"anged by sim('y c"anging t"e software associated wit" it.
2. ,u!titas+in%$
&n embedded system em('oyed in a modern refrigerator (erforms door sense and
tem(erat#re sense at t"e same times: w"ic" are two f#nctions at t"e same time.
3. -ea! time resonse$
%t is t"e abi'ity of an embedded system to res(ond to ambient conditions s#dden'y. 0"at
is: a smart 0V ad?#sts (ict#re /#a'ity s#dden'y in res(onse to s#dden environmenta'
brig"tness variations.
4. *mbedded systems are designed to do some s(ecific tas4: rat"er t"an be a genera'2
(#r(ose com(#ter for m#'ti('e tas4s. Some a'so "ave rea'2time (erformance constraints
t"at m#st be met: for reasons s#c" as safety and #sabi'ityS ot"ers may "ave 'ow or no
(erformance re/#irements: a''owing t"e system "ardware to be sim('ified to red#ce costs.
5. *mbedded systems are not a'ways standa'one devices. 9any embedded systems consist
of sma'': com(#teri1ed (arts wit"in a 'arger device t"at serves a more genera' (#r(ose.
3or e,am('e: t"e Cibson .obot C#itar feat#res an embedded system for t#ning t"e
strings: b#t t"e overa'' (#r(ose of t"e .obot C#itar is: of co#rse: to ('ay m#sic. T2U
Simi'ar'y: an embedded system in an a#tomobi'e (rovides a s(ecific f#nction as a
s#bsystem of t"e car itse'f.
6. 0"e (rogram instr#ctions written for embedded systems are referred to as firmware: and
are stored in read2on'y memory or 3'as" memory c"i(s. 0"ey r#n wit" 'imited com(#ter
"ardware reso#rces< 'itt'e memory: sma'' or non2e,istent 4eyboard and/or screen.
20
C<A1TE- 4(

SMART CAMARAS AS EMBEDDED
SYSTEMS
#ideo camera coupled to a computer #ision system is
a tiny package of embedded systems.
$
Co working of hardware and software engineers is the
basis of !mbedded pplication %e#elopment.
21
4.1 O'er'ie5 o" Smart Cameras in embedded systems
0"e system a'so (rovides in(#t and o#t(#t to t"e ('ant contro' system. -ata from t"e cameras is
ac/#ired by t"e frame grabber: assemb'ed into images: and t"en transferred to t"e "ost com(#ter
rea' time dis('ay on t"e defect monitor: and stored to a database residing on t"e fi'e server via an
*t"ernet connection. S#bse/#ent ana'ysis of t"e data is (erformed at t"e ana'ysis wor4station
wit" ana'ysis software t"at a''ows e,traction of data from t"e database for creation of re(orts. &''
system and ana'ysis software is m#'tit"readed and (rovides rea' time data access and dis('ay.
Via a modem connection t"e system is a'so o(erab'e remote'y. 0o ens#re smoot" and constant
i''#mination of t"e web t"e system software a'so contro's t"e i''#mination so#rce wit" a f#11y
'ogic contro' sc"eme.
.ecent tec"no'ogica' advances are enab'ing a new generation of smart cameras t"at re(resent a
/#ant#m 'ea( in so("istication. 8"i'e todayEs digita' cameras ca(t#re images: smart cameras
ca(t#re "ig"2'eve' descri(tions of t"e scene and ana'y1e w"at t"ey see.
0"ese devices co#'d s#((ort a wide variety of a(('ications inc'#ding "#man and anima'
detection: s#rvei''ance: motion ana'ysis: and facia' identification.
Video (rocessing "as an insatiab'e demand for rea'2time (erformance. 3ort#nate'y: 9ooreEs 'aw
(rovides an increasing (oo' of avai'ab'e com(#ting (ower to a(('y to rea'2time ana'ysis. Smart
cameras 'everage very 'arge2sca'e integration =V6S%> to (rovide s#c" ana'ysis in a 'ow2cost: 'ow2
(ower system wit" s#bstantia' memory. 9oving we'' beyond (i,e' (rocessing and com(ression:
t"ese systems r#n a wide range of a'gorit"ms to e,tract meaning from streaming video.
7eca#se t"ey (#s" t"e design s(ace in so many dimensions: smart cameras are a 'eading edge
a(('ication for embedded system researc".
4.2 Detection and Recognition Algorithms
&'t"o#g" t"ere are many a((roac"es to rea'2time video ana'ysis: we c"ose to foc#s initia''y on
"#man gest#re recognitionQidentifying w"et"er a s#b?ect is wa'4ing: standing: waving "is arms:
and so on. 7eca#se m#c" wor4 remains to be done on t"is (rob'em: we so#g"t to design an
embedded system t"at can incor(orate f#t#re a'gorit"ms as we'' as #se t"ose we created
e,c'#sive'y for t"is a(('ication.
O#r a'gorit"ms #se bot" 'ow2'eve' and "ig"2'eve' (rocessing. 0"e 'ow2'eve' com(onent identifies
different body (arts and categori1es t"eir movement in sim('e terms. 0"e "ig"2'eve' com(onent:
22
w"ic" is a(('ication2de(endent: #ses t"is information to recogni1e eac" body (artEs action and
t"e (ersonEs overa'' activity based on scenario (arameters.
D#man detection and activity/gest#re recognition a'gorit"m "as two ma?or (arts< 6ow2'eve'
(rocessing =b'#e b'oc4s in 3ig#re !.1> and "ig"2'eve' (rocessing =green b'oc4s in 3ig#re !.1>.
Figure &.1 ' lgorithm.
A3 =o5$!e'e! rocessin%
0"e system ca(t#res images from t"e video in(#t: w"ic" can be eit"er #ncom(ressed or
com(ressed =9P*C and motion VP*C>: and a(('ies fo#r different a'gorit"ms to detect and
identify "#man body (arts.
23
(egion e)traction:
0"e first a'gorit"m transforms t"e (i,e's of an image 'i4e t"at s"own in 3ig#re !.2 a: into an 9 W
G bitma( and e'iminates t"e bac4gro#nd. %t t"en detects t"e body (artEs s4in area #sing a NFV
co'or mode' wit" c"rominance va'#es down sam('ed
0i%ure 4.2 a
Ge,t: as 3ig#re !.2 b i''#strates: t"e a'gorit"m "ierarc"ica''y segments t"e frame into s4in tone
and non2s4in2tone regions by e,tracting foregro#nd regions ad?acent to detected s4in areas and
combining t"ese segments in a meaningf#' way.
24
0i%ure 4.2 b
Contour following'
0"e ne,t ste( in t"e (rocess: s"own in 3ig#re !.2 c: invo'ves 'in4ing t"e se(arate gro#(s of
(i,e's into conto#rs t"at geometrica''y define t"e regions. 0"is a'gorit"m #ses a 3 W 3 fi'ter to
fo''ow t"e edge of t"e com(onent in any of eig"t different directions.
0i%ure 4.2 c
!llipse fitting'
0o correct for deformations in image (rocessing ca#sed by c'ot"ing: ob?ects in t"e frame: or
some body (arts b'oc4ing ot"ers: an a'gorit"m fits e''i(ses to t"e (i,e' regions as 3ig#re !.2d
s"ows to (rovide sim('ified (art attrib#tes. 0"e a'gorit"m #ses t"ese (arametric s#rface
a((ro,imations to com(#te geometric descri(tors for segments s#c" as area: com(actness
=circ#'arity>: wea4 (ers(ective invariants: and s(atia' re'ations"i(s.
25
0i%ure 4.2 d
*raph matching'
*ac" e,tracted region mode'ed wit" e''i(ses corres(onds to a node in a gra("ica' re(resentation
of t"e "#man body. & (iecewise /#adratic 7ayesian c'assifier #ses t"e e''i(ses (arameters to
com(#te feat#re vectors consisting of binary and #nary attrib#tes. %t t"en matc"es t"ese attrib#tes
to feat#re vectors of body (arts or meaningf#' combinations of (arts t"at are com(#ted off'ine.
0o e,(edite t"e branc"ing (rocess: t"e a'gorit"m begins wit" t"e face: w"ic" is genera''y easiest
to detect.
B) High-level processing
0"e "ig"2'eve' (rocessing com(onent: w"ic" can be ada(ted to different a(('ications: com(ares
t"e motion (attern of eac" body (artQdescribed as a s(atiotem(ora' se/#ence of feat#re vectors
Qin a set of frames to t"e (atterns of 4nown (ost#res and gest#res and t"en #ses severa' "idden
9ar4ov mode's in (ara''e' to eva'#ate t"e bodyEs overa'' activity.
8e #se discrete D99s t"at can generate eig"t directiona' code words t"at c"ec4 t"e #(: down:
'eft: rig"t: and circ#'ar movement of eac" body (art.
D#man actions often invo've a com('e, series of movements. 8e t"erefore combine eac" body
(artEs motion (attern wit" t"e one immediate'y fo''owing it to generate a new (attern. Fsing
dynamic (rogramming: we ca'c#'ate t"e (robabi'ities for t"e origina' and combined (atterns to
identify w"at t"e (erson is doing. Ca(s between gest#res "e'( indicate t"e beginning and end of
discrete actions.
& /#adratic 9a"a'anobis distance c'assifier combines D99 o#t(#t wit" different weig"ts to
generate reference mode's for vario#s gest#res. 3or e,am('e: a (ointing gest#re co#'d be
recogni1ed as a command to @go to t"e ne,t s'ide@ in a smart meeting room or @o(en t"e
26
window@ in a smart car: w"ereas a smart sec#rity camera mig"t inter(ret t"e gest#re as
s#s(icio#s or t"reatening.
0o "e'( com(ensate for occ'#sion and ot"er image2(rocessing (rob'ems: we #se two cameras set
at a 9$2degree ang'e to eac" ot"er to ca(t#re t"e best view of t"e face and ot"er 4ey body (arts.
8e can #se "ig"2'eve' information ac/#ired t"ro#g" one view to switc" cameras to activate t"e
recognition a'gorit"ms #sing t"e second camera. Certain actions: s#c" as t#rning to face anot"er
direction or e,ec#ting a (redefined gest#re: can a'so trigger t"e system to c"ange views Soft2
tiss#e reconstr#ction.
8e can #se 9at6ab to deve'o( o#r a'gorit"ms. 0"is tec"nica' com(#tation and vis#a'i1ation
(rogramming environment r#ns orders of magnit#de more s'ow'y t"an embedded ('atform
im('ementations: a s(eed difference t"at becomes critica' w"en (rocessing video in rea' time.
8e can t"erefore (ort o#r 9at6ab im('ementation to C code r#nning on a very 'ong instr#ction
word =V6%8> video (rocessor: w"ic" 'ets #s ma4e many arc"itect#ra' meas#rements on t"e
a(('ication and ma4e t"e necessary o(timi1ations to arc"itect a c#stom V6S% smart camera.
4.3 Requirements
&t t"e deve'o(ment stage: we can eva'#ate t"e a'gorit"ms according to acc#racy and ot"er
fami'iar standards. Dowever: an embedded system "as additiona' rea'2time re/#irements<
Frame rate'
0"e system m#st (rocess a certain amo#nt of frames (er second to (ro(er'y ana'y1e motion and
(rovide #sef#' res#'ts. 0"e a'gorit"ms we #se as we'' as t"e ('atformEs com(#tationa' (ower
determine t"e ac"ievab'e frame rate: w"ic" can be e,treme'y "ig" in some systems.
+atency'
0"e amo#nt of time it ta4es to (rod#ce a res#'t for a frame is a'so im(ortant beca#se smart
cameras wi'' 'i4e'y be #sed in c'osed2'oo( contro' systems: w"ere "ig" 'atency ma4es it diffic#'t
to initiate events in a time'y fas"ion based on action in t"e video fie'd.
9oving to an embedded ('atform a'so meant t"at we "ave to conserve memory. 6oo4ing a"ead
to "ig"'y integrated smart cameras: we want to incor(orate as 'itt'e memory in t"e system as
(ossib'e to save on bot" c"i( area and (ower cons#m(tion. Crat#ito#s #se of memory a'so often
(oints to inefficient im('ementation.
4.4 Components
27
O#r deve'o(ment strategy ca''s for 'everaging off2t"e2s"e'f com(onents to (rocess video from a
standard so#rce in rea' time: deb#g a'gorit"ms and (rograms: and connecting m#'ti('e smart
cameras in a networ4ed system. 8e #se t"e 1$$29D1 P"i'i(s 0ri9edia 09213$$ as o#r video
(rocessor. 0"is 322bit fi,ed2 and f'oating2(oint (rocessor feat#res a dedicated image
co(rocessor: a variab'e 'engt" decoder: an o(timi1ing C/CJJ com(i'er: integrated (eri("era's for
V6%8 conc#rrent rea'2time in(#t/o#t(#t: and a ric" set of a(('ication 'ibrary f#nctions inc'#ding
9P*C: motion VP*C: and 2- te,t and gra("ics.
4. !est"ed Architecture
O#r testbed arc"itect#re: s"own in 3ig#re !.3: #ses two 0ri9edia boards attac"ed to a "ost PC
for (rogramming s#((ort. *ac" PC% b#s board is connected to a Di8 camera t"at (rovides G0SC
com(osite video. Severa' boards can be ('#gged into a sing'e com(#ter for sim#'taneo#s video
o(erations. 0"e s"ared memory interface offers "ig"er (erformance t"an t"e networ4s 'i4e'y to
be #sed in V6S% cameras: b#t t"ey 'et #s f#nctiona''y im('ement and deb#g m#'ti('e2camera
systems wit" rea' video data.
28
Fig 4.3 Testbed architecture
4.# $%periments and &ptimi'ations
&s data re(resentation becomes more abstract: in(#t/o#t(#t data vo'#me decreases. 0"e c"ange
in re/#ired memory si1e: "owever: is 'ess (redictab'e given t"e com('e, re'ations"i(s t"at can
form between abstract data. 3or e,am('e: #sing si, sing'e (recision: f'oating2(oint (arameters to
describe 1$$ e''i(ses re/#ires on'y 2.! Xbytes of memory: b#t it ta4es 1$ Xbytes to store
information abo#t two ad?oining e''i(ses.
7ased on t"ese ear'y e,(eriments: we o(timi1e o#r smart camera im('ementation by a(('ying
tec"ni/#es to s(eed #( video o(erations s#c" as s#bstit#ting new a'gorit"ms better s#ited to rea'
time (rocessing and #sing 0ri9edia 'ibrary ro#tines to re('ace C'eve' code.
4.( Algorithmic changes
8e origina''y fit s#(er e''i(ses =genera'i1ed e''i(ses> to conto#r (oints: and t"is was t"e most
time2cons#ming ste(. .at"er t"an trying to o(timi1e t"e code: we decided to #se a different
a'gorit"m. 7y re('acing t"e origina' met"od deve'o(ed from (rinci(a' com(onent ana'ysis wit"
29
moment2based initia'i1ation: we red#ced t"e 6evenberg2 9ar/#ardt fitting (roced#re: t"#s
decreasing t"e e,ec#tion time.
&fter converting t"e origina' 9at'ab im('ementation into C: we (erformed some e,(eriments to
ga#ge t"e smart camera systemEs effectiveness and eva'#ate bott'enec4s. 0"e #no(timi1ed code
too4: on average: 2$.! mi''ion cyc'es to (rocess one in(#t frame: e/#a' to a rate of ) frames (er
second.
8e first meas#re t"e CPF times of eac" 'ow2'eve' (rocessing ste( to determine w"ere t"e cyc'es
were being s(ent. 9icrosoft Vis#a' CJJ is more s#itab'e for t"is (#r(ose t"an t"e 0ri9edia
com(i'er beca#se it can co''ect t"e r#nning time of eac" f#nction as we'' as its s#b f#nctionsE
times.
3ig#re !.!a s"ows t"e (rocessing2time distrib#tion of t"e fo#r body2(art2detection a'gorit"ms
3ig#re !.!b s"ows t"e memory c"aracteristics of eac" 'ow2'eve' (rocessing stage.
0i%ure 4.4 a) 4.4b
4.) Control-to-data trans*ormation
%ncreasing t"e (rocessorEs iss#e widt" can e,('oit t"e "ig" degree of (ara''e'ism t"at region
e,traction offers. Fsing a (rocessor wit" more f#nctiona' #nits co#'d t"#s red#ce (rocessing time
d#ring t"is stage. Dowever: conto#r fo''owing: w"ic" converts (i,e's to abstract forms s#c" as
30
'ines and e''i(ses: cons#mes even more time. 0"e a'gorit"m a'so o(erates seria''y< %t finds a
regionEs bo#ndary by 'oo4ing at a sma'' window of (i,e's and se/#entia''y moving aro#nd t"e
conto#rS at eac" c'oc4wise ste( it m#st eva'#ate w"ere to 'ocate t"e conto#rEs ne,t (i,e'. 8"i'e
t"is a((roac" is correct and int#itive: it (rovides 'imited %6P.
8e eva'#ate a'' (ossib'e directions in (ara''e' and combined t"e tr#e/fa'se res#'ts into a byte:
w"ic" serves as an inde, to 'oo4 #( t"e bo#ndary (i,e' in a tab'e. 8e a'so mani(#'ate t"e
a'gorit"mEs contro'2f'ow str#ct#re to f#rt"er increase %6P. 0"ese o(timi1ations do#b'e t"e
conto#r2fo''owing stageEs r#nning s(eed.
31
C<A1TE- 5(
APPLICATIONS
OF SMART CAMERAS
pplication of smart cameras takes place where
#olume, pricing or reliability constraints forbid use of bulkier
de#ices and ,C-s.
5.10ie!ds o" a!ication(
32
Smart cameras can in genera' be #sed for t"e same 4ind of a(('ications w"ere more com('e,
vision systems are #sed: and can additiona''y be a(('ied in some a(('ications w"ere vo'#me:
(ricing or re'iabi'ity constraints forbid #se of b#'4ier devices and PCEs.
0y(ica' fie'ds of a(('ication are<
Vision2based smart environments.
S#rvei''ance and trac4ing a(('ications.
&(('ications based on f#sion of vis#a' and ot"er sensory data.
9#'ti2view vision for D#man2Com(#ter %nterfaces =DC%>.
3- scene ana'ysis wit" distrib#ted sensors.
-istrib#ted m#'timedia and gaming a(('ications.
&#tomated ins(ection for /#a'ity ass#rance =detection of defects: f'aws: missing (arts...>
Gon contact meas#rements.
Part sorting and identification.
Code reading and verification =barcode: -ata 9atri,: a'("an#meric etc.>
8eb ins(ection =ins(ection of contin#o#s'y f'owing materia's s#c" as coi's: t#bes: wires:
e,tr#ded ('astic> for defect detection and dimensiona' ga#ging.
detection of (osition and rotation of (arts for robot g#idance and a#tomated (ic4ing
#nattended s#rvei''ance =detection of intr#ders: fire or smo4e detection>
biometric recognition and access contro' =face: finger(rint: iris recognition>
vis#a' sensor networ4s
-eve'o(ers can (#rc"ase smart cameras and deve'o( t"eir own (rograms for s(ecia': c#stom
made a(('ications: or t"ey can (#rc"ase ready made a(('ication software from t"e camera
man#fact#rer or from t"ird (arty so#rces. C#stom (rograms can be deve'o(ed by (rogramming
in vario#s 'ang#ages =ty(ica''y C or CJJ> or by #sing more int#itive: a'beit somew"at 'ess
f'e,ib'e: deve'o(ment too's w"ere e,isting f#nctiona'ities =often ca''ed too' or b'oc4s> can be
connected in a 'ist =a se/#ence or a bidimensiona' f'owc"art> t"at describes t"e desired f'ow of
o(erations wit"o#t any need to write (rogram code. 0"e main advantage of t"e vis#a' a((roac"
33
Vs. (rogramming is in a m#c" s"orter and somew"at easier deve'o(ment (rocess: avai'ab'e a'so
to non2(rogrammers. Ot"er deve'o(ment too's are avai'ab'e wit" re'ative'y few b#t
com(arative'y "ig" 'eve' f#nctiona'ities: w"ic" can be config#red and de('oyed wit" very
'imited effort.
Smart cameras r#nning software tai'ored for a sing'e s(ecific a(('ication are often ca''ed @vision
sensors.@
0"e term @smart camera@ for an image ca(t#re system wit" embedded com(#ting was #sed by
.eticon for one of t"eir (rod#cts: by 1977.
5.2 A!ications o" Smart Camera Net5or+s
Se'f 'oca'i1ing smart camera networ4s can serve as an enab'ing tec"no'ogy for a wide range of
"ig"er 'eve' a(('ications. Dere we foc#s on two a(('ications w"ere t"e images from t"e camera
systems are #sed to derive information abo#t t"e geometric str#ct#re of t"e environment.
5.2.1 .isua! <u!! -econstruction
9#'ti camera systems are common'y #sed to derive information abo#t t"e t"ree dimensiona'
str#ct#re of a scene. One a((roac" to t"e reconstr#ction (rob'em w"ic" is (artic#'ar'y we'' s#ited
to t"e (ro(osed se'f 'oca'i1ing smart camera networ4 is t"e met"od of vo'#me intersection w"ic"
"as been em('oyed in vario#s forms by a n#mber of researc"ers. 0"is met"od can be #sed to
detect and 'oca'i1e dynamic ob?ects moving t"ro#g" t"e fie'd of view of t"e smart camera
networ4. Dere a set of stationary cameras are #sed to observe one or more ob?ects moving
t"ro#g" t"e scene. Sim('e bac4gro#nd s#btraction is em('oyed to de'ineate t"e (ortions of t"e
images t"at corres(ond to t"e transient ob?ects. Once t"is "as been accom('is"ed one can
interrogate t"e occ#(ancy of any (oint in t"e scene: P: by (ro?ecting it into eac" of t"e images in
t#rn and determining w"et"er or not it 'ies wit"in t"e intersection of t"e swe(t regions. 0"is
(rocess can be #sed to (rod#ce an a((ro,imation for t"e 3- str#ct#re of t"e transient ob?ects by
sam('ing (oints in t"e vo'#me. 0"e res#'ts of s#c" an ana'ysis are s"own in 3ig#re).1.
34

Figure 5.1a! "ac#ground image of a scene 5.1b! $mage %ith ob&ect inserted

5.1c! 'esults of the bac#ground subtraction o(eration 5.1d! 'esults of a((lying the )olumetric
reconstruction (rocedure to the difference images deri)ed from the three smart camera nodes.
%n t"is a(('ication t"e abi'ity to ra(id'y 'oca'i1e a set of wide'y se(arated cameras is a distinct
advantage. Ot"er im('ementations of t"is reconstr#ction sc"eme invo've com('e,: time
cons#ming ca'ibration o(erations. 0"is im('ementation: in contrast: co#'d be be /#ic4'y
de('oyed in an ad2"oc manner and wo#'d a''ow a #ser to 'oca'i1e and trac4 moving ob?ects s#c"
as (eo('e: cars or anima's as t"ey move t"ro#g" t"e scene.
35
C<A1TE- 6(
CONCLUSION AND
SOME
FUTURE TRENDS
Musial fountain dances according to tunes of a music.
this is the smart world created by embedded technology.
In "uture, Smart Cameras will be able analy/e and
understand scenes they obser#e in a cooperati#e manner.

36
6.1 Otimi&ation resu!ts and Conc!usion
0"e combination of t"ese met"ods radica''y im(roves CPF (erformance for t"e a(('ication.
O(timi1ation boosts t"e (rogramEs frame rate from ) to 31 frames (er second. %n addition:
'atency decreases from abo#t 3!$ to !$2+$ mi''iseconds (er frame. 8e can add D99s and ot"er
"ig"2'eve' (rocessing (arts: and t"at ma4es t"e (rogram now r#ns at abo#t 2) frames (er second.
O#r board2'eve' system is a critica' first ste( in t"e design of a "ig"'y integrated smart camera.
&'t"o#g" t"e c#rrent system is direct'y #sef#' for some a(('ications: inc'#ding sec#rity and
medicine: a V6S% system wi'' enab'e t"e deve'o(ment of "ig"2vo'#me: embedded com(#ting
(rod#cts.
7eca#se t"e digita' (rocessors and memory #se advanced sma''2feat#re fabrication and t"e
sensor re/#ires re'ative'y 'arge (i,e's to efficient'y co''ect 'ig"t: it ma4es sense to design t"e
system as two c"i(s and "o#se t"em in a m#'tic"i( mod#'e. Se(arating t"e sensor and t"e
(rocessor a'so ma4es sense at t"e arc"itect#ra' 'eve' given t"e we'' #nderstood and sim('e
interface between t"e sensor and t"e com(#tation engine.
0"e advantages of 'everaging e,isting sensor tec"no'ogy far o#tweig" any benefits of #sing
(i,e'2('ane (rocessors #nti' t"ey become more ('entif#'. Dowever: attac"ing s(ecia'2(#r(ose
S%9- (rocessors to t"e m#'ti(rocessor can be #sef#' for bo#ndary ana'ysis and ot"er o(erations.
S#c" acce'erators can a'so save (ower: w"ic" is im(ortant given t"e cost and effort re/#ired to
de('oy m#'ti('e cameras: es(ecia''y in an o#tdoor setting. Dig"2frame2rate cameras: w"ic" are
#sef#' for a(('ications ranging from vibration ana'ysis to mac"inery design: wi'' 'i4e'y re/#ire
many s(ecia'i1ed (rocessing e'ements t"at are fast as we'' as area efficient.
%n contrast to common digita' cameras: as 4nown for e,am('e for ta4ing "o'iday (ict#res: Smart
Cameras are e/#i((ed wit" a com(#ting #nit t"at is #sed for t"e ana'ysis of ca(t#red
37
images. .ecent advances in t"e researc" fie'd of Com(#ter Vision a''ow for t"e detection of
"#man faces and gest#res and for t"e meas#rement of distances between ob?ects. %n f#t#re: Smart
Cameras wi'' be ab'e ana'yse and #nderstand scenes t"ey observe in a coo(erative manner.
-istrib#ted Smart Cameras can be connected to eac" ot"er by ad2"oc or infrastr#ct#re networ4s.
0"is -istrib#ted System does not re'y on a centra' contro' conso'e and "as t"erefore no sing'e
(oint of fai'#re 2 an im(ortant as(ect w"en safety and rob#stness are concerned.
0"is (a(er describes a sc"eme for determining t"e re'ative 'ocation and orientation of a set of
smart camera nodes and sensor mod#'es. 0"e sc"eme is we'' s#ited for im('ementation on
wire'ess sensor networ4s since t"e comm#nication and com(#tationa' re/#irements are /#ite
minima'. Se'f 'oca'i1ation is a basic ca(abi'ity on w"ic" "ig"er 'eve' a(('ications can be b#i't.
3or e,am('e: t"e sc"eme co#'d be #sed to s#rvey t"e 'ocation of ot"er sensor motes enab'ing a
range of 'ocation based sensor ana'yses s#c" as sni(er detection: c"emica' ('#me detection and
target trac4ing.
3#rt"er: t"e abi'ity to a#tomatica''y 'oca'i1e a set of smart cameras de('oyed in an ad2"oc
manner a''ows #s to a(('y a n#mber of m#'ti2camera 3- ana'ysis tec"ni/#es to recover as(ects
of t"e 3- geometry of a scene from t"e avai'ab'e imagery. F'timate'y we envision being ab'e to
constr#ct acc#rate 3- mode's of e,tended environments based on images ac/#ired by a networ4
of ine,(ensive smart camera systems.
6.2 =oo+in% A7ead
0"e vision ind#stry is ra(id'y moving away from t"e video camera/frame grabber systems of t"e
twentiet" cent#ry to a new generation of smart2camera2based systems for t"e 21
st
cent#ry. 0"ese
21st cent#ry smart2camera systems wi'' (erform rea'2time: (i,e'2data e,traction and (rocessing
o(erations wit"in t"e camera at e,treme'y "ig" s(eeds and at a cost: w"ic" is considerab'y 'ess
t"an re/#ired today for com(arab'e ca(abi'ities. *vent#a''y: com('ete vision2(rocessing2
systems2on2a2sensor2c"i( wi'' be avai'ab'e.
Com(onents in smart cameras wi'' #ndo#bted'y c"ange d#e to t"e (#s" from semicond#ctors and
new micro(rocessors coming onto t"e mar4et. 0"e trends for t"e ne,t year wi'' be towards
mega(i,e' sensors: "ig"er reso'#tion: faster (rocessing (ower: and co'o#r. & ty(ica' standard
CC- based camera "as a matri, of !8$:$$$ (i,e's: "owever t"e new mega(i,e' cameras offer at
'east 1:$$$,1:$$$ (i,e's ; or 1 mi''ion (i,e's. Some man#fact#rers a'ready offer cameras of 2
mi''ion (i,e's.
38
6.2.1 0uture A!ications
0o date: e,('oitation of smart camera tec"no'ogy "as been main'y for ind#stria' vision systems:
b#t a crossover is ?#st starting to ta4e ('ace. Smart camera tec"no'ogy wi'' begin to enter new
a(('ications: for e,am('e: in t"e sec#rity and access contro' mar4ets: in t"e a#tomotive ind#stry:
for co''ision avoidance: and even ; one day ; for t"e toy ind#stry.
*ven o#r a#tomobi'es may soon be o#tfitted wit" miniat#re eyes. 7#i't into a cr#ise contro'
system: for instance: s#c" a camera wo#'d s#dden'y a'ert t"e driver if it noted a ra(id'y
dece'erating ve"ic'e. 0"e cameras co#'d a'so ta4e t"e ('ace of t"e rear view and side2view
mirrors: t"ereby e'iminating dangero#s b'ind s(ots and 2 in t"e event of an accident ; recording
t"e seconds (rior to a co''ision.
&not"er e,am('e wo#'d be wit" inte''igent 'ifts. &n office b'oc4: wit" many 'ifts and f'oors: may
see a 'ot of (eo('e trave''ing #( and down between f'oors: (artic#'ar'y at "ig" traffic times s#c"
as ear'y morning or end of t"e wor4ing day. &t t"e moment: 'ifts are ca''ed by somebody
(ressing a b#tton and (#tting in a re/#est for t"e 'ift to sto( at a (artic#'ar f'oor. Connected wit"
smart camera tec"no'ogy: 'ifts co#'d be ro#ted on demand: wor4ing inte''igent'y: sto((ing on'y
w"en t"ere was a (re2set n#mber of (assengers waiting at a f'oor ; and missing o#t a f'oor if too
many (eo('e were waiting to meet t"e ma,im#m ca(acity of t"e 'ift.
6oo4ing into t"e f#t#re: we can foresee an infinite n#mber of a(('ications for t"e smart cameraS
in fact: as many as t"ere are (otentia' image (rocessing #ses.
-istrib#ted Smart Cameras can be #sed for a wide range of a(('ications. 0o name a few: t"e
fo''owing<
1.Assisted=i'in%)<ea!t7$Care(
O#r society is aging constant'y. 0"is 'eads to a 'arge n#mber of e'der'y (eo('e t"at need to be
cared for. 9any e'der'y move to residentia' "omes beca#se t"ey are frig"tened of accidents t"at
mig"t occ#r at "ome and 'eave t"em "e'('ess wit"o#t t"e (ossibi'ity to ca'' for "e'(. Smart
Cameras can detect accidents on t"eir own and inform eit"er re'atives or a mobi'e n#rsing
service.
2.Sur'ei!!ance o" !ar%e) sa"ety$critica! areas >airorts) train stations3(
Common s#rvei''ance systems are (rone to errors since "#man staff in centra' contro' rooms can
#s#a''y not ana'yse a'' video data ca(t#red by t"e s#rvei''ance cameras. Critica' events may (ass
#nnoticed and ca#se great "arm t"at co#'d "ave been avoided by t"e #se of Smart Cameras.
39
Smart Cameras are ab'e to detect critica' events =e.g. (ersons 'eaving s#s(icio#s '#ggage or
entering forbidden areas> on t"eir own and raise an a'arm. Gotifications abo#t s#s(icio#s
incidents can be sent direct'y from a Smart Camera to a P-& carried by sec#rity staff nearby.
3.Industry(
Common sensors #sed for a#tomation often re'y on wired: e'ectromec"anica' devices. 0"e #se of
wire'ess: contact'ess vision2based devices can "e'( to red#ce costs: since maintenance costs of
common sensors are "ig".
4.-etai!Ana!ysis(
%nternet on'ine s"o(s offer t"e (ossibi'ity to ana'y1e t"e c#stomersE be"avior in detai'. *ac" c'ic4
and 'engt" of stay on (ages can be ana'y1ed in detai' and t"ereby t"e c#stomers satisfaction and
t"e s"o(Es conversion can be increased. 3or s"o((ing ma''s and retai' stores: t"is is c#rrent'y not
(ossib'e. Smart Cameras can "e'( to ana'y1e c#stomersE be"avior in detai' w"i'e (reserving t"e
c#stomersE (rivace by not transmitting video data b#t o(erating fig#res =(eo('e co#nter: d#ration
ofstay:ro#testa4en>on'y.
0"e -%SC team "as been awarded a (ri1e for t"is innovation. 0"e (ro?ect is f#nded by t"e
german federa' ministry of economics and tec"no'ogy =*M%S0 (rogramme>. 0"e e,ce''ence2
oriented *M%S0 (rogramme 'ine @0ransfer of .esearc"@ (romotes es(ecia''y so("isticated
tec"no'ogy2based b#siness start2#( (ro?ects in t"e (re2start2#( ("ase.
Y2Y2Y
40
2I2I=IO:-A1<?
T1U. 8ayne 8o'f: 7#ra4 O1er: 0ie"an 6v: HSmart Cameras as Dig"2 Performance *mbedded
SystemsI.
T2U. 7#ra4 O1er: 8ayne 8o'f: @& Dierarc"ica' D#man -etection System in Com(ressed and
Fncom(ressed -omainsI.
T3U. 0ie"an 6v: 7#ra4 O1er: 8ayne 8o'f: @Smart Camera System -esign:@ %nvited Pa(er:
%nternationa' Pac4et Video 8or4s"o(: Pittsb#rg": &(ri' 2$$2.
T!U &dvanced %maging *#ro(e 9aga1ine< 0"e inte''igent camera: October =2$$2> 12 ; 1+
T)U 8intriss *ngineering Cor(oration< Near 2$$2 Smart Cameras
T+U Smart Cameras vs. PC2based 9ac"ine Vision Systems: "tt(<//www.coreco.com/: =2$$2>
T7U 6ongbottom: -.< 6atest -eve'o(ments in Sensor and Camera 0ec"no'ogy: &'rad %nstr#ments
6td.: 8"ite (a(er: "tt(<//www.#4iva.org/%PO09V$26atest.(df: =2$$2>
T8U Smart Cameras 2 & com('ete vision system in a camera body: Vision Com(onents:
"tt(<//www.is.ir'.cri.n1/(rod#cts/smartcam."tm'
T9U 6e"ots4y: -. &.< %nte''igent Dig" Sensitivity CC- 6ine Scan Camera wit" embedded %mage
Processing a'gorit"ms: -&6S& %GC: =2$$2>
T1$U Siemens .5%< 0"e Camera t"at Crew a 7rain:"tt(<//w!.siemens.de/3#%/en/arc"iv/1eitsc"rift/
41
"eft1Z99/arti4e'$2/
T11U "tt(<//en.wi4i(edia.org/wi4i/SmartZcamera
T12U "tt(<//www.icdsc.org/
T13U"tt(<//boo4s.goog'e.co.in/boo4sA
s(e''[15/[SmartJCamerasJasJDig"JPerformanceJ*mbeddedJSystems5btnC[Searc"J7oo4s
[14] Cami''o V. 0ay'or -e(artment of Com(#ter and %nformation Science Fniversity of
Pennsy'vania c?tay'or\cis.#(enn.ed#
T1)U 7aba4 S"irmo"ammadi -e(artment of Com(#ter and %nformation Science Fniversity of
Pennsy'vania baba4s\gras(.cis.#(enn.ed#
T1+U "tt(<//www.sra.#ni2"annover.de/forsc"#ng/(ro?e4te/a4t#e''e2(ro?e4te/distrib#ted2smart2
cameras2disc."tm'
T17U "tt(<//www.scribd.com/doc/3+7)+12/*97*--*-2SNS0*9S2
2222222222222[[[[[[[2222222222222222222[[[[[[[[222222222222222222[[[[[[[[22222222222222222222222
42