&ROODERUDWLQJRQ0RELOH$SS'HVLJQ

WKURXJK3DLU3URJUDPPLQJ
$3UDFWLFH2ULHQWHG$SSURDFK2YHUYLHZDQG([SHUW5HYLHZ

0RKDPPHG6H\DP 6FRWW0F&ULFNDUG
'HSDUWPHQWRI&RPSXWHU6FLHQFH 'HSDUWPHQWRI&RPSXWHU6FLHQFH
9LUJLQLD7HFK 9LUJLQLD7HFK
%ODFNVEXUJ9$ %ODFNVEXUJ9$
VH\DP#YWHGX PFFULFNV#FVYWHGX


Abstract,Q PDQ\RIWRGD\VVPDOOFRPSDQLHVDQGVWDUWXSV
developers may not give enough attention to the importance of
UX/UI design for the product under development. Moreover,
such small teams may lack the required organization skills that
help them to collaborate and work together. As a result, such
teamsusuallyfacepUREOHPVZLWKGHOLYHULQJWKHULJKWSURGXFW
as well as not being able to follow a sustainable development
process.Inasimilarcontext,CSstudentsinprogrammingclasses
arefacingalmostthesameproblemsofsmallteamsinindustry.
To tackle these problems, various approaches for integrating
agile development methods and UX design methods have been
proposed to help developers carefully consider the UX
requirements,andtobeabletoorganizetheirworkenvironment.
Inthispaper,weexplorehowsuchintegrationcanbeagoodfit
forbothCSstudentsandsoftwaredevelopers,especiallyforthose
whowork on mobiledevelopment. We present ourproposed set
of integration guidelines, and then we focus on Pair
WKH QRQIXQFWLRQDO UHTXLUHPHQWV VSHFLDOO\ 8, UHTXLUHPHQWV
DQGWKH\ QHHG WR IROORZ D IOH[LEOHGHYHORSPHQW DSSURDFK WKDW
KHOSV WKHP VWD\ IRFXVHG ZLWK PLQRU SODQQLQJ DQG ZLWKRXW
KLQGHULQJWKHLUSURJUHVVWRZDUGVGHOLYHULQJZRUNLQJVRIWZDUH
%\GHILQLWLRQDJLOHPHWKRGRORJLHVDUHFRQFHUQHGZLWKKRZ
WKHVRIWZDUHVKRXOGEHGHYHORSHGZKLOH8,GHVLJQIRFXVHVRQ
KRZ WKH HQG XVHUV ZLOO ZRUN ZLWK WKH VRIWZDUH $OWKRXJK WKH
WZR DVSHFWV DUH IROORZLQJ GLIIHUHQW JXLGHOLQHV IRU HDFK WR
DFKLHYH WKHLU JRDOV LQWHJUDWLQJ DJLOH DQG 8, GHVLJQ SUDFWLFHV
VHHP WR EH D SURPLVLQJ FRPELQDWLRQ WKDW FDQ KHOS VRIWZDUH
GHYHORSHUV WR EHWWHU KDQGOH WKH 8, UHTXLUHPHQWV DV ZHOO DV WR
IROORZ D VHPLVWUXFWXUHG GHYHORSPHQW DSSURDFK WR KHOS WKHP
PDQDJHWKHLUSURJUDPPLQJZRUN
6LQFH FROODERUDWLYH GHYHORSPHQW HQYLURQPHQWV DUH YHU\
LPSRUWDQWWRVXFFHVVRIDQ\DJLOHWHDPLQWKLVSDSHUZHH[SORUH
Programming (PP) as an agile practice that promotes a
collaborative work environment. The expert reviews conducted
in this paper helped us exlore how PP can be introduced to CS
students (and later, to developers in the market) to support
collaborative work environments. Moreover, Using PP in class
provides an adaptive and collaborative teaching approach that
canbeusedinCSprogramminglabs.Wealsodiscussthepitfalls
that can affect developers during PP sessions, and how to avoid
such negative effects. We conclude by providing a set of
recommended practices that can be applied to introduce PP for
developersinbothacademicandindustrialcontexts.
KeywordsFrameworks for collaboration collaborative
software development collaboration in education agile methods
pair programming UX design
, ,1752'8&7,21
6RIWZDUH GHYHORSHUV LQ VPDOO WHDPV DQG VWDUWXSV DQG &6
VWXGHQWV LQSURJUDPPLQJ FODVVHV EDVLFDOO\ VKDUH WZR FRPPRQ
KRZ 3DLU 3URJUDPPLQJ 33 FDQ KHOS GHYHORSHUV FROODERUDWH
HIIHFWLYHO\ RQ WKHLU GHYHORSPHQW SURFHVVHV 2QH RI RXU
UHVHDUFK REMHFWLYHV LV WR VWXG\ KRZ D FROODERUDWLYH ZRUN
HQYLURQPHQW FDQ KHOS GHYHORSHUV EXLOG EHWWHU DQG XVDEOH
LQWHUIDFHV+RZHYHUIRUWKLVVSHFLILFSDSHUZH IRFXVRQKRZ
ZH FDQ SURPRWH D PRUH FROODERUDWLYH ZRUN HQYLURQPHQW
WKURXJKDQDJLOHSUDFWLFHVXFKDV33
,, 5(/$7,216+,3%(7:((1$*,/,7<$1'86$%,/,7<
$JLOH PHWKRGRORJLHV KDYH DSSHDUHG WR GHDO ZLWK WKH QHZ
SUREOHPVWKDWEHJDQWRHYROYHZLWKWKHQHZHUDRIZHE DQG
WKHQ PRELOH  DSSOLFDWLRQV )RU D ZKLOH WKHVH ZHUH NQRZQ DV
OLJKWZHLJKW PHWKRGRORJLHV EXW WKHQ WKH WHUP $JLOH
PHWKRGRORJLHV DSSHDUHG WR GHVFULEH WKLV JURXS RI QHZ
PHWKRGRORJLHV 7KHVH PHWKRGRORJLHV DWWHPSW D XVHIXO
FRPSURPLVH EHWZHHQ QR SURFHVV DQG WRR PXFK SURFHVV
SURYLGLQJMXVWHQRXJKSURFHVVWRDFKLHYHUHDVRQDEOHUHVXOWV
SUREOHPVWKH\DUHXVXDOO\FRQFHUQHGZLWKKRZWREXLOGWKH $JLOH PHWKRGRORJLHV KDYH DWWUDFWHG D ORW RI DWWHQWLRQ WKH
UHTXLUHG SURGXFW WKH ODE DVVLJQPHQW RU WKH FRXUVH SURMHFW LQ PDLQ UHDVRQ ZDV WKDW WKH\ VHHN WR FXW RXW LQHIILFLHQF\
WKHFDVHRIVWXGHQWVZLWKOHVVIRFXVRQKRZWKH\DSSURDFKWKH EXUHDXFUDF\ DQG DQ\WKLQJ WKDW DGGV QR YDOXH WR D VRIWZDUH
DFWXDO FXVWRPHU QRQIXQFWLRQDO UHTXLUHPHQWV WKDW DUH PRVWO\ SURGXFW3URSRQHQWVRIDJLOHPHWKRGRORJLHVRIWHQVHHVRIWZDUH
8VHU ,QWHUIDFH 8, UHTXLUHPHQWV DQG  WKH\ GRQW JLYH
HQRXJK WLPH WR GHFLGH KRZ WKH\ DUH JRLQJ WR ZRUN RQ WKHLU
SURMHFW DQG WKH\ XVXDOO\ ZRUN ZLWK XQSODQQHG DSSURDFK ZLWK
WKH PDLQ JRDO RI GHOLYHULQJ D ZRUNLQJ SLHFH RI VRIWZDUH 7R
WDFNOHWKHVHSUREOHPVGHYHORSHUVQHHGWRSD\PRUHDWWHQWLRQWR
978-1-4673-7648-8/15/$31.00 2015 IEEE
VSHFLILFDWLRQDQGGRFXPHQWDWLRQDVDGGLQJQRRUPLQLPXP
YDOXH >@ $JLOH PHWKRGRORJLHV DUH OHVV GRFXPHQWRULHQWHG
XVXDOO\HPSKDVL]LQJD VPDOOHUDPRXQWRIGRFXPHQWDWLRQIRUD
JLYHQ WDVN DQG VWURQJO\ DGYRFDWH IRU KXPDQ FRPPXQLFDWLRQ
DQG FROODERUDWLRQ RYHU GHILQHG DQG UHSHDWDEOH DFWLYLWLHV DV
PHFKDQLVPVIRUGHYHORSLQJTXDOLW\VRIWZDUH>@
124
 :KDWLVLPSRUWDQWDERXWDJLOHPHWKRGRORJLHVLVQRWRQO\WKH
SUDFWLFHV WKH\ XVH EXW DOVR WKHLU UHFRJQLWLRQ RI SHRSOH DV WKH
SULPDU\ GULYHUV RI SURMHFW VXFFHVV FRXSOHG ZLWK DQ LQWHQVH
IRFXV RQ HIIHFWLYHQHVV 7KH\ VWUHVV WZR FRQFHSWV WKH
XQIRUJLYLQJ KRQHVW\ RI ZRUNLQJ FRGH DQGWKH HIIHFWLYHQHVV RI
SHRSOHZLOOLQJWRZRUNWRJHWKHU>@
IURQWGHVLJQDQGIRFXVLQVWHDGRQGHOLYHULQJZRUNLQJ
VRIWZDUHHDUO\
 2UJDQL]DWLRQDO FKDOOHQJHV 7KH DJLOH FRPPXQLW\
IROORZVDKLJKO\FROODERUDWLYHRUJDQL]DWLRQDOVWUDWHJ\
ZKHUH WHDPV DUH VHOIRUJDQL]LQJ ZKLFK LV QRW WKH
FRPPRQFDVHZLWK8;WHDPV+RGJHWWVLQ>@VKRZV
8VDELOLW\(QJLQHHULQJ8(DQG8VHUH;SHULHQFH8;DUH
FXUUHQWO\ FRQVLGHUHG LPSRUWDQW FRQFHSWV WR WKH VRIWZDUH
PDLQVWUHDP >@ 8( GHDOV ZLWK LVVXHV VXFK DV V\VWHP
OHDUQDELOLW\ HIILFLHQF\ PHPRUDELOLW\ HUURUV DQG XVHU
VDWLVIDFWLRQ>@8;DQGLWVUHODWHGDVSHFWVDUHEHLQJFRQVLGHUHG
E\ VRIWZDUH HQJLQHHUV DQG UHVHDUFKHUV EHFDXVH RI WKH KXJH
GHPDQG RQ WKH ZHE PRELOH DQG LQWHUQHW DSSOLFDWLRQV LQ
WKDWDFHQWHUIRU8;ZLWKDVWURQJRUJDQL]DWLRQDODQG
PDQDJHPHQWKLHUDUFK\FDQEHSUREOHPDWLFDVRSSRVHG
WR WKH IOXLG RUJDQL]DWLRQDO VWUXFWXUH WKDW DJLOH WHDPV
KDYH
 3URFHVVLPSHGDQFHPLVPDWFK7KHDJLOHFRPPXQLW\LV
DOZD\VDJDLQVWHDUO\GHWDLOHGGHVLJQVZKLFKWKH\UHIHU
JHQHUDO 7KH 8( DQG 8; SURFHVVHV IRFXV RQ GHYHORSLQJ
V\VWHPVWKDWDUHDGDSWHGIRUHQGXVHUV
&KDPEHUODLQDQG6KDUSLQ>@DUJXHWKDWDOWKRXJKWKHUHDUH
VLPLODULWLHV EHWZHHQ DJLOH GHYHORSPHQW SULQFLSOHV DQG 8;
GHVLJQJXLGHOLQHVWKHUHDUHDOVRGLIIHUHQFHVWKDWPDNHLWKDUGWR
FRPELQH ERWK LQ D VLQJOH SURMHFW 7KH WKUHH PDLQ VLPLODULWLHV
DUH
 %RWK RI WKHP UHO\ RQ DQ LWHUDWLYH GHYHORSPHQW
WR DV %LJ'HVLJQ8S)URQW %'8) 2Q WKH RWKHU
KDQG PDQ\ ZLWKLQ WKH 8; FRPPXQLW\ SUHIHU PRUH
FRPSUHKHQVLYH PRGHOOLQJ HDUO\ LQ WKH SURMHFW WR
GHVLJQ WKH XVHU LQWHUIDFH SURSHUO\ EHIRUH DFWXDO
GHYHORSPHQWEHJLQV
7KHVH GLIIHUHQFHV EDVLFDOO\ H[LVW EHFDXVH RI WKH IDFW WKDW
WKRVH ZKR LQYHQWHG DJLOH PHWKRGRORJLHV DUH PDLQO\
SURJUDPPHUV ZKR IRFXV RQ EXLOGLQJ ZRUNLQJ VRIWZDUH UDWKHU
WKDQXVDEOHVRIWZDUH7KH\GHDOZLWKZKDWXVHUVZDQWLHZKDW
SURFHVV EXLOGLQJ RQ HPSLULFDO LQIRUPDWLRQ IURP XVHUV VD\ WKH\ ZDQW UDWKHU WKDQ ZKDW XVHUV QHHG LH ZKDW
SUHYLRXVF\FOHVRUURXQGV
 7KH\ERWKSODFHDQHPSKDVLVRQWKHXVHUHQFRXUDJLQJ
SDUWLFLSDWLRQWKURXJKRXWWKHGHYHORSPHQWSURFHVV
 %RWK DSSURDFKHV HPSKDVL]H WKH LPSRUWDQFH RI WHDP
FRKHUHQFH
2Q WKH RWKHU KDQG WKH PDLQ GLIIHUHQFHV EHWZHHQ WKH WZR
DSSURDFKHVDFFRUGLQJWR>@DUH
 8; DGYRFDWHV UHTXLUH FHUWDLQ GHVLJQ SURGXFWV WR
VXSSRUW FRPPXQLFDWLRQ ZLWK GHYHORSHUV ZKLOH DJLOH
PHWKRGVVHHNPLQLPDOGRFXPHQWDWLRQ
 8; HQFRXUDJHVWKHWHDPWRXQGHUVWDQGWKHLUXVHUVDV
PXFK DV SRVVLEOH EHIRUH WKH SURGXFW EXLOG EHJLQV
ZKHUHDVDJLOHPHWKRGVDUHODUJHO\DJDLQVWDQXSIURQW
SHULRGRILQYHVWLJDWLRQDWWKHH[SHQVHRIZULWLQJFRGH
,Q >@ $PEOHU VKRZHG ZKDW KH FDOOHG FKDOOHQJHV RI
FRPELQLQJ DJLOLW\ ZLWK 8; GHVLJQ )RXU RI $PEOHUV PDLQ
FKDOOHQJHVDUH
 'LIIHUHQW JRDOV EDVHG RQ /HHV ZRUN LQ >@ $PEOHU
DUJXHV WKDW DJLOH PHWKRGV IRFXV RQ WKH GHVLJQ
LPSOHPHQWDWLRQ DQG PDLQWHQDQFH RI VRIWZDUH
XVHUVZLOODFWXDOO\XVHWKHVRIWZDUHIRU
7KHVLPLODULWLHVEHWZHHQDJLOHGHYHORSPHQWJXLGHOLQHVDQG
8; GHVLJQ SULQFLSOHV WRJHWKHU ZLWK WKH DGYDQWDJHV RI
IROORZLQJ WKH WZR IDPLOLHV RI PHWKRGV KDYH HQFRXUDJHG
UHVHDUFKHUVDQGSUDFWLWLRQHUVWRILQGZD\VWRFRPELQH WKHPLQ
WKHLUGHYHORSPHQWSURMHFWV+RZHYHUWKHGLIIHUHQFHVWKDWZHUH
SUHVHQWHGLQWKHVDPHVHFWLRQKDYHDOZD\VEHHQFKDOOHQJLQJIRU
WKRVHZKRZDQWWRJRIRUVXFKFRPELQDWLRQ6RPHH[DPSOHVRI
VXFKLQWHJUDWLRQDSSURDFKHVDUHSUHVHQWHGLQ>@>@>@>@
>@DQG>@
,,, 5(&200(1'$7,216)25,17(*5$7,1*$*,/,7<$1'
86$%,/,7<
%DVHGRQRXUOLWHUDWXUHUHYLHZZHIRXQGWKDWWKHVXJJHVWHG
LQWHJUDWLRQ DSSURDFKHV PD\ QRW ILW WKH PDLQ WDUJHWV RI RXU
UHVHDUFKZKLFKDUHGHYHORSHUVRIVPDOOWHDPVDQG&6VWXGHQWV
7KHUHIRUH ZH SURFHHGHG E\  H[SORULQJ KRZ WR LQWHJUDWHDJLOH
PHWKRGVZLWK8;GHVLJQSULQFLSOHVLQDOLJKWSUDFWLFHRULHQWHG
DSSURDFKWKDWFDQEHDFFHSWHGGLJHVWHGDQGHDVLO\DSSOLHGE\
GHYHORSHUV
:H VWDUWHG ZRUNLQJ RQ WKLV E\ VXUYH\LQJ VRPH RI WKH
VXJJHVWHGLQWHJUDWLRQ DSSURDFKHV H[DPSOHVZHUHJLYHQLQWKH
SUHYLRXV VHFWLRQ :H DOVR KDG D FDVH VWXG\ ZKHUH SUDFWLFHV
V\VWHPV ZKLOHRYHUORRNLQJWKHGHVLJQRIWKHKXPDQ
IURP D PRUH JHQHUDO IUDPHZRUN WKH 7UDJLOH )UDPHZRUN
FRPSXWHULQWHUIDFHVWKURXJKZKLFKWKRVHV\VWHPVDUH
XVHG 2Q WKH RWKHU KDQG 8; GHVLJQHUV IRFXV RQ
GHYHORSLQJ V\VWHPV VR HQGXVHUV FDQ XVH WKHP
HIIHFWLYHO\ EXW GR QRW DFFRXQW IRU WKH XQGHUO\LQJ
V\VWHP GHVLJQ LPSOHPHQWDWLRQ RU PDUNHWGULYHQ
IRUFHV WKDW DUH RI PRVW LPSRUWDQFH WR VRIWZDUH
HQJLQHHULQJ
 'LIIHUHQW DSSURDFKHV 7KH 8; PHWKRGV WU\ WR JHW D
KROLVWLF YLHZ RI XVHU QHHGV DQG FRPH XS ZLWK DQ
RYHUDOO SODQ IRU WKH XVHU LQWHUIDFH EHIRUH VWDUWLQJ
LPSOHPHQWDWLRQ ZKLOH DJLOH PHWKRGV IDYRU OLWWOH XS
125
>@ KDYH EHHQ DSSOLHG LQ D VPDOO SURMHFW WR GHPRQVWUDWH WKH
DSSOLFDELOLW\ RI FRPELQLQJ SUDFWLFHV IURP ERWK DJLOH DQG 8;
DUHDV %DVHG RQ RXU OLWHUDWXUH UHYLHZ DQG FRQVLGHULQJ WKH
LPSOHPHQWDWLRQ RI VRPH SUDFWLFHV RQ D FDVH VWXG\ ZH FDQ
VXPPDUL]H RXU UHFRPPHQGDWLRQV IRU DSSO\LQJ 8; SULQFLSOHV
ZLWKLQ DJLOH HQYLURQPHQWV IRU VPDOO WHDPV LQ WKH IROORZLQJ
SRLQWV
 $OWKRXJK DJLOH PHWKRGV DLP DW VDWLVI\LQJ FXVWRPHUV
WKH\ QHHG H[SOLFLW SUDFWLFHV WR VDWLVI\ 8;8,
UHTXLUHPHQWV
 6DWLVI\LQJ FXVWRPHUV KDV DOZD\V EHHQ FODLPHG WREH WKH ,9 72:$5'6$&2//$%25$7,9(35$&7,&(25,(17('
PDLQ REMHFWLYH RI DJLOH PHWKRGV 7KHUHIRUH PRVW RI WKH ,17(*5$7,21$3352$&+
SURSRVHG DJLOH SUDFWLFHV WHQG WR IRFXV RQ UDSLGO\ SURYLGLQJ
%DVHGRQWKHDERYHUHFRPPHQGDWLRQVZHGHFLGHGWRIRFXV
KLJKTXDOLW\VRIWZDUHWKDWDFKLHYHVXVHUJRDOV+RZHYHUVXFK
RQ VWXG\LQJ VSHFLILF SUDFWLFHV WKDW DUH ERWK DJLOH DQG 8;
JRDOVDUH XVXDOO\UHODWHGWRV\VWHPV IXQFWLRQDOLW\JLYLQJOHVV
RULHQWHG 2QH RI VXFK SUDFWLFHV LV 3DLU 3URJUDPPLQJ 33
FRQVLGHUDWLRQ IRU WKH QRQIXQFWLRQDO UHTXLUHPHQWV +HQFH
ZKLFK LV DQROG WHFKQLTXH WKDW JRHVEDFN WRWKH PLGV DV
SURYLGLQJ 8;VSHFLILF SUDFWLFHV HQVXUHV WKDW WKLV FDWHJRU\ RI
VKRZQ LQ >@ +RZHYHU33 KDVQW JDLQHG WKH ,7FRPPXQLW\
UHTXLUHPHQWV LH WKH LQWHUIDFH UHTXLUHPHQWV ZLOO KDYH WKH
DWWHQWLRQXQWLOLWVUHYLYDOE\.HQW%HFNZKHQKHLQWURGXFHGKLV
VDPHSULRULW\DVWKHUHJXODUIXQFWLRQDOUHTXLUHPHQWVZKLOHVWLOO
PDLQWDLQLQJ WKH DJLOLW\ RI WKH VRIWZDUH GHYHORSPHQW SURFHVV
DQGWKHTXDOLW\RIWKHSURGXFHGVRIWZDUH
 ,WV XVXDOO\ EHWWHU DQG HDVLHU IRU VPDOO WHDPV WR EH
JLYHQ SUDFWLFHV WR DSSO\ UDWKHU WKDQ JXLGHOLQHV WR
IROORZ
$OWKRXJK DJLOH PHWKRGV DUH DERXW IOH[LELOLW\ DQG
DGDSWDELOLW\ WKH\ VWLOO QHHG WR EH PDQDJHG DQG FRQWUROOHG
HVSHFLDOO\IRUWKHOHVVH[SHULHQFHGWHDPV+LJKOHYHOJXLGHOLQHV
DUHXVHIXODVLQWURGXFWRU\LGHDVWRWKHDJLOHWKLQNLQJDQGWKH\
JLYHSURMHFWPDQDJHUVDQGH[SHULHQFHGGHYHORSHUVWKHVSDFHWR
LQQRYDWH E\ DSSO\LQJ VXFK JXLGHOLQHV RQ WKHLU RZQ ZD\V
+RZHYHU JXLGHOLQHV FDQ EH PLVLQWHUSUHWHG RU PLVDSSOLHG E\
OHVVH[SHULHQFHG WHDP PHPEHUV ZKLFK FDQ OHDG WR PDMRU
GHYHORSPHQW SUREOHPV 3URYLGLQJ GHWDLOHG SUDFWLFHV
HQFRXUDJHV WHDP PHPEHUV WR FROODERUDWH UHJDUGOHVV WKHLU
H[SHULHQFH OHYHO DQG DYRLGV WKH SUREOHPV DVVRFLDWHG ZLWK
H;WUHPH3URJUDPPLQJ;3PHWKRGRORJ\>@
33 WRJHWKHU ZLWK 6&580 GDLO\ PHHWLQJV >@ KDYH
DOZD\V EHHQ DUJXHG WR EH WKH PRVW HIIHFWLYH SUDFWLFHV WKDW
VXSSRUW WKH DJLOH PDQLIHVWRV ILUVW SULQFLSOH ZKLFK YDOXHV
,QGLYLGXDOV DQG LQWHUDFWLRQV RYHU SURFHVVHV DQG WRROV >@
33 LV DOVR FRQVLGHUHG WR EH D W\SLFDO DSSOLFDWLRQ IRU WKH VL[WK
SULQFLSOH RI WKH WZHOYH DJLOH SULQFLSOHV ZKLFK VWDWHV 7KH
PRVW HIILFLHQW DQG HIIHFWLYH PHWKRG RIFRQYH\LQJ LQIRUPDWLRQ
WRDQGZLWKLQDGHYHORSPHQWWHDPLVIDFHWRIDFHFRQYHUVDWLRQ
>@7KHUHIRUH33 VHHPV WREH D UHDVRQDEOH FKRLFH IRU XV WR
EHJLQH[SORULQJDVDSUDFWLFHWRVXSSRUWFROODERUDWLYHVRIWZDUH
GHYHORSPHQW ZKLFK ZLOOKHOSXVUHDFKRXU REMHFWLYHRIEHWWHU
WHDPRUJDQL]DWLRQDQGHQKDQFHGGHYHORSPHQWSURFHVV
33KDVDOZD\VEHHQVWXGLHGDVDQDJLOHSUDFWLFHJLYLQJOHVV
FRQVLGHUDWLRQ RQ KRZ LW FRXOG DIIHFW 8, GHVLJQ )RU H[DPSOH
>@ SUHVHQWHG D VWXG\ RI XVLQJ 33 IRU WHDFKLQJ +XPDQ
&RPSXWHU,QWHUDFWLRQ+&,FODVVEXWWKHLUVWXG\GLGQWVKRZ
GRXEWLQJWKHFRUUHFWPHDQLQJRISDUWLFXODUJXLGHOLQHV
WKHLPSDFWRIDSSO\LQJ33RQWKH8,VGHYHORSHGLQVXFKFODVV
 7KH VLPSOHU OLJKWHU WKH SUDFWLFHV DUH WKH HDVLHU IRU
GHYHORSHUVWRDSSO\
:LWK OHVVH[SHULHQFH GHYHORSHUV RU GHYHORSHUV ZKR DUH
QHZ WR WKH DJLOH ZRUOG LWG EH LPSRUWDQW WR LQWURGXFH DJLOH
PHWKRGV IRU WKHP WKURXJK VWUDLJKWIRUZDUG HDV\WRDSSO\
SUDFWLFHV UDWKHU WKDQ FRPSOLFDWHG RQHV %RWK DJLOH DQG 8;
SUDFWLFHV KDYH GLIIHUHQW OHYHOV RI FRPSOH[LW\ FRQVLGHULQJ
LQWURGXFLQJ DQG DSSO\LQJ WKHP 7KH FDVH VWXG\ VKRZHG WKDW
GHYHORSHUV QRW RQO\ OHDUQHG WKH VLPSOH SUDFWLFH IDVWHU DQG
7KHUHIRUHZHDUHH[WHQGLQJRXUZRUNWRVWXG\KRZ33FDQEH
XVHGWRGHVLJQEHWWHULQWHUIDFHVWKURXJKFRQVLGHULQJ8,GHVLJQ
SULQFLSOHVWRJHWKHUZLWKUHJXODUDJLOHUHTXLUHPHQWV
8QOLNH WKH SUHYLRXV VWXGLHV WKDW DSSOLHG DJLOH PHWKRGV LQ
FODVVURRPVZHDUHLQWHUHVWHGZLWKWKH8;UHTXLUHPHQWVDVZHOO
DV WKH UHJXODU GHYHORSPHQW WDVNV 0RUHRYHU ZH DUH IRFXVLQJ
RQ DSSO\LQJ FHUWDLQ SUDFWLFHV WKDW ZH EHOLHYH WR EH HDVLHU WR
IROORZWKDQWKH UHJXODUDJLOHJXLGHOLQHV )LJFRQFHSWXDOL]HV
RXU LGHD RI LQWHJUDWLQJ DJLOH SUDFWLFHV ZLWK 8; GHVLJQ
HDVLHUZKLFKLVH[SHFWHGEXWWKH\DOVRJRWWKHVHQVHRIDJLOH
WKLQNLQJ WKURXJK VXFK VLPSOH SUDFWLFHV DQG WKHQ WKH\ JDLQHG
WKH UHTXLUHG OHYHO RI VHOIFRQILGHQFH IRU WKHP WR GHOYH LQ
KHDYLHU SUDFWLFHV WKDW QHHGGHHS XQGHUVWDQGLQJ RI DJLOH DQG
8;FRQFHSWV
 6RPH SUDFWLFHVRI DJLOHGHYHORSPHQW DQG 8; GHVLJQ
DUHFRPPRQVHQVHKRZHYHUWKH\QHHGWREHSODQQHG
JXLGHGDQGPDGHVXUHWREHDSSOLHG
SULQFLSOHV IRU PRELOH DSS GHYHORSPHQW LQ &6 FODVVHV 2XU
SURSRVHG SUDFWLFHV DUH UHODWHG WR KRZ WR LQWURGXFH 33 WR
VWXGHQWV DFWLYLWLHV WKDW HQVXUH WKDW VWXGHQWV IXOO\ FRPSUHKHQG
WKH QHZ HQYLURQPHQW SUDFWLFHV WR JUDGXDOO\ DSSO\ 33 LQ ERWK
LQFODVV DQG WDNHKRPH DVVLJQPHQWV DQG VSHFLILF SUDFWLFHV WR
HQIRUFHDQLQFODVVDJLOHHQYLURQPHQWUDWKHUWKDQWKHWUDGLWLRQDO
FODVVVHWWLQJV
%\ H[SORULQJ33DV D FROODERUDWLRQSURPRWLQJ SUDFWLFH IRU
VRIWZDUH GHYHORSHUV HLWKHU RQ FRPSDQLHV RU &6 FODVVHV ZH
%DVHG RQ WKH GHYHORSHUV IHHGEDFN IURP WKH FDVH VWXG\ DLPDWDFKLHYLQJWKUHHPDLQJRDOV
WKH\ZHUHVXUSULVHGWKDWVRPHSUDFWLFHVWKDWWKH\GRIURPWLPH
WR WLPH DUH DFWXDOO\ UHFRPPHQGHG DJLOH DQG 8;8,SUDFWLFHV ^
+RZHYHU WKH\ ZHUH QRW DEOH WR HYDOXDWH WKHLU SHUIRUPDQFH
EHFDXVH WKH\ ZHUH QRW FRQVLVWHQW LQ DSSO\LQJ VXFK SUDFWLFHV W
7KHUHIRUHHYHQZLWKWKHVLPSOHFRPPRQSUDFWLFHVSODQQLQJ
DQGJXLGDQFHDUHLPSRUWDQWVRWKDWGHYHORSHUVSHUIRUPDQFHFDQ
EH PHDVXUHG DQG WR KHOSSURMHFW PDQDJHUDQG WKH RWKHU WHDP
PHPEHUV WR UHIOHFW RQ WKHLU RZQ ZRUN 6XFK FRQVLVWHQF\ LQ
DSSO\LQJ SUDFWLFHV KHOSV WHDPV DGMXVWWKHLUSHUIRUPDQFH ZKLOH hy
D
SURJUHVVLQJWRZDUGVDSSO\LQJKHDYLHUSUDFWLFHV 

+HQFHWKHJHQHUDOWKHPHRIWKHIRXUUHFRPPHQGDWLRQVFDQ
EHVHHQDVSUDFWLFHVDUHDOZD\VPRUHPDQDJHDEOHDQGHDVLHUWR
IROORZWKDQKLJKOHYHOUHFRPPHQGDWLRQV )LJXUH7KHPDLQUHVHDUFKFRPSRQHQWV

126
  7R SURYLGH &6 HGXFDWRUV ZLWK D QHZ DGDSWLYH
WHDFKLQJ DSSURDFK WKDW LV PRUH FROODERUDWLYH DQG
VWXGHQWRULHQWHG ZKLFK WKH\ FDQ XVH LQ WKHLU
SURJUDPPLQJ ODEV LQVWHDG RI WKH WUDGLWLRQDO WDVN
DVVLJQPHQWDSSURDFKHV
 7R H[WHQG RXU ZRUN WR EH DSSOLHG LQ LQGXVWU\ IRU
WHDPVZKRDUHGHVLJQLQJ8,VIRUPRELOHGHYLFHV
 7R HQKDQFH VRIWZDUH GHYHORSPHQW SURFHVV IRU VPDOO
WHDPV E\ SURYLGLQJ SUDFWLFHV WKDW VXSSRUW
FROODERUDWLRQDQGNQRZOHGJHWUDQVIHU
9 81'(567$1',1*,1&/$66337+528*+(;3(575(9,(:6
7R EHWWHU XQGHUVWDQG KRZ GHYHORSHUV LQWHUDFW ZLWK HDFK
RWKHUZKLOHZRUNLQJLQSDLUVDQGWRHQVXUHWKDWZHKDYHEHWWHU
XQGHUVWDQGLQJ IRU WKH FRQWH[W RI GHYHORSHUV LQ D FODVVURRP
ILIWK JUDGHUV DJHV  \HDUV ROG WR LQWURGXFH WKHP
WRFDOFXODWRUVDQGJHWWKHPWREHIDPLOLDUZLWKWKHP
 7R KDYH QR VSHFLILF LQWHUIDFH UHTXLUHPHQWV DV ZH
ZDQWHG WR REVHUYH KRZ SDLU ZLOO FRPH XS ZLWK
LQWHUIDFHV WKDW ILW WKH UHTXLUHG WDVN EDVHG VROHO\ RQ
WKHLUXQGHUVWDQGLQJRIWKHXVDJHVFHQDULRV
 7REHWLPHOLPLWHGSDLUVVKRXOGGHOLYHUWKHLUZRUNLQJ
SURWRW\SHZLWKLQRQHKRXUWLPH IUDPH:HZDQWHGWR
VHHKRZWLPH FRQVWUDLQWVZRXOGDIIHFWLQWHUDFWLRQVDV
ZHOODVGHFLVLRQVPDGHE\GHYHORSHUV
7KH H[SHUWV FROODERUDWLYHO\ GHFLGHG WKDW IRU RQHKRXU
VHVVLRQH[FKDQJLQJUROHVEHWZHHQGULYHUDQGQDYLJDWRUZRXOG
RFFXUHYHU\PLQXWHVZLWKWKHVHVVLRQIDFLOLWDWRUZRUNLQJDV
D WLPHNHHSHU $IWHU JHWWLQJ VHDWHG DQG SUHSDUHG ZLWK WKH
UHTXLUHGLQIRUPDWLRQDQGWRROVWKHVHVVLRQVZHUHUHDG\WRVWDUW
VHWWLQJLHVWXGHQWVZHFRQGXFWHGDQH[SHUWUHYLHZZLWKWZR
SDLUV RI H[SHULHQFHG $QGURLG GHYHORSHUV 2QH SDLU RI H[SHUWV
ZDVJUDGXDWHVWXGHQWVDQGWKHRWKHUVZHUHXQGHUJUDGXDWHV
 )RU JUDGXDWH VWXGHQWV WKH\ DUH FXUUHQW &6 3K'
VWXGHQWV ZKR ZRUNHG LQ WKH ILHOG RI PRELOH
GHYHORSPHQW IRU ORQJ WLPH 7KH\YH DOVR ZRUNHG DV
A. Session1(Graduateexperts)
'HYHORSHU $ ZDV PRUH H[SHULHQFHG LQ $QGURLG
GHYHORSPHQWWKDQGHYHORSHU%DVKHZRUNHGPRUHWLPHZLWKLW
DQGWDXJKWPRUHFODVVHVUHODWHGWRLWWKDQ%+RZHYHUERWKRI
WKHPZHUHH[SHULHQFHGGHYHORSHUVDQGWKH\GHILQHGWKHPVHOYHV
*UDGXDWH7HDFKLQJ$VVLVWDQWV*7$VIRUWKH PRELOH
GHYHORSPHQWFODVV7KHUHIRUHWKH\DUHH[SHULHQFHGLQ
SURJUDPPLQJ LQ JHQHUDO PRELOH GHYHORSPHQW DQG
HGXFDWLRQ,WZDVLPSRUWDQWIRURXUUHVHDUFKWRJHWWKLV
OHYHORIH[SHULHQFHGGHYHORSHUVWRSUDFWLFH33VRWKDW
WKH\ VKDUH WKHLU WKRXJKWV DQG IHHGEDFN RQ KRZ WKH
SURFHVV ZDV IRU WKHP DV ZHOO DV ZKDW ZRUNHG DQG
ZKDWGLGQWZRUNZKHQLWFRPHVWRSUDFWLFLQJ33
 )RUXQGHUJUDGXDWHVWXGHQWWKH\DUHVHQLRUVWXGHQWVLQ
&6 GHSDUWPHQW DQG ERWK RI WKHP ZRUNHG DV
DV ZH GRQW NQRZ HYHU\WKLQJ DERXW $QGURLG SURJUDPPLQJ
EXW ZH DUH FRQILGHQW WKDW ZH FDQ ILQG VROXWLRQV IRU WKH
SUREOHPV ZH IDFH HYHQ LI WKH\ DUH FRPSOHWHO\ QHZ WR XV
$OWKRXJK WKH\ GHDOW ZLWK HDFK RWKHU EHIRUH LW ZDV WKHLU ILUVW
WLPHWRZRUNWRJHWKHULQSDLURQDSURJUDPPLQJDVVLJQPHQW
7KHILUVWGHFLVLRQPDGHE\WKHPZDVDERXWZKLFK,'(WKH\
VKRXOGXVH%RWKRIWKHPZHUHH[SHULHQFHGZLWK(FOLSVHZKLOH
RQO\ % ZDV WKH RQH IDPLOLDU ZLWK WKH QHZHU $QGURLG 6WXGLR
SODWIRUP)URPWKDWSRLQW$KDGWKHVXJJHVWLRQWRJRZLWKWKH
FRPPRQ JURXQG ZKLFK WKH (FOLSVH VR WKDW WKH\ FDQ VDYH
8QGHUJUDGXDWH 7HDFKLQJ $VVLVWDQWV 87$V IRU
PRELOH GHYHORSPHQW FODVV 7KHLU $QGURLG
GHYHORSPHQW H[SHULHQFH OHYHO LV ORZHU WKDQ WKH
JUDGXDWH OHYHO EXW WKH\ DUH H[SHUWV LQ GHDOLQJ ZLWK
VWXGHQWVLQSURJUDPPLQJFODVVHV
%DVHG RQ WKDW EDFNJURXQG ZH PRYHG IRUZDUG WR FRQGXFW
WZR DFWLYH ZDONWKURXJK 33 VHVVLRQV WR JHW WKH SDUWLFLSDQWV
H[SHUW UHYLHZV DERXW WKH SURFHVV :H REVHUYHG WKHLU
LQWHUDFWLRQV WKURXJKRXW WKH VHVVLRQ DQG WKH\ VKDUHG WKHLU
LQVLJKWV GXULQJ DQGDIWHU WKH VHVVLRQV:H WRJHWKHU ZLWK WKH
H[SHUWVDJUHHGWKDWWKHSURJUDPPLQJDVVLJQPHQWWKDWZDVWR
EH ZRUNHG RQ GXULQJ WKH VHVVLRQV VKRXOG KDYH VRPH PDMRU
FULWHULD ZKLFK OHG XV WR FRPH XS ZLWK WKH IROORZLQJ WDVN
WLPH WR IRFXV RQ WKH DSSOLFDWLRQ UDWKHU WKDQ JHWWLQJ WR NQRZ
DERXWWKHQHZWRRO
,WZDVLPSRUWDQWWRQRWLFHWKDWEHIRUHPDNLQJWKLVGHFLVLRQ
% H[SODLQHG WKH PDLQ EHQHILWV RI XVLQJ WKH QHZHU WRRO ZKLFK
ZHUH XQNQRZQ WR $ % KDV HYHQ GLG D VKRUW GHPR VKRZLQJ
VRPH RI WKH QLFH IHDWXUHV SURYLGHG E\ $QGURLG 6WXGLR
+RZHYHUERWKRIWKHPZHUHVDWLVILHGE\ZRUNLQJRQWKH,'(
WKDWWKH\ERWKNQHZDERXW6RWKHLUGHFLVLRQRIZRUNLQJRQWKH
ROGHUWRROGLGQWSUHYHQW$IURPOHDUQLQJVRPHIHDWXUHVDERXW
WKHWRROWKDWWKH\GHFLGHGQRWWRXVH7KLVLVDQLPSRUWDQWSRLQW
DERXW 33 DQG FROODERUDWLYH HQYLURQPHQWV ZKLFK LV WKDW
GHYHORSHUVGRQWRQO\OHDUQDERXWZKDWWKH\DUHXVLQJEXWDOVR
DERXWWKHRSWLRQVWRROVDSSURDFKHVWKDWWKH\GHFLGHGQRWWRXVH
UHTXLUHPHQWVFDWHJRUL]HGEDVHGRQWKHFRUUHVSRQGLQJFULWHULD
 7R EH D IDPLOLDU WDVN WR LPSOHPHQW D FDOFXODWRU WKDW
SHUIRUPV WKH EDVLF RSHUDWLRQV DQG UXQV RQ DQGURLG
GHYLFHV
 7R EH IOH[LEOH WKH WDVN ZDV RSHQ IRU ZKDWHYHU
DVVXPSWLRQV DQG GHFLVLRQV PDGH E\ SDLUV $
FDOFXODWRUFDQEHDQ\WKLQJIURPWZRVLPSOHWH[WER[HV
WKDW XVH WKH GHYLFHV NH\ERDUG WR D IXOOVL]HG VFUHHQ
ZLWKDOOEXWWRQVDQGRSHUDWLRQV
 7R KDYH XVDJH FRQWH[W WKDW DVVLJQPHQW GHILQLWLRQ
VWDWHG WKDW WKLV FDOFXODWRU ZLOO EH XVHG E\ IRXUWK DQG
127
2QFHGHYHORSHU$WKHILUVWGULYHUZDVGRQHZLWKVHWWLQJ
XS WKH QHZ SURMHFW WKH\ VWRSSHG WR WDON DERXW WKH OD\RXW WKDW
WKH\ VKRXOG XVH IRU WKHLU DSSOLFDWLRQ $IWHU H[FKDQJLQJ VRPH
YHUEDO LGHDV WKH\ KROG PDUNHUV DQG EHJDQ GUDZLQJ RQ D
ZKLWHERDUGULJKWEHKLQG WKHP7KH\ DSSURYHG XVLQJ WKH JULG
YLHZ HOHPHQW DOWKRXJK ERWK RI WKHP GLGQW ZRUN ZLWK LW
EHIRUH7KH\ZHUHFRQILGHQWDERXWWKHLUDELOLW\WRWU\VRPHWKLQJ
IRUWKHILUVWWLPHDVORQJDVLWZLOODFKLHYHWKHUHTXLUHGUHVXOWV
7KHUHIRUH WKH\ GLUHFWO\ PRYHG IRUZDUG DQG EHJDQ
LPSOHPHQWLQJWKHLUEDVLFVROXWLRQLGHD
2SSRVLWH WR WKHLU H[SHFWDWLRQV GHDOLQJ ZLWK JULG YLHZ
ZDVQWWKDWHDV\DQGWKH\KDGWRKDQGOHVRPHLVVXHVUHODWHGWR
VL]LQJSRVLWLRQLQJDQGDOLJQPHQWRIFHOOV6XJJHVWLRQVWRVROYH
VXFK SUREOHPV FDPH IURP WKH WZR GHYHORSHUV DV WKH\ IDFHG
WKRVHSUREOHPVGXULQJWKHILUVWWKUHHURXQGVRIWKHIRXUURXQG
VHVVLRQ 0RVW RI WKH WLPHV GHYHORSHU $V VXJJHVWLRQV ZHUH
DSSURYHGDQGRXWWRDFWLRQHYHQLI%KDGVRPHRWKHUVROXWLRQV
EXW WKDW GLGQW FDXVH DQ\ SUREOHPV DV $V VXJJHVWLRQV ZHUH
EDVHG RQ H[SHULHQFH DQG XVXDOO\ SURYLGHG EHWWHU VROXWLRQV
+RZHYHUWZRPDLQSRLQWVDIIHFWHGWKLVW\SHRILQWHUDFWLRQ
'HYHORSHU$VXJJHVWHGDOLQHRIFRGHWREHZULWWHQE\%
ZKLOH%VKRZHGKLPWKDWWKLVPLJKWQRWEHWKHULJKWZD\WRGR
LW+RZHYHU$LQVLVWHGWKDWLWVDJRRGZD\WRGRLWVR%MXVW
GLGDVZKDW$SURSRVHGDQGWKH\PRYHGIRUZDUG6WDUWLQJIURP
WKLV SRLQW %V OHYHO RI LQWHUDFWLRQ DQG VXJJHVWLRQV ZHUH OHVV
WKDQEHIRUH
 /DWHU ZKHQ DQ HUURU ZDV IRXQG E\ WKH GHEXJJHU
GHYHORSHU$WULHGWRIL[LWE\HGLWLQJVRPHSDUWVZULWWHQE\%
+RZHYHU WKH\ GLVFRYHUHG WKDW WKH SUREOHP DURXVHG IURP WKH
SUHYLRXVVXJJHVWLRQRI$,WWKHQEHFDPHFOHDUWKDW%ZDVULJKW
RQ WKH ILUVW SODFH 6WDUWLQJ IURP WKDW SRLQW %V OHYHO RI
LQWHUDFWLRQ LQFUHDVHG DV KLV VXJJHVWLRQV FRPPHQWV DQG
LQVLJKWV ZHUH PXFK PRUH WKDQ EHIRUH 0RUHRYHU $ EHJDQ WR
FRQVLGHU%VLQSXWVDQGDVNHGIRUKLVDSSURYDOPRUHFDUHIXOO\
ZDVPLQLPL]HGWRVDYHWLPHDQGGHYHORSHU%ZDVGULYLQJWKH
NH\ERDUGZLWK$VLQWHUYHQWLRQRQO\WRFRUUHFWPLVWDNHVRUWR
VXJJHVWLGHDVWKDWKHOS%JRHVIDVWHU$WWKHHQGWKH\ZHUHDEOH
WRSURYLGH D ZRUNLQJ $QGURLG DSSOLFDWLRQ WKDW FDQEH XVHGWR
LQSXWWZRQXPEHUVDQGFDOFXODWHWKHLUVXP
7KH ILUVW ZRUGV VDLG E\ H[SHUWV DIWHU WKH\ ZHUH GRQH ZLWK
WKH VHVVLRQ ZHUH LW ZDV IXQ DQG LW ZDV H[FLWLQJ ,W ZDV
REYLRXV WKDW WKH\ ZHUH SUDFWLFLQJ D JDPHVW\OH SDLULQJ ZKHUH
WKHLU WHDP ZDV SOD\LQJ DJDLQVW WLPH (YHQ LI WKH\ GLGQW
SURYLGH WKH FRPSOHWH UHTXLUHG SURGXFW WKH\ ZHUH DEOH WR
GHOLYHUDVPDOOZRUNLQJYHUVLRQRILW7KHWZRH[SHUWVGHFODUHG
WKDWWKH\HQMR\HGWKHWLPHVSHQWGXULQJWKDWVHVVLRQ'HYHORSHU
$ VKRZHG WKDW LWG WDNH KLP PRUH WLPH WR ZRUN RQ VXFK
DVVLJQPHQWLIKHZDVWRGRLWRQKLVRZQEHFDXVHGHYHORSHU%
XVHG VRPH FRGLQJ SUDFWLFHV WKDW $ ZRXOG RWKHUZLVH QRW XVH
WKHP'HYHORSHU%LQGLFDWHGWKDWLWGWDNHKLPDOPRVWWKHVDPH
WLPHWRGRWKHVDPHMREEXWKHEHOLHYHGWKDWWKHTXDOLW\RIWKH
WDVNFRPLQJIURPWKHSDLUZRXOGGHILQLWHO\EHEHWWHUWKDQRIWKH
RQH KHG ZRUN RQ E\ KLV RZQ7KH\ ERWK DJUHHG WKDW WKH RQH
ZKRGULYHVWKHNH\ERDUGLVXVXDOO\LQDEHWWHUSRVLWLRQWRGHFLGH
WKDQ WKH QDYLJDWRU 7KDW FOHDUO\ DSSHDUHG RQ WKH ILQDO URXQG
ZKHUHGHYHORSHU%GHFLGHGWRJRIRUDFRGLQJSUDFWLFHWKDW$
WKDQWKHSUHYLRXVURXQG
7KHVH WZR SRLQWV VKRZ WKH LPSRUWDQFH RI VHOIFRQILGHQFH
PXWXDOUHVSHFWDQGRSHQO\VKDULQJWKRXJKWVIRUWKHVXFFHVVRI
33VHVVLRQV)RUWKLVVSHFLILFVHVVLRQWKHWZRH[SHUWVKDGJRRG
OHYHOV RI VHOIFRQILGHQFH DQG HYHQ KLJKHU OHYHO RI PXWXDO
UHVSHFW IRU HDFK RWKHU +RZHYHU $ VHHPHG WR EH PRUH
FRQILGHQWDERXWKLVDELOLWLHVZKLFKPDGHKLPWRVRPHH[WHQW
 GLVUHJDUG %V VXJJHVWLRQV 7KLV VW\OH RI LQWHUDFWLRQ GRHVQW
OHDG WR D JRRG 33 VHVVLRQ +RZHYHU LW ZDV VPRRWKO\
FRUUHFWHGZLWK%VWULDOVWRDOZD\VVKDUHKLVWKRXJKWVDQGZLWK
ERWK RI WKHP GLVFXVVLQJ ZKDW WKH\ DUH ZRUNLQJ RQ )DFLQJ
VLWXDWLRQV OLNH WKH WZR SUHYLRXVO\VKRZQ DERYH FRXOG OHDG33
VHVVLRQWRVXFFHVVRUIDLOXUHEDVHGRQKRZWKHSDLUKDQGOHLW
7KH GLVFXVVLRQ DERXW XVHUV DQG XVDJH VFHQDULRV DSSHDUHG
HDUO\ ZKHQ WKH SDLU ZRUNHG RQ WKH PDLQ OD\RXW 7KH\ EHJDQ
DVNLQJDQGDQVZHULQJTXHVWLRQVDERXWZKDWZRXOGWKHXVHUV
GLGQW SUHIHU XVLQJ KDUGFRGHG OLVWHQHUV EXW $ GLGQW VWRS
KLP EHFDXVH WLPH ZDV UXQQLQJ DQG WKH\ ZDQWHG WR KDYH D
ZRUNLQJ GHPR 7KH\ DOVR IHOW WKDW VRPHWLPHV WKH QDYLJDWRU
ZDQWHG WR JHW WKH NH\ERDUG WR GR VRPHWKLQJ WKDW ZRXOG EH
IDVWHU WKDQ OHDYLQJ LW IRU WKH GULYHU WR GR EXW RI FRXUVH WKH\
GLGQWEHFDXVHLWZDVQWDOORZHGGXULQJWKDW33VHVVLRQ
'HYHORSHU % VKRZHG WKDW KH ZDVQW DEOH WR WU\ VRPH
VROXWLRQVXVLQJKLVRZQZD\EHFDXVHGHYHORSHU$ZDVXVXDOO\
GHFLGLQJ RQ WKH IO\ ZKLOH KHV GULYLQJ WKH NH\ERDUG 7KLV
REYLRXVO\FKDQJHGDIWHUWKHVHFRQGURXQGZKLFKLVUHODWHGWR
ZKDWGHYHORSHU$DGPLWWHG%LVUHDOO\DJRRGSURJUDPPHU,
OLNHG KLP +HV UHDOO\ EHWWHU WKDQ ZKDW , WKRXJKW 7KLV
IHHGEDFN PDGH LW FOHDU WKDW HYHQ ZLWK VRPH SUHDVVXPSWLRQV
WKDW FDQ H[LVW DPRQJ GHYHORSHUV 33 XVXDOO\ KHOSV WR FRUUHFW
VRPHRIWKRVHIDOVHDVVXPSWLRQVDIWHUWKHILUVWURXQGV7KHWZR
GHYHORSHUV VDLG WKDW WKH\ ERWK OHDUQW QHZ WKLQJV IURP HDFK
 \HDUV ROG NLGV QHHG WR ILQG LQ VXFK DSSOLFDWLRQ %DVHG RQ RWKHU7KRVHQHZWKLQJVZHUHPRUHUHODWHGWRFRGLQJSUDFWLFHV
WKHLU DVVXPSWLRQV WKH\ GHFLGHG WR LQFOXGH RQO\ WKH PDLQ WLSV DQG KDELWV 7KH\ DOVR VKRZHG WKDW WKH\ HQMR\HG OHDUQLQJ
DULWKPHWLFRSHUDWLRQVDGGLWLRQVXEWUDFWLRQPXOWLSOLFDWLRQDQG KRZ WR GHDO ZLWK D QHZ OD\RXW VWUXFWXUH WKH JULG OD\RXW
GLYLVLRQ7KH\H[FOXGHGIUDFWLRQVDQGWKDWVZK\WKH\GLGQW
LQFOXGHWKHGRWEXWWRQLQWKHLUILQDOYHUVLRQ7KH\SODQQHGWR
ZRUN RQ VRPH JUDSKLFV WR EHWWHU ILW WKH \RXQJ XVHUV EXW WKH
WLPHGLGQWDOORZIRUWKLV+RZHYHUWKH\VNLSSHGDQLPSRUWDQW
IXQFWLRQ ZKLFK LV WKH DELOLW\ WR GHOHWH RQH RU PRUH GLJLWV LQ
FDVH RI HUURUV 7KDW PLVWDNH DSSHDUHG RQO\ RQ WKHLU ILQDO WHVW
EHIRUHGHOLYHULQJWKHSURGXFW$OWKRXJKLWGEHHDV\IRUWKHPWR
FRUUHFW WKDW SUREOHP LW VWLOO VKRZV WKDW WKHLU GLVFXVVLRQV KDYH
PLVVHG VRPH PDMRU XVDJH VFHQDULRV 7KLV VKRZV WKDW 33
HQKDQFHVWKHGHYHORSHUVDZDUHQHVVDERXWXVHUH[SHULHQFHEXW
LW VWLOO QHHGV VXSSRUWLQJ VWHSV WR HQVXUH WKDW GHYHORSHUV IXOO\
XQGHUVWDQGKRZXVHUVZLOOLQWHUDFWZLWKWKHLUDSSOLFDWLRQ
,WZDVFOHDUIRUWKHSDLURQWKHILQDOURXQGWKDWWKH\ZLOOQRW
EHDEOHWRGHOLYHUDIXOO\IXQFWLRQDOSURWRW\SHVRWKH\GHFLGHG
WR JR IRU LPSOHPHQWLQJ RQO\ RQH RSHUDWLRQ ZKLFK ZDV WKH
WRJHWKHU ZKLFK PDGH WKHLU OHDUQLQJ WLPH VKRUWHU $V IRU WKHLU
SHUVRQDOIHHOLQJVUHJDUGLQJWKH33VHVVLRQWKH\FRQFOXGHGZLWK
DOPRVW WKH VDPH VHQWHQFH 7KDW ZDV SUHWW\ DZHVRPH DQG ,
ZRXOGOLNHWRGRWKDWPRUH
B. Session2(Undergraduateexperts)
%RWKGHYHORSHUV&DQG'KDGDOPRVWWKH VDPH H[SHULHQFH
OHYHOLQSURJUDPPLQJLQJHQHUDODQGLQ$QGURLGSURJUDPPLQJ
VSHFLILFDOO\7KH\NQHZHDFKRWKHUEHIRUHWKLVVHVVLRQEXWWKH\
QHYHU ZRUNHG WRJHWKHU RQ SURJUDPPLQJ DVVLJQPHQWV ,Q
DVVHVVLQJ WKHLU H[SHULHQFH OHYHO WKH\ GHVFULEHG WKHPVHOYHV DV
EHLQJDEOHWRJHWWKHUHTXLUHGNQRZOHGJHWRJHWWKHMREGRQH
%RWK RI WKHP ZHUH IDPLOLDU ZLWK (FOLSVH EXW RQO\ &
ZRUNHGZLWKWKHQHZHU$QGURLG6WXGLR8QOLNHWKHSDLURIWKH
ILUVW VHVVLRQ WKH\ GHFLGHG WR JR IRU $QGURLG 6WXGLR DIWHU &
DGGLWLRQ 7KH\ VNLSSHG VRPH LQWHUIDFH UHTXLUHPHQWV LQ IDYRU H[SODLQHG LWV DGYDQWDJHV DQG KRZ LW PDGH VRPH LVVXHV HDVLHU
RI SURYLGLQJ D SURWRW\SH WKDW ZRUNV HYHQ ZLWK VRPH HUURUV IRUKHU6WDUWLQJIURPWKLVSRLQWDOPRVWDOOGHFLVLRQVZHUHGRQH
H[FHSWLRQV2QWKDWILQDOURXQGWKHGLVFXVVLRQEHWZHHQWKHP WKHVDPHZD\RQHGHYHORSHUVXJJHVWVVRPHWKLQJVKRZLQJKHU

128
UDWLRQDOHWKHRWKHUDSSURYHV,WZDVFOHDUWKDWWKLVSDLUGHDOWLQ
DSHHUWRSHHULQWHUDFWLRQOHYHOUDWKHUWKDQWKHOHDGHUHPSOR\HH
LQWHUDFWLRQPRGHOWKDWDSSHDUHGRQWKHILUVWURXQGRIVHVVLRQ
7KHUHIRUH LQWURGXFLQJ 33 VHHPV WR ZRUN EHWWHU ZLWK
GHYHORSHUV RI VLPLODU H[SHULHQFH OHYHOV 2QFH GHYHORSHUV LQ
FHUWDLQHQYLURQPHQWJHWXVHGWRWKHFRQFHSWVRI33SDLULQJFDQ
WKHQ GRQH DPRQJ GLIIHUHQW OHYHOV RI GHYHORSHUV ZKHUH WKH
OHDUQLQJSURFHVVFDQEHPRUHYDOXDEOHDQGEHQHILFLDO
'HYHORSHU&ZDVWKHILUVWGULYHUDVVKHZDQWHGWRLQWURGXFH
WKH QHZ ,'( WR GHYHORSHU ' $IWHU D YHU\ VKRUW LQWURGXFWLRQ
WKH\ GHFLGHG WR ZULWH VRPH EDVLF OLQHV RI FRGH 7KH\ WKHQ
VWRSSHGDIWHUPLQXWHVZKHQWKH\UHDOL]HGWKDWWKH\GLGQWDJUHH
RQ KRZ WKH LQWHUIDFH ZRXOG ORRN OLNH $IWHU VSHQGLQJ WKUHH
PLQXWHVWDONLQJDERXWWKDWWKH\KROGVRPHPDUNHUVDQGEHJDQ
GUDZLQJWKHLULGHDVRQDZKLWHERDUG(DFKRIWKHPKDGKHURZQ
GHVLJQ LGHD ZLWK & VXJJHVWLQJ D YHU\ EDVLF DQG VLPSOH
LQWHUIDFH ZKLOH ' VXJJHVWLQJ D PRUH DWWUDFWLYH LQWHUIDFH ZLWK
PRUH IHDWXUHV %RWK RI WKHP DJUHHG WKDW 'V LGHD ZDV EHWWHU
EXWWKH\DOVRDJUHHGWKDW&VSURSRVDOZRXOGEHPRUHIHDVLEOH
EHFDXVH RI WLPH FRQVWUDLQWV 2QFH DJUHHG RQ WKH LQLWLDO
FRPSRQHQWVRIWKHOD\RXWWKH\EHJDQLPSOHPHQWLQJWKHFRGHWR
PDNH LW IXQFWLRQDO ZLWKRXW JLYLQJ DQ\ WLPH WR GLVFXVV WKH
SRVLWLRQLQJORRNRUDQ\GHWDLOVUHJDUGLQJWKH8,HOHPHQWV
7KH WUDQVLWLRQ IURP EHLQJ GULYHU WR QDYLJDWRU ZHQW
VPRRWKO\ HYHU\ WLPH DQG DOO WKH LPSOHPHQWDWLRQ GHFLVLRQV
ZHUHHDVLO\DSSURYHGE\WKHWZRGHYHORSHUV,WZDVQRWLFHDEOH
WKDW WKH\ GLGQW KDYH WR SHUIRUP DQ\ RQOLQH VHDUFK IRU WKHLU
ZRUN7KH\GHSHQGHGKHDYLO\RQWKHLUSUHYLRXVNQRZOHGJHDQG
ZKDWWKH\DOUHDG\NQRZ7KDWOHGWKHPWREHPRUHFRQVHUYDWLYH
LQWKHLULPSOHPHQWDWLRQFKRLFHVVRWKDWWKH\GRQWKDYHWRIDFH
VRPH VXGGHQ QHZ VLWXDWLRQV WKDW ZRXOG SUHYHQW WKHP IURP
EHLQJDEOHWRGHOLYHUDIXQFWLRQDOSURWRW\SHRQWLPH
2QHRIWKHLPSRUWDQWREVHUYDWLRQVDERXWWKLVVHVVLRQLVKRZ
WKH SDLU ZDV VR FDUHIXO DERXW JHWWLQJ WKH DVVLJQPHQW GRQH
UDWKHU WKDQ EXLOGLQJ D XVDEOH DSSOLFDWLRQ 7KH\ ZHUH GHDOLQJ
VWUXFWXUH DQG LQWHUIDFH %RWK RI WKHP WKRXJKW WKH\G KDYH
SURYLGHG D EHWWHU SURGXFW LI WKH\ KDYH SDLG PRUH DWWHQWLRQ WR
GHVLJQ
:KHQLWFRPHVWRLQWHUDFWLRQDQGFRPPXQLFDWLRQEHWZHHQ
WKHP GHYHORSHU & OLNHG WKDW KHU SDUWQHU ZDV DOZD\V WDONLQJ
ZLWK KHU DQG WKDW VKH ZDVQW WKH RQO\ RQH ZKR WDONHG DOOWKH
WLPH %HLQJ DEOH WR WDON DQG OLVWHQ ZKLOH FRGLQJ ZDV DQ
LPSRUWDQW LVVXH WKDW WKH WZR H[SHUWV HPSKDVL]HG LWV YDOXH IRU
WKH VXFFHVV RI D 33 VHVVLRQ 7KH\ DOVR IHOW WKDW WKH\ ZHUH
OHDUQLQJ WRJHWKHU UDWKHU WKDQ OHDUQLQJ IURP HDFK RWKHU 6LQFH
ERWKRIWKHPZHUHRIDVLPLODUDFDGHPLFLQWHOOLJHQFHOHYHODV
VWDWHG E\ GHYHORSHU & LW ZDV HDV\ IRU WKHP WR H[SUHVV WKHLU
LGHDVDQG WREH VXUH WKDW WKHSDUWQHU ZLOO XQGHUVWDQG ZKDW WKH
LPSOHPHQWDWLRQVXJJHVWLRQVDQGFRGLQJVWXII
,WZDVLPSRUWDQWIRUWKHSDLUWKDWWKHLUH[SHULHQFHOHYHOVDUH
FORVH WR HDFK RWKHU 'HYHORSHU & WDONHG DERXW KHU UHDFWLRQ LI
VKH GHDOV ZLWK VRPHRQH ZLWK PRUH H[SHULHQFH VKRZLQJ WKDW
VKHHDVLO\JHWVLQWLPLGDWHGLQVXFKVLWXDWLRQVDQGWKDWVKHJHWV
VK\DQGVWUHVVHGZKLFKOHDGKHUQRWWRJDLQIURPZRUNLQJZLWK
WKH H[SHULHQFHG GHYHORSHUV 2Q WKH RWKHU KDQG GHYHORSHU '
KDG QR SUREOHPV LQ GHDOLQJ ZLWK H[SHULHQFHG GHYHORSHUV EXW
KHU UHDFWLRQ ZRXOG EH WR OHDYH KLPKHU GR WKH UHTXLUHG MRE
WUXVWLQJWKDWVKHOOEHRQO\FDOOHGLIVKHVQHHGHGWR)RU'WKH
H[SHULHQFHGGHYHORSHUZLOOEHWKHOHDGHUZKRVUHVSRQVLEOHIRU
WKH KDUG ZRUN ZKLOH VKH ZLOO EH WKH DVVLVWDQW ZKR ZLOO KHOS
RQO\ ZKHQ UHTXLUHG 7KHVH WZR GLIIHUHQW UHDFWLRQV WR GHDOLQJ
ZLWK PRUH H[SHULHQFHG SDUWQHUV HPSKDVL]H WKH LPSRUWDQFH RI
SDLULQJ GHYHORSHUV RI VLPLODU H[SHULHQFH OHYHOV HVSHFLDOO\ RQ
WKHILUVW33VHVVLRQV
7KH WZR H[SHUWV DJUHHG WKDW IRU D VLPSOH DVVLJQPHQW OLNH
WKDW RQH 33 ZRXOGQW HQKDQFH WKHLU SHUIRUPDQFH QRU TXDOLW\
ZKLOH LW PD\ GRHV IRU ODUJHU SURMHFWV ZLWK PRUH UHTXLUHPHQWV
DQG VRSKLVWLFDWHG LPSOHPHQWDWLRQ LVVXHV +RZHYHU WKHLU
RSLQLRQ PD\ EH DIIHFWHG E\ WKH IDFW WKDW WKH\ GLGQW GHGLFDWH
HQRXJKWLPHWRWKHUHTXLUHPHQWVWKDWWKH\VKRXOGKDYHIRFXVHG
ZLWKWKHWDVNDVDFODVVDVVLJQPHQWWKDWZLOOEHJUDGHGZLWKRXW
FRQVLGHULQJKRZXVHUVZLOOGHDOZLWKLW7KHVL]HVDQGORFDWLRQV
RI WKHWH[WER[HV WKHDOLJQPHQW DQGSRVLWLRQVRIEXWWRQVDQG
DOO WKH DVSHFWV UHODWHG WR WKH LQWHUIDFH ZHUH OHIW WR WKH ODVW
URXQG7KH\ZDQWHGWRPDNHVXUHWKDWXVHUVFDQLQSXWQXPEHUV
FOLFN WKH UHTXLUHG RSHUDWLRQ DQG JHW UHVXOWV WR DSSHDU RQ WKH
VFUHHQ $W WKH HQG WKH\ SURYLGHG D IXQFWLRQDO SURGXFW ZLWK D
SRRUGHVLJQWKDWODFNVVRPHEDVLFXVDELOLW\UHTXLUHPHQWV
$OWKRXJK WKH WZR GHYHORSHUV FODLPHG WKDW WKH\ ZHUH
RQLHGHVLJQXVDELOLW\DQGXVDJHVFHQDULRV7KHHIIHFWRI33
IRUWKHPZDVQWFOHDUEHFDXVHWKH\MXPSHGGLUHFWO\LQWRFRGLQJ
ZKLFKOHGWKHPWRPLVVWKHPDLQDQGLPSRUWDQWDGYDQWDJHVRI
33WKDWZRXOGKDYHKHOSHGWKHPGHVLJQLQJDEHWWHUDSSOLFDWLRQ
C. CollaborationpitfallsduringPPsessions
%DVHG RQ RXU OLWHUDWXUH UHYLHZ RQ 33 WRJHWKHU ZLWK WKH
H[SHUW UHYLHZV GLVFXVVHG DERYH ZH FDPH WR KLJKOLJKW VRPH
SLWIDOOVWKDWFDQDIIHFWGHYHORSHUVSHUIRUPDQFHDQGLQWHUDFWLRQ
GXULQJ 33 VHVVLRQV 7KH IROORZLQJ HLJKW LVVXHV DUH UDQNHG
DIIHFWHGZLWKWKHWLPHOLPLWVDQGWKDWWKH\ZRXOGKDYHFRQVLGHU
XVDELOLW\ LVVXHV LI WKH\ KDG PRUH WLPH LW ZDV FOHDU WKDW WKHLU
LQWHUHVW LQ XVHU H[SHULHQFH ZDVQW D SULRULW\ UHJDUGOHVV WLPH
FRQVWUDLQWV 7KH\ GLGQW GHVLJQ D FRPSOHWH OD\RXW EHIRUH
FRGLQJ WKH\ GLGQW WDON DERXW LQWHUIDFH FRPSRQHQWV EXW LQ
LPSOHPHQWDWLRQFRQWH[WWKH\OHIWWKHOD\RXWGHVLJQIRUWKHODVW
URXQGDQGWKH\GLGQWFRQVLGHUXVDJHVFHQDULRV
7KH WZR H[SHUWV VKRZHG WKDW WKH\ GLGQW FRQVLGHU
GHVLJQLQJ IRU VFKRRO NLGV RU VHUYLQJ DV DQ LQWURGXFWRU\
FDOFXODWRUDVUHTXLUHPHQWVRIWKHDSSOLFDWLRQ7KHLUVXJJHVWLRQ
ZDV ZH FDQ ODWHU DGG VRPH FRORUV DQG JUDSKLFV WR EH PRUH
DSSHDOLQJ WR VWXGHQWV 'HYHORSHU & VKRZHG WKDW LI WKHUHV
VRPHWKLQJ WKDW VKHG FKDQJH LI VKH ZRXOG UHSHDW WKDW 33
VHVVLRQ LWG EH WR VSHQG PRUH WLPH RQ GHVLJQ IRU SURJUDP
129
IURP KLJKHU WR ORZHU EDVHG RQ WKH H[SHUWV HYDOXDWLRQ IRU
ZKLFK SLWIDOO ZRXOG KDYH WKH PRVW QHJDWLYH HIIHFW RQ 33
VHVVLRQV FRQGXFWHG E\ VWXGHQWV 7KH ILUVW OLVWHG LVVXH ZRXOG
FDXVH VHYHUH SUREOHPV ZKLOH WKH ODVW ZRXOG KDYH PLQLPDO
QHJDWLYHHIIHFWV
 'HYHORSHUVFRPLQJIURPGLIIHUHQWEDFNJURXQGV
:KHQ LQWURGXFLQJ 33 IRU WKH ILUVW WLPH LW LV KDUG WR JHW
GHYHORSHUV WR WDON WR HDFK RWKHU LI WKH\ GLGQW DOUHDG\ KDYH D
FRPPRQ EDFNJURXQG WR VWDUW IURP DQG PRYH IRUZDUG 7KLV
KRZHYHU FDQ EH XVHIXO ZLWK GHYHORSHUV ZKR DUH H[SHULHQFHG
DQGFRPIRUWDEOHZLWK33 ZKHUH WKH GLYHUVHEDFNJURXQGVZLOO
DGG WR WKHLU VNLOOV DQG ZLGHQ WKH VFRSH RI WKHLU GLVFXVVLRQV
DERXWWKHSURGXFWXQGHUGHYHORSPHQW
  'HYHORSHUVZLWKGLIIHUHQWVNLOOH[SHULHQFHOHYHOV
7KLVFDQZRUNZHOOZLWKSDLUVZKRDUHIDPLOLDUZLWK33VR
WKDWNQRZOHGJHWUDQVIHUFDQEHDPDMRUEHQHILWIURPSUDFWLFLQJ
33 +RZHYHU IRU EHJLQQHUV LQ 33 LWV PRUH LPSRUWDQW WR JHW
IDPLOLDUZLWK33WKURXJKEHLQJDEOHWRWDONZLWKWKHLUSDUWQHUV
DVSHHUVUDWKHUWKDQDVVWXGHQWVRUOHDUQHUV$VZH VKRZHGRQ
WKH ILUVW VHVVLRQ WKH ODFN RI SHHUWRSHHU LQWHUDFWLRQ KDG
DIIHFWHG WKH ILUVW WZR 33 URXQGV XQWLO WKDW JRW IL[HG DV WKH\
SURJUHVVHG LQ FRGLQJ +DYLQJ SDLUV ZLWK SDUWQHUV RI GLIIHUHQW
H[SHULHQFHV OHYHOV ZLOO EH UHTXLUHG IRU WKRVH ZKR DUH DOUHDG\
FRPIRUWDEOHZLWK33DVWKDWHQKDQFHVWKHOHDUQLQJFXUYHKHOSV
WUDQVIHUULQJ NQRZOHGJH DQG SXWV FROODERUDWLYH HQYLURQPHQW
LQWRDFWLRQIRUWKHEHQHILWRIWKHZKROHWHDP
 /DFNRISODQQLQJDQGWLPHPDQDJHPHQW
7KH WZR SDLUV GLGQW ZRUN DV WLPH NHHSHUV GXULQJ WKHLU
VHVVLRQVDQGRXUREMHFWLYHZDVWRDOORZWKHPWRIRFXVPRUHRQ
WKHLU ZRUN UDWKHUWKDQ FKHFNLQJ WKH WLPH HYHU\ QRZ DQG WKHQ
+RZHYHU WKH\ NQHZ WKDW WKH\ ZHUH RQO\ DOORZHG RQ KRXU WR
ILQLVKWKHLUZRUN,WZDVQRWLFHDEOHWKDWGXULQJWKHWZRVHVVLRQV
QRQHRIWKHPKDYH PHQWLRQHGDQ\WKLQJDERXWWLPHUHPDLQLQJ
7KH\ GLGQW KDYH DQ\ WHQWDWLYH SODQ RQ KRZ ZRUN ZLOO JR RQ
WKURXJK WKH KRXU WLPH VORW 7KDWV ZK\ WKH WZR SDLUV IDFHG
WKH VDPH SUREOHP RQ WKH ILQDO URXQG ZKHUH WKH\ KDG WR ZUDS
WKHLUZRUNXSWREHDEOHWRGHOLYHUDZRUNLQJGHPR7KHUHIRUH
LWV LPSRUWDQW WR FRQVLGHU WLPH PDQDJHPHQW EHWZHHQ SDLUV DV
VRPHWKLQJWKDWWKH\VKRXOGFRQVLGHUHDUO\RQWKHILUVW33URXQG
 -XPSLQJ GLUHFWO\ LQWR FRGLQJ ZLWKRXW ZRUNLQJ RQ
GHVLJQ
$OWKRXJKWKLVLVWKHFDVHZLWKPRVWRIGHYHORSPHQWWHDPV
LWFRPHVLQWRIRFXVZLWK33WHDPV2QHRIWKHPDMRUEHQHILWVRI
33 LV WR HQVXUH VRIWZDUH TXDOLW\ EHFDXVH RI WKH LQVWDQWDQHRXV
WHVWLQJDQGWKHLGHDWLRQWKDWRFFXUVZLWKLQWZRPLQGVLQVWHDGRI
RQH 6R LI WKH TXDOLW\ LV QRW DFKLHYHG 33 ORVHV RQH RI LWV
PDLQDGYDQWDJHV7KHUHDVRQIRUGHYHORSHUVHLWKHULQGLYLGXDOV
RU LQ SDLUV WR VDFULILFH TXDOLW\ LV WKH OLPLWHG WLPH +RZHYHU
SDLUV LQ 33 VHVVLRQV VKRXOG PDQDJH WR JHW WKH EHVW XVH RI
DYDLODEOHWLPHWRSURGXFHWKHUHTXLUHGIXQFWLRQDOSURGXFWZLWK
DQDFFHSWDEOHTXDOLW\OHYHO7KHSUREOHPZLWKWKHWZRSDLUVLQ
RXU WZR VHVVLRQV ZDV ZLWK WKHLU PDLQ JRDO ZKLFK ZDV WR
GHOLYHUDZRUNLQJSLHFHRIVRIWZDUHRQWLPHQRWWRGHOLYHU
TXDOLW\VRIWZDUHRQWLPH0RUHSUDFWLFHVVKRXOGEHSXWWRXVH
WR HQVXUH WKDW TXDOLW\ LV SDUW RI WKH GHOLYHUDEOH QRW D
FRPSOHPHQWDU\IHDWXUH
 'LVUHJDUGLQJ FUHDWLYH LGHDV LQ IDYRU RI WUDGLWLRQDO
VROXWLRQV
7U\LQJ QHZ FRGLQJ DSSURDFKHV ZRUNLQJ RQ XQIDPLOLDU
WRROV DQG LPSOHPHQWLQJ XQFRPPRQ VROXWLRQV KDYH DOZD\V
EHHQ GLVFRXUDJHG ZLWK WKH H[FXVH RI WLPH OLPLWV 7KLV IXOO\
FRQWUDGLFWVZLWKWKH REMHFWLYHVRI33ZKHUH DQLPSRUWDQWRQH
RI WKHP LV WR SURPRWH FUHDWLYH VROXWLRQV DQG EXLOG DQ
LQQRYDWLYH HQYLURQPHQW ,QWURGXFLQJ 33 WR GHYHORSHUV VKRXOG
IRFXVRQWKHUHDOREMHFWLYHVEHKLQG33QRWMXVWWRGHDOZLWKLW
DVDGHYHORSPHQWWHFKQLTXH7KHWZRVHVVLRQVZLWQHVVHGVRPH
LGHDVWKDWKDYHEHHQUHMHFWHGEHFDXVHRIWKHKRXUWLPHOLPLW
ZKLOHWKHREMHFWLYHRI33LVWRHQFRXUDJHSDLUVWRZRUNRQWKHLU
LGHDVDQGWU\WRPDQDJHWKHLUWLPHWREHDEOHWRZRUNRQWKHLU
LGHDV HYHQ E\ DVNLQJ IRU PRUH WLPH LI UHTXLUHG DV FUHDWLYH
6HVVLRQ VKRZHG KRZ WKH ODFN RI SURSHU VRIWZDUH GHVLJQ
KDVDIIHFWHGWKHGHYHORSHUVDELOLW\WRGHOLYHUDTXDOLW\SURGXFW
DQG LW DOVR DIIHFWHG WKHLU FRGLQJ GHEXJJLQJ DQG WHVWLQJ
H[SHULHQFH /HVVH[SHULHQFHG GHYHORSHUV PD\ RYHUVLJKW VRPH
LPSRUWDQW DVSHFWV RI VRIWZDUH GHVLJQ ZKHQ WKH\ JHW H[FLWHG
VROXWLRQVDUHDOZD\VHDV\WRJHWDSSURYHGIRUPRUHWLPH
 *LYLQJOHVVFRQVLGHUDWLRQWR8,GHVLJQ
7KHSDLURQVHVVLRQGLVFXVVHGVRPHDVSHFWVUHODWHGWR8,
GHVLJQ WKDW OHG WKHP WR DVVXPH FHUWDLQ XVDJH VFHQDULRV DQG
KHOSHGWKHPGHFLGHRQVRPHLQWHUIDFHGHVLJQLVVXHV+RZHYHU
DERXW WU\LQJ VRPH QHZ DSSURDFKHV VXFK DV 33 7KDWV ZK\
LWVLPSRUWDQWWRGLUHFWSDLUVRQWKHLUILUVW33VHVVLRQVDQGJXLGH
WKHP WKURXJKRXW WKH GHYHORSPHQW SURFHVV WR PDNH VXUH WKDW
WKH\ PDLQWDLQ WKH EDVLF JXLGHOLQHV RI VRIWZDUH TXDOLW\
SURFHGXUHVZKLOHZRUNLQJZLWKWKHLUSDUWQHUV
 7KLQNLQJDERXWWKHDVVLJQPHQWDVDWDVNWREHJUDGHG
WKDW SDUW ZDV JLYHQ D YHU\ VPDOO DPRXQW RI WLPH ZKHQ
FRPSDUHG WR DFWXDO LPSOHPHQWDWLRQ WLPH 2Q WKH RWKHU KDQG
VHVVLRQ  GHYHORSHUV GLGQW FRQVLGHU 8, GHVLJQ XQWLO WKH YHU\
HQGRIWKHSURFHVVDQGWKH\GLGQWGLVFXVVXVHUSUHIHUHQFHVRU
DQ\ XVDJH VFHQDULRV $OWKRXJK GLVFXVVLRQ DQG FROODERUDWLRQ
EHWZHHQSDLUVZRXOGOHDGWREHWWHUGHVLJQGHFLVLRQVDVVKRZQ
7KDWSUREOHPZDVFOHDUZKHQGHDOLQJZLWKXQGHUJUDGXDWHV
ZKR ZHUH NHHQ WR IROORZ WKH SUREOHP VSHFLILFDWLRQV DQG
WUDQVODWH WKH YDJXH UHTXLUHPHQWV LQ WKH VDIHVW DQG VLPSOHVW
SRVVLEOH ZD\V 6LQFH WKH\ ZHUHQW H[SRVHG WR GHYHORSPHQW
HQYLURQPHQWV RWKHU WKDQ WKHLU SURJUDPPLQJ FODVVHV DQG
SURMHFWVHYHU\WKLQJIRUWKHPVHHPHGWREHDJUDGHGWDVNWKDW
WKH\ VKRXOG JHW DQ $ LQ LW 7KHUHIRUH WKH\ LJQRUHG DQ\
FRQWH[WXDOLVVXHVUHODWHGWRWKHDVVLJQPHQWZKLOHIRFXVLQJRQO\
RQ WKH FOHDU IXQFWLRQDO UHTXLUHPHQWV VWDWHG RQ WKH SUREOHP
GHILQLWLRQ$OWKRXJKWKDWVHHPHGQRUPDOIRUWKRVHVWXGHQWV33
FDQWVXFFHHGZLWKVXFKFRQVHUYDWLYHZD\RIWKLQNLQJ,I33LV
WR EH DSSOLHG ZLWK XQGHUJUDGV D PRUH FROODERUDWLYH
HQYLURQPHQWVKRXOGEHHQFRXUDJHGZLWKVRPHRWKHUVXSSRUWLQJ
DJLOH SUDFWLFHV ZKLFK SURPRWHV WKH FRQFHSWV RI FROOHFWLYH
RZQHUVKLSRIFRGHDQGZKROHWHDPSDUWLFLSDWLRQ
RQ VHVVLRQ  8, GHVLJQ VKRXOG EH DVVLJQHG PRUH WLPH DQG
VKRXOGEHWUHDWHGFDUHIXOO\E\WKHSDLUV
9, 5(&200(1'$7,216)25,1752'8&,1*3372678'(176
)25&2//$%25$7,9((19,5210(17
2QHRIWKHPDLQDGYDQWDJHVRIFROODERUDWLRQLVWRSURPRWH
WKH FXOWXUH RI DJLOLW\ ZKHUH LQQRYDWLRQ LV FRQVLGHUHG D FRUH
YDOXH UDWKHU WKDQ EHLQJ D VLGH JDLQ %DVHG RQ RXU GLVFXVVLRQ
DERYH ZH DUH UHFRPPHQGLQJ FHUWDLQ SUDFWLFHV WKDW ZLOO KHOS
LQVWUXFWRUV DFKLHYH EHWWHU UHVXOWV ZKHQ WKH\ LQWURGXFH 33 WR
WKHLUSURJUDPPLQJFODVVHV
 )RU WKH ILUVW33 VHVVLRQV VWXGHQWV DUH EHWWHU QRW WREH
SDLUHGUDQGRPO\
,WZLOOEHEHWWHUWRHLWKHUOHWVWXGHQWVVHOHFWWKHLUSDUWQHUVRU
SDLU WKHP EDVHG RQ WKHLU *3$ RU WKHLU JUDGHV RQ SUHYLRXV
 &RQVLGHULQJ RQWLPH VRIWZDUH GHOLYHU\ RYHU SURGXFW SURJUDPPLQJFODVVHV7KLVKRZHYHUFDQEHFKDQJHGLQODWHU
TXDOLW\

130
VHVVLRQV VR WKDW VWXGHQWV FDQ EH SDLUHG ZLWK QHZ SDUWQHUV WR
HQKDQFHNQRZOHGJHWUDQVIHUDPRQJVWXGHQWV
 $VVLJQ WKH ILUVW 33 URXQG WR KLJKOHYHO GHVLJQ DQG
VHVVLRQSODQQLQJ
6WXGHQWVVKRXOGEHHQFRXUDJHGWRVSHQGUHDVRQDEOHDPRXQW
RI WLPH RQO\ IRU VRIWZDUH GHVLJQ 8, KLJKOHYHO GHVLJQ DQG
SODQQLQJIRUWKHLUGHYHORSPHQWWLPHOLQHDQGPLOHVWRQHV/DWHU
VWXGHQWV ZLOO QDWXUDOO\ VWDUW ZRUNLQJ RQ GHVLJQ DQG SODQQLQJ
ZLWKRXW EHLQJ UHTXLUHG WR GR VR DV WKH DGYDQWDJHV RI
VSHQGLQJVRPHWLPHRQWKRVHQRQFRGLQJWDVNVZLOOSRVLWLYHO\
DIIHFWVWKHLUSURJUDPPLQJZRUN
 ,QWURGXFHTXDOLW\DVDEDVLFUHTXLUHPHQWQRWDVDERQXV
:KHQSUHVHQWLQJWKHSUREOHPVWDWHPHQWHQVXUHWKDWTXDOLW\
VKRXOG EH FRQVLGHUHG FDUHIXOO\ E\ GHYHORSHUV DQG LQFOXGH
H[DPSOHVRQKRZXVHUVZLOODVVHVVWKHSURGXFWTXDOLW\4XDOLW\
DWWULEXWHVVKRXOGDGGUHVVGHVLJQFRGHDQG8,
 (PSKDVL]HWKHUROHRIWDONLQJDQGOLVWHQLQJ
HVSHFLDOO\IRUWKHPRELOHDSSGHYHORSPHQW 2XUILUVWVWHS ZDV
WRLQYHVWLJDWHKRZ33FDQEHLQWURGXFHGWRGHYHORSHUVLQVPDOO
FRPSDQLHVRULQSURJUDPPLQJFODVVHVZKLFKLVWKHPDLQVFRSH
RI WKLV SDSHU :H FRPELQHG RXU OLWHUDWXUH UHYLHZ ZLWK H[SHUW
UHYLHZ VHVVLRQV WR FRPH XS ZLWK VRPH SUDFWLFHV WR KHOS
LQWURGXFLQJ 33 WR &6 FODVV VWXGHQWV ZLWK WKH REMHFWLYH RI
KHOSLQJ VWXGHQWV FROODERUDWH LQ D GLVFLSOLQHG \HW IOH[LEOH ZD\
7KHVHSUDFWLFHVZLOOEHH[WHQGHGWRKHOSGHYHORSHUVZRUNLQJRQ
UHDOZRUOGGHYHORSPHQWHQYLURQPHQWV
5()(5(1&(6
>@ : 7LFK\$JLOHGHYHORSPHQWHYDOXDWLRQDQGH[SHULHQFHLQ6RIWZDUH
(QJLQHHULQJ WK ,QWHUQDWLRQDO &RQIHUHQFH RQ 6RIWZDUH (QJLQHHULQJ
,&6(
>@ 0/\FHWW50DFUHGLH&3DWHODQG53DXO0LJUDWLQJDJLOHPHWKRGV
WR VWDQGDUGL]HG GHYHORSPHQW SUDFWLFH &RPSXWHU YRO  1R  
SS
>@ - +LJKVPLWK DQG $ &RFNEXUQ $JLOH VRIWZDUH GHYHORSPHQW WKH
EXVLQHVVRILQQRYDWLRQ&RPSXWHUYRO1RSS
>@ 2 6RKDLE DQG . .KDQ ,QWHJUDWLQJ XVDELOLW\ HQJLQHHULQJ DQG DJLOH
VRIWZDUH GHYHORSPHQW $ OLWHUDWXUH UHYLHZ 7KH ,QWHUQDWLRQDO
33 VHVVLRQ IDFLOLWDWRU VKRXOG PDNH VXUH WKDW SDLUV DUH LQ
FRQWLQXRXVWZRVLGHGFRQYHUVDWLRQV6WXGHQWVZKRDUHIRXQGWR
EH VLOHQW IRU ORQJ WLPHV VKRXOG EH DVNHG IRU WKH UHDVRQV DQG
JXLGHGWRSDUWLFLSDWHLQGLVFXVVLRQVZLWKWKHLUSDUWQHUV
 3UHVHQW WKH DVVLJQPHQW DV D FKDOOHQJLQJ SURJUDPPLQJ
WDVNQRWDVDUHJXODUFODVVDVVLJQPHQW
6WXGHQWV JHW PRWLYDWHG ZKHQ WKH\ IHHO WKH\ DUH VROYLQJ D >@ 6 $PEOHU 7DLORULQJ 8VDELOLW\ LQWR $JLOH 6RIWZDUH 'HYHORSPHQW
UHDO SUREOHP WKDW UHTXLUHV WKHP WR EH QRWLFHDEO\ VPDUW
:RUNLQJLQWHDPVRIWZRPDNHVWKLVIHHOLQJHYHQVWURQJHUZLWK
WKH KLJKHU OHYHOV RI FRPSHWLWLRQ DPRQJ WHDPV 7KHUHIRUH WKH
([WHQGHG$EVWUDFWVRQ+XPDQ)DFWRUVLQ&RPSXWLQJ6\VWHPV
IDFLOLWDWRUVKRXOGVKLIWVWXGHQWVWKLQNLQJIURPZKDWJUDGHZLOO
,JHWIRUWKDWWDVN"WR+RZJRRGLVP\WDVNFRPSDUHGWRWKH >@ 3 +RGJHWWV ([SHULHQFHV LQWHJUDWLQJ VRSKLVWLFDWHG XVHU H[SHULHQFH
UHTXLUHGOHYHORITXDOLW\"7KLVFDQEHGRQHE\SUHVHQWLQJWKH
FXOWXUH RI DJLOH GHYHORSPHQW ZKHUH WHDPV DUH FRPSHWLQJ WR
SUHVHQWWKHKLJKHVWTXDOLW\ZLWKLQWKHDOORZHGWLPHOLPLWV
 6WXGHQWV VKRXOG EH DVNHG WR H[SOLFLWO\ FRQVLGHU XVHU
FRQWH[WDQGXVDJHVFHQDULRV
$VLGHIURP8,LVVXHVVWXGHQWVQHHGWRVSHQGHQRXJKWLPH
GLVFXVVLQJ ZKR ZLOO XVH WKHLU VRIWZDUH KRZ ZLOO XVHUV GHDO
ZLWKLWDQGZKDWDUHWKHQRQIXQFWLRQDOUHTXLUHPHQWVWKDWPD\
&RQIHUHQFHRQ&RPSXWHU'HVLJQDQG$SSOLFDWLRQV,&&'$
>@ -1LHOVHQ7KHXVDELOLW\HQJLQHHULQJOLIHF\FOH&RPSXWHU YRO1R
SS
>@ 6 &KDPEHUODLQ + 6KDUSDQG 10DLGHQ 7RZDUGV D )UDPHZRUN IRU
,QWHJUDWLQJ $JLOH 'HYHORSPHQW DQG 8VHU&HQWUHG 'HVLJQ ([WUHPH
3URJUDPPLQJ DQG $JLOH 3URFHVVHV LQ 6RIWZDUH (QJLQHHULQJ 3
$EUDKDPVVRQ 0 0DUFKHVL DQG * 6XFFL (GLWRUV 6SULQJHU %HUOLQ
+HLGHOEHUJSS
3URMHFWV0DWXULQJ 8VDELOLW\ ( /DZ ( +YDQQEHUJ DQG * &RFNWRQ
(GLWRUV6SULQJHU/RQGRQSS
>@ -& /HH (PEUDFLQJ DJLOH GHYHORSPHQW RI XVDEOH VRIWZDUH V\VWHPV
&+,
$&00RQWUDO4XEHF&DQDGDSS
GHVLJQSUDFWLFHVLQWRDJLOHSURFHVVHV$JLOH&RQIHUHQFH
>@ ' 6\ $GDSWLQJ 8VDELOLW\ ,QYHVWLJDWLRQV IRU $JLOH 8VHUFHQWHUHG
'HVLJQ-RXUQDORI8VDELOLW\6WXGLHVYRO1RSS
>@ +%H\HU8VHU&HQWHUHG$JLOH0HWKRGV6\QWKHVLV/HFWXUHVRQ+XPDQ
&HQWHUHG,QIRUPDWLFVYRO1RSS
>@ -1LHOVHQ$JLOH'HYHORSPHQW3URMHFWVDQG8VDELOLW\>5HWULHYHG
@ $YDLODEOH IURP KWWSZZZQQJURXSFRPDUWLFOHVDJLOH
GHYHORSPHQWDQGXVDELOLW\
>@ =+XVVDLQ:6ODQ\DQG$+RO]LQJHU &XUUHQW6WDWHRI$JLOH8VHU
&HQWHUHG'HVLJQ$6XUYH\3URFHHGLQJVRIWKHWK6\PSRVLXPRIWKH
KDYHQRWEHHQVWDWHGRQWKHSUREOHPGHILQLWLRQ2QRQHVLGHLW
JLYHV WKHP PRUH LQVLJKWV DERXW WKH DSSOLFDWLRQ WKH\ DUH
ZRUNLQJRQ2QWKHRWKHUVLGHWKDWZLOOHQVXUHVRIWZDUHTXDOLW\
DV LW FRQVLGHUV WKH XQFOHDU  \HW LPSRUWDQW  QRQIXQFWLRQDO
UHTXLUHPHQWV 0RUHRYHU VXFK EUDLQVWRUPLQJ ZLOO JHW VWXGHQWV
:RUNJURXS +XPDQ&RPSXWHU ,QWHUDFWLRQ DQG 8VDELOLW\ (QJLQHHULQJ RI
WKH $XVWULDQ &RPSXWHU 6RFLHW\ RQ +&, DQG 8VDELOLW\ IRU H,QFOXVLRQ
6SULQJHU9HUODJ/LQ]$XVWULDSS
>@ 06H\DPDQG*+*DODO(GHHQ7UDGLWLRQDOYHUVXV$JLOH7KH7UDJLOH
)UDPHZRUNIRU,QIRUPDWLRQ6\VWHPVGHYHORSPHQW,QWHUQDWLRQDO-RXUQDO
WRFRPHXSZLWKLQQRYDWLYHLGHDVWKDWPLJKWKDYHQRWDSSHDULI
WKH\ZRUNEDVHGRQWKHJLYHQDVVLJQPHQWGHILQLWLRQRQLWVRZQ
9,,&21&/86,216
,QWHJUDWLQJ XVDELOLW\ ZLWK DJLOLW\ KDV EHHQ
UHFRPPHQGHGE\PDQ\UHVHDUFKHUVDQGSUDFWLWLRQHUVWRDFKLHYH
YDULRXV EHQHILWV +RZHYHU WKH SURSRVHG DSSURDFKHV IRU VXFK
LQWHJUDWLRQ KDYHQW GHDO ZLWK PDQ\ RI LWV SUREOHPV 6LQFH
FROODERUDWLRQDPRQJGHYHORSHUVLVHVVHQWLDOIRUERWKDJLOHDQG
RI6RIWZDUH(QJLQHHULQJ,-6(YRO1RSS
>@ / :LOOLDPV DQG 5 .HVVOHU 3DLU 3URJUDPPLQJ ,OOXPLQDWHG $GGLVRQ
:HVOH\/RQJPDQ3XEOLVKLQJ&R
>@ . %HFN DQG & $QGUHV ([WUHPH 3URJUDPPLQJ ([SODLQHG (PEUDFH
&KDQJHQGHG$GGLVRQ:HVOH\3URIHVVLRQDO
>@ - 6XWKHUODQG 6FUXP 7KH $UW RI 'RLQJ 7ZLFH WKH :RUN LQ +DOI WKH
7LPH&URZQ%XVLQHVV
>@ 0DQLIHVWR IRU $JLOH 6RIWZDUH 'HYHORSPHQW   >5HWULHYHG
@$YDLODEOHIURPKWWSDJLOHPDQLIHVWRRUJ
>@ 3ULQFLSOHV EHKLQG WKH $JLOH 0DQLIHVWR   >5HWULHYHG @
8;WHDPVZHGHFLGHGWRH[SORUHKRZ3DLU3URJUDPPLQJ33
FDQ KHOS LQ FUHDWLQJ D EHWWHU FROODERUDWLYH ZRUN HQYLURQPHQW
IRU GHYHORSHUV :H FODLP WKDW WKH KLJKHU WKH LQWHUDFWLRQ OHYHO
&RQIHUHQFH
DPRQJ GHYHORSHUV XVLQJ 33 WKH EHWWHU 8; WKH\ ZLOO DFKLHYH
$YDLODEOHIURPKWWSDJLOHPDQLIHVWRRUJSULQFLSOHVKWPO
>@ / :LOOLDPV 6 0F&ULFNDUG / /D\PDQ DQG . +XVVHLQ (OHYHQ
*XLGHOLQHV IRU ,PSOHPHQWLQJ 3DLU 3URJUDPPLQJ LQ WKH &ODVVURRP
$*,/(
131