Whole Brain Emulation

A Roadmap

(2008)TechnicalReport#20083 AndersSandberg* NickBostrom FutureofHumanityInstitute FacultyofPhilosophy&JamesMartin21stCenturySchool OxfordUniversity

CITE:Sandberg,A.&Bostrom,N.(2008):WholeBrainEmulation:ARoadmap,TechnicalReport#20083,Futureof HumanityInstitute,OxfordUniversity URL:www.fhi.ox.ac.uk/reports/20083.pdf

(*)Correspondingauthor:anders.sandberg@philosophy.ox.ac.uk

In memoriam: Bruce H. McCormick (1930 2007)

Contents
WholeBrainEmulation............................................................................................................................1 ARoadmap ................................................................................................................................................1 Inmemoriam:BruceH.McCormick(19302007)...........................................................................2 Contents..................................................................................................................................................3 Introduction ...........................................................................................................................................5 Thanksto ............................................................................................................................................6 Theconceptofbrainemulation..........................................................................................................7 Emulationandsimulation...............................................................................................................7 Littleneedforwholesystemunderstanding...............................................................................8 Levelsofemulationandsuccesscriteria.....................................................................................10 Scaleseparation...............................................................................................................................12 Simulationscales.............................................................................................................................13 WBEassumptions ...........................................................................................................................15 Roadmap ..............................................................................................................................................16 Requirements...................................................................................................................................16 Linkages............................................................................................................................................19 Roadmap ..........................................................................................................................................20 Technologydrivers.........................................................................................................................23 Uncertaintiesandalternatives......................................................................................................24 Alternativepathways.....................................................................................................................27 Relatedtechnologiesandspinoffs ..............................................................................................28 Issues.....................................................................................................................................................30 Emulationsystems..........................................................................................................................30 Complicationsandexotica ............................................................................................................31 Summary ..........................................................................................................................................39 Scanning ...............................................................................................................................................40 Embedding,fixationandstainingtechniques ...........................................................................52 Conclusion .......................................................................................................................................53 Imageprocessingandscaninterpretation......................................................................................55 Geometricadjustment....................................................................................................................55 Noiseremoval .................................................................................................................................56 Datainterpolation...........................................................................................................................56 Celltracing .......................................................................................................................................57 Synapseidentification....................................................................................................................59 Identificationofcelltypes .............................................................................................................60 Estimationofparametersforemulation .....................................................................................61 Connectivityidentification............................................................................................................62 Conclusion .......................................................................................................................................63 Neuralsimulation ...............................................................................................................................64 Howmuchneurondetailisneeded?...........................................................................................64 Neuralmodels .................................................................................................................................66 Simulators ........................................................................................................................................70 Parallelsimulation..........................................................................................................................70 Currentlargescalesimulations....................................................................................................71 Conclusion .......................................................................................................................................72 Bodysimulation ..................................................................................................................................74 Conclusion .......................................................................................................................................75 Environmentsimulation....................................................................................................................76

Vision ................................................................................................................................................76 Hearing .............................................................................................................................................77 SmellandTaste ...............................................................................................................................77 Haptics..............................................................................................................................................77 Conclusion .......................................................................................................................................78 Computerrequirements ....................................................................................................................79 Conclusions......................................................................................................................................81 Validation.............................................................................................................................................82 Discussion ............................................................................................................................................83 AppendixA:Estimatesofthecomputationalcapacity/demandsofthehumanbrain ..........84 AppendixB:ComputerPerformanceDevelopment ....................................................................86 ProcessingPower............................................................................................................................86 Memory ............................................................................................................................................95 Discdrives........................................................................................................................................97 Future................................................................................................................................................98 AppendixC:Largescaleneuralnetworksimulations...............................................................101 AppendixD:Historyandpreviouswork.....................................................................................105 AppendixE:Nondestructiveandgradualreplacement...........................................................107 NonDestructiveScanning ..........................................................................................................107 Gradualreplacement....................................................................................................................108 AppendixF:Glossary.......................................................................................................................110 References ..........................................................................................................................................113

Introduction
Wholebrainemulation(WBE),thepossiblefutureonetoonemodellingofthefunctionofthe humanbrain,isacademicallyinterestingandimportantforseveralreasons: Research o Brainemulationisthelogicalendpointofcomputationalneurosciences attemptstoaccuratelymodelneuronsandbrainsystems. o Brainemulationwouldhelpustounderstandthebrain,bothintheleadup tosuccessfulemulationandafterwardsbyprovidinganidealtestbedfor neuroscientificexperimentationandstudy. o Neuromorphicengineeringbasedonpartialresultswouldbeusefulina numberofapplicationssuchaspatternrecognition,AIandbraincomputer interfaces. o Asalongtermresearchgoalitmightbeastrongvisiontostimulate computationalneuroscience. o Asacaseoffuturestudiesitrepresentsacasewherearadicalfuture possibilitycanbeexaminedinthelightofcurrentknowledge. Economics o Theeconomicimpactofcopyablebrainscouldbeimmense,andcouldhave profoundsocietalconsequences(Hanson,1994,2008b).Evenlowprobability eventsofsuchmagnitudemeritinvestigation. Individually o Ifemulationofparticularbrainsispossibleandaffordable,andifconcerns aboutindividualidentitycanbemet,suchemulationwouldenablebackup copiesanddigitalimmortality. Philosophy o Brainemulationwoulditselfbeatestofmanyideasinthephilosophyof mindandphilosophyofidentity,orprovideanovelcontextforthinking aboutsuchideas. o Itmayrepresentaradicalnewformofhumanenhancement. WBErepresentsaformidableengineeringandresearchproblem,yetonewhichappearsto haveawelldefinedgoalandcould,itwouldseem,beachievedbyextrapolationsofcurrent technology.Thisisunlikemanyothersuggestedradicallytransformativetechnologieslike artificialintelligencewherewedonothaveanyclearmetricofhowfarwearefromsuccess. InordertodevelopideasaboutthefeasibilityofWBE,groundtechnologyforesightand stimulateinterdisciplinaryexchange,theFutureofHumanityInstitutehostedaworkshopon May26and27,2007,inOxford.Invitedexpertsfromareassuchascomputational neuroscience,brainscanningtechnology,computing,nanotechnology,andneurobiology presentedtheirfindingsanddiscussedthepossibilities,problemsandmilestonesthatwould havetobereachedbeforeWBEbecomesfeasible. Theworkshopavoideddealingwithsocioeconomicramificationsandwithphilosophical issuessuchastheoryofmind,identityorethics.Whileimportant,suchdiscussionswould undoubtedlybenefitfromamorecomprehensiveunderstandingofthebrainanditwasthis understandingthatwewishedtofocusonfurtheringduringtheworkshop.Suchissueswill likelybedealtwithatfutureworkshops.

Thisdocumentcombinesanearlierwhitepaperthatwascirculatedamongworkshop participants,andadditionssuggestedbythoseparticipantsbefore,duringandafterthe workshop.ItaimsatprovidingapreliminaryroadmapforWBE,sketchingoutkey technologiesthatwouldneedtobedevelopedorrefined,andidentifyingkeyproblemsor uncertainties. Brainemulationiscurrentlyonlyatheoreticaltechnology.Thismakesitvulnerableto speculation,handwavinganduntestableclaims.AsproposedbyNickSzabo,falsifiable designisawayofcurbingtheproblemswiththeoreticaltechnology: thedesignersofatheoreticaltechnologyinanybutthemostpredictableofareas shouldidentifyitsassumptionsandclaimsthathavenotalreadybeentestedina laboratory.Theyshoulddesignnotonlythetechnologybutalsoamapofthe uncertaintiesandedgecasesinthedesignandaseriesofsuchexperimentsandtests thatwouldprogressivelyreducetheseuncertainties.Aproposalthatlacksthis admissionofuncertaintiescoupledwithdesignsofexperimentsthatwillreducesuch uncertaintiesshouldnotbedeemedcredibleforthepurposesofanyimportant decision.Wemightcallthisrequirementarequirementforafalsifiabledesign.(Szabo, 2007) Inthecaseofbrainemulationthiswouldmeannotonlysketchinghowabrainemulator wouldworkifitcouldbebuiltandaroadmapoftechnologiesneededtoimplementit,but alsoalistofthemainuncertaintiesinhowitwouldfunctionandproposedexperimentsto reducetheseuncertainties. Itisimportanttoemphasizethelongtermandspeculativenatureofmanyaspectsofthis roadmap,whichinanycaseistoberegardedonlyasafirstdrafttobeupdated,refined, andcorrectedasbetterinformationbecomesavailable.Giventhedifficultiesand uncertaintiesinherentinthistypeofwork,onemayaskwhetherourstudyisnotpremature. Ourviewisthatwhenthestakesarepotentiallyextremelyhigh,itisimportanttoapplythe bestavailablemethodstotrytounderstandtheissue.Evenifthesemethodsarerelatively weak,itisthebestwecando.Thealternativewouldbetoturnablindeyetowhatcouldturn outtobeapivotaldevelopment.Withoutfirststudyingthequestion,howisonetoformany wellgroundedviewonewayortheotherastothefeasibilityandproximityofaprospectlike WBE?

Thanks to
Wewouldliketowarmlythankthemanypeoplewhohavecommentedonthepaperand helpedextendandimproveit: Workshopparticipants:JohnFiala,RobinHanson,KennethJeffreyHayworth,Todd Huffman,EugeneLeitl,BruceMcCormick,RalphMerkle,TobyOrd,PeterPassaro,Nick Shackel,RandallA.Koene,RobertA.FreitasJrandRebeccaRoache. Otherusefulcomments:StuartArmstrong.

Theconceptofbrainemulation
Wholebrainemulation,ofteninformallycalleduploadingordownloading,hasbeenthe subjectofmuchsciencefictionandalsosomepreliminarystudies(seeAppendixDforhistory andpreviouswork).Thebasicideaistotakeaparticularbrain,scanitsstructureindetail, andconstructasoftwaremodelofitthatissofaithfultotheoriginalthat,whenrunon appropriatehardware,itwillbehaveinessentiallythesamewayastheoriginalbrain.

Emulation and simulation

Thetermemulationoriginatesincomputerscience,whereitdenotesmimickingthefunction ofaprogramorcomputerhardwarebyhavingitslowlevelfunctionssimulatedbyanother program.Whileasimulationmimicstheoutwardresults,anemulationmimicstheinternal causaldynamics(atsomesuitablelevelofdescription).Theemulationisregardedas successfuliftheemulatedsystemproducesthesameoutwardbehaviourandresultsasthe original(possiblywithaspeeddifference).Thisissomewhatsofterthanastrictmathematical definition1. AccordingtotheChurchTuringthesis,aTuringmachinecanemulateanyotherTuring machine.ThephysicalChurchTuringthesisclaimsthateveryphysicallycomputable functioncanbecomputedbyaTuringmachine.Thisisthebasisforbrainemulation:ifbrain activityisregardedasafunctionthatisphysicallycomputedbybrains,thenitshouldbe possibletocomputeitonaTuringmachine.Eveniftrue,however,itdoesnotdemonstrate thatitisacomputationallyfeasibleprocess. Inthefollowing,emulationwillrefertoa1to1modelwhereallrelevantpropertiesofa systemexist,whileasimulationwilldenoteamodelwhereonlysomepropertiesexist. Emulationsmaybehavedifferentlyfromeachotherortheoriginalduetonoiseorintrinsic chaos,butbehavewithintherangeofwhatonewouldexpectfromtheoriginalifithad experiencedthesamenoiseorchaos. Byanalogywithasoftwareemulator,wecansaythatabrainemulatorissoftware(and possiblydedicatednonbrainhardware)thatmodelsthestatesandfunctionaldynamicsofa brainatarelativelyfinegrainedlevelofdetail. Inparticular,amindemulationisabrainemulatorthatisdetailedandcorrectenoughto producethephenomenologicaleffectsofamind. 1AstrictdefinitionofsimulationmightbethatasystemSconsistsofastatex(t)evolvingbyaparticulardynamicsf,
influencedbyinputsandproducingoutputs:x(t+1)=f(I,x(t)),O(t)=g(x(t)).AnothersystemTsimulatesSifitproduces thesameoutput(withinatolerance)forthesameinputtimeseriesstartingwithagivenstate(withinatolerance): X(t+1)=F(I,X(t)),O(t)=G(X(t))where|x(t)X(t)|<1andX(0)=x(0)+2.ThesimulationisanemulationifF=f(uptoa bijectivetransformationofX(t)),thatis,theinternaldynamicsisidenticalandsimilaroutputsarenotduetotheform ofG(X(t)). Chaoticsystemsarenotsimulablebythisdefinition,sinceafterenoughtimetheywilldivergeiftheinitialconditions differ.Sinceevenathreeneuronsystemcanbecomechaotic(Li,Yuetal.,2001)itisveryplausiblethatthebrain containschaoticdynamicsanditisnotstrictlysimulable.However,thereexistsasignificantamountofnoiseinthe brainthatdoesnotpreventmeaningfulbrainstatesfromevolvingdespitetheindeterminacyoftheirdynamics.A softerformofemulationmaybepossibletodefinethathasamodelorparametererrorsmallerthanthenoiselevel andishencepracticallyindistinguishablefromapossibleevolutionoftheoriginalsystem.

Apersonemulationisamindemulationthatemulatesaparticularmind. Whattherelevantpropertiesareisacrucialissue.Intermsofsoftwareemulationthisis oftenthebitsstoredinmemoryandhowtheyareprocessed.Acomputeremulatormay emulatetheprocessor,memory,I/Oandsoonoftheoriginalcomputer,butdoesnotsimulate theactualelectronicworkingsofthecomponents,onlytheirqualitativefunctiononthestored information(anditsinteractionwiththeoutsideworld).Whilelowerlevelemulationof computersmaybepossibleitwouldbeinefficientandnotcontributemuchtothefunctions thatinterestus. Dependingonthedesiredsuccesscriterionemulationmayrequiredifferentlevelsofdetail.It mightalsousedifferentlevelsofdetailindifferentpartsofthesystem.Inthecomputer example,emulatingtheresultofamathematicalcalculationmaynotrequiresimulatingthe executionofalloperatingsystemcallsformathfunctions(sincethesecanbedonemore efficientlybytheemulatingcomputersprocessor)whileemulatingthebehaviourofan analoguevideoeffectmayrequireadetailedelectronicssimulation.

Little need for whole-system understanding

AnimportanthypothesisforWBEisthatinordertoemulatethebrainwedonotneedto understandthewholesystem,butratherwejustneedadatabasecontainingallnecessary lowlevelinformationaboutthebrainandknowledgeofthelocalupdaterulesthatchange brainstatesfrommomenttomoment.Afunctionalunderstanding(whyisaparticularpiece ofcortexorganizedinacertainway)islogicallyseparatefromdetailknowledge(howisit organised,andhowdoesthisstructurerespondtosignals).Functionalunderstandingmaybe apossibleresultfromdetailknowledeanditmayhelpgatheronlytherelevantinformation forWBE,butitisentirelypossiblethatwecouldacquirefullknowledgeofthecomponent partsandinteractionsofthebrainwithoutgaininganinsightintohowtheseproduce(say) consciousnessorintelligence. Evenadatabasemerelycontainingthecompletepartslistofthebrain,includingthe morphologyofitsneurons,thelocations,sizesandtypesofsynapticconnections,wouldbe immenselyusefulforresearch.Itwouldenabledatadrivenresearchinthesamewayas genomicshasdoneinthefieldofcellbiology(Fiala,2002). Computationalneuroscienceattemptsto understandthebrainbymaking mathematicalorsoftwaremodelsofneural systems.Currently,themodelsareusually farsimplerthanthestudiedsystems,with theexceptionofsomesmallneural networkssuchasthelobsterstomatogastric ganglion(NusbaumandBeenhakker,2002) andthelocomotornetworkofthelamprey spinalcord(Kozlov,Lansneretal.,2007). Oftenmodelsinvolveacombinationof simplifiedparts(simulatedneuronsand synapticlearningrules)andnetwork structures(subsamplingofbiological neurons,simpletopologies).Such networkscanthemselvesconstitute Figure1:Understandingoffunctionvs. understandingofdetails.

learningorpatternrecognizingsystemsontheirown,artificialneuralnetworks(ANNs). ANNmodelscanbeusedtoqualitativelymodel,explainandanalyzethefunctionsofbrain systems(Rumelhart,McClellandetal.,1986).Connectionistmodelsbuildmorecomplex modelsofcognitionorbrainfunctiononthesesimplerparts.Theendpointofthispursuit wouldbemodelsthatencompassafullunderstandingofthefunctionofallbrainsystems. Suchqualitativemodelsmightnotexhibitintelligenceorthecomplexityofhumanbehaviour butwouldenableaformalizedunderstandingofhowtheycomeaboutfromsimpleparts. Anotherapproachincomputationalneuroscienceinvolvescreatingmorebiologicallyrealistic models,whereinformationaboutthebiologicaldetailsofneuronssuchastheir electrochemistry,biochemistry,detailedmorphologyandconnectivityareincluded.Atits simplestwefindcompartmentmodelsofindividualneuronsandsynapses,whilemore complexmodelsincludemultiplerealisticneuronsconnectedintonetworks,possiblytaking interactionssuchaschemicalvolumetransmissionintoaccount.Thisapproachcanbeseenas aquantitativeunderstandingofthebrain,aimingforacompletelistofthebiologicalparts (chemicalspecies,neuronmorphologies,receptortypesanddistributionetc.)andmodelling asaccuratelyaspossiblethewayinwhichthesepartsinteract.Giventhisinformation increasinglylargeandcomplexsimulationsofneuralsystemscanbecreated.WBErepresents thelogicalconclusionofthiskindofquantitativemodel:a1to1modelofbrainfunction. Notethattheamountoffunctionalunderstandingneededtoachievea1to1modelissmall. Itsbehaviourisemergentfromthelowlevelproperties,andmayormaynotbeunderstood bytheexperimenters.Forexample,ifcoherentoscillationsareimportantforconceptual bindingandtheseemergefromthelowlevelpropertiesofneuronsandtheirnetworks,a correctandcompletesimulationofthesepropertieswillproducethecoherence. Inpracticecomputationalneuroscienceworksinbetweenquantitativeandqualitative models.Qualitativemodelsareusedtoabstractcomplex,uncertainandpotentiallyirrelevant biologicaldata,andoftenprovidesignificantimprovementsinsimulationprocessing demands(inturnenablinglargersimulations,whichmayenableexplorationofdomainsof moreinterest).Quantitativemodelsaremoreconstrainedbyknownbiology,chemistryand physicsbutoftensufferfromanabundanceoffreeparametersthathavetobeset(Herz, Gollischetal.,2006).Hybridmodelsmayincludepartsusingdifferentlevelsofabstraction, orexistasafamilyofmodelsrepresentingthesamesystematdifferentlevelsofabstraction. Theinterplaybetweenbiologicalrealism(attemptingtobefaithfultobiology),completeness (usingallavailableempiricaldataaboutthesystem),tractability(thepossibilityof quantitativeorqualitativesimulation)andunderstanding(producingacompressed representationofthesalientaspectsofthesysteminthemindoftheexperimenter)willoften determinewhatkindofmodelisused.Theappropriatelevelofabstractionandmethodof implementationdependsontheparticulargoalofthemodel.InthecaseofWBE,thesuccess criteriadiscussedbelowplacelittleemphasisonunderstanding,butmuchemphasison qualitativelycorrectdynamics,requiringmuchbiologicalrealism(uptoapoint,setbyscale separation)andtheneedfordatadrivenmodels.Whethersuchmodelsforwholebrain systemsaretractablefromamodellingandsimulationstandpointisthecrucialissue. Brainemulationcannotbeachievedwithoutsomefunctionalunderstanding.Itneedsmodels andtheoriesforrecognizingwhatdataisrelevant,andwouldprovidedatafordeveloping andtestingthesefurther.WhileintheorybrainemulationmighthugthelowerlineinFigure 1,inpracticeitwilllikelyoccursomewherealongtherightedgestillfarbelowafull understandingofthetoplevelphenomena,butincludingabroadunderstandingofmany

kindsoflowlevelphenomena.Wealsoneedsomeunderstandingofhigherlevelphenomena totestoursimulationsandknowwhatkindofdataweneedtopursue.Fosteringtheright researchcyclefordevelopingtherightunderstanding,collectingdata,improving instrumentation,andexperimentingwithlimitedemulationsinadditiontoproviding usefulservicestorelatedfieldsandbeneficialspinoffswouldbeindispensableforthe developmentofWBE.

Levels of emulation and success criteria

Forthebrain,severallevelsof 6a 6b 6c successcriteriaforemulation Mind emulation Social role-fit Personal emulation identity canbeused.Theyforma emulation hierarchyextendingfrom lowleveltargetstocomplete emulation.SeeTable1on 5 page11. Individual brain emulation Notshowninthishierarchy areemulationsofsubsystems orsmallvolumesofthebrain, 4 partialbrainemulations. Species generic brain emulation Properlyspeaking,a completescan,partslistand braindatabase(1a,1band2) 3 Functional donotconstitutesuccessful brain emulation brainemulation,butsuch achievements(andpartial brainemulations)wouldin 2 anycasebeimportant Brain database milestonesandusefulin themselves. Similarly,thehighlevel achievementsrelatedtosocial 1a 1b Parts List Complete scan roles,mentalstates,and personalidentify(6a,6band 6c)arebothpoorly understoodandhardto Figure2:SuccesslevelsforWBE. operationalize,butgiventhe philosophicalinterestinWBEwehaveincludedthemhereforcompleteness.Itisnotobvious howthesecriteriarelatetooneanother,ortowhatextenttheymightbeentailedbythe criteriafor4and5. Achievingthethirdsuccesscriterionbeyondacertainresolutionwould,assumingsome superveniencethesis,implysuccessofsomeoralloftheothercriteria.Afullquantum mechanicalNbodyorfieldsimulationencompassingeveryparticlewithinabrainwould plausiblysufficeevenifquantummindtheoriesarecorrect.Attheveryleasta1to1 materialcopyofthebrain(asomewhatinflexibleandveryparticularkindofemulating computer)appearstoachieveallcriteria,possiblyexceptingthosefor6c.However,thisis

10

likelyanexcessivelydetailedlevelsincetheparticularphenomenaweareinterestedin(brain function,psychology,mind)appeartobelinkedtomoremacroscopicphenomenathan detailedatomicactivity. Giventhecomplexitiesandconceptualissuesofconsciousnesswewillnotexaminecriteria 6abc,butmainlyexamineachievingcriteria15. Table1:SuccessCriteria

1a Level Partslist Successcriterion Aninventoryofallobjectsonaparticular sizescale,theirpropertiesandinteractions. Acomplete3Dscanofabrainathigh resolution. Combiningthescanandpartslistintoa databasemappingthelowlevelobjectsofa brain. Theemulationsimulatestheobjectsina braindatabasewithenoughaccuracyto produce(atleast)asubstantialrangeof speciestypicalbasicemergentactivityofthe samekindasabrain(e.g.aslowwavesleep stateoranawakestate). Theemulationproducesthefullrangeof (human)speciestypicalemergentbehavior andlearningcapacity. Theemulationproducesemergentactivity characteristicofthatofoneparticular(fully functioning)brain.Itismoresimilartothe activityoftheoriginalbrainthananyother brain. Relevantproperties Lowlevelneuralstructure, chemistry,dynamicsaccurateto resolutionlevel. Resolution,informationenabling structuretofunctionmapping. 1to1mappingofscanto simulation/emulationobjects.

1b 2

Completescan Braindatabase

Functionalbrain emulation

Genericallycorrectcausalmicro dynamics.

Speciesgenericbrain emulation

Individualbrain emulation

6a

Socialrolefit emulation/Person emulation

Theemulationisabletofillandbeaccepted intosomeparticularsocialrole,forexample toperformallthetasksrequiredforsome normallyhumanjob.(Socioeconomic criteriainvolved)

6b

Mindemulation

Theemulationproducessubjectivemental states(qualia,phenomenalexperience)ofthe samekindthatwouldhavebeenproduced bytheparticularbrainbeingemulated. (Philosophicalcriteriainvolved)

Longtermdynamicsand adaptation.Appropriate behaviourresponses.Fullrange learningcapacity. Correctinternalandbehaviour responses.Retainsmost memoriesandskillsofthe particularbrainthatwas emulated.(Inanemulationofan animalbrain,itshouldbe possibletorecognizethe particular(familiar)animal.) Propertiesdependonwhich (rangeof)socialrolesthe emulationwouldbeabletofit.In alimitingcase,theemulation wouldbeabletopassa personalizedTuringtest: outsidersfamiliarwiththe emulatedpersonwouldbe unabletodetectwhether responsescamefromoriginal personoremulation. Theemulationistrulyconscious inthesamewayasanormal humanbeing.

6c

Personalidentity emulation

Theemulationiscorrectlydescribedasa continuationoftheoriginalmind;eitheras numericallythesameperson,orasa survivingcontinuerthereof.(Philosophical criteriainvolved)

Theemulationisanobjectof prudentiallyrationalselfconcern forthebraintobeemulated.

11

Scale separation
Atfirstitmayappearunlikelythatacomplexsystemwithmanydegreesoffreedomlikethe braincouldbemodelledwiththerightcausaldynamics,butwithouttakingintoaccountthe smallestparts.Microstimulationofindividualneuronscaninfluencesensorydecisions (HouwelingandBrecht,2008),showingthatverysmalldisturbancescanundertheright circumstancesscaleuptobehaviouraldivergences. However,statevariablesofcomplexsystemscanbequantitativelypredictedwhenthereis scaleseparation:whendifferentaspectsofthesystemexistonsufficiently(ordersof magnitude)differentscales(ofsize,energy,timeetc),theycanbecomeuncoupled (Hillerbrand,2008).Atypicalexampleishowthemicroscopicdynamicsofalaser(atoms interactingwithanoscillatingelectromagneticfield)givesrisetoamacroscopicdynamics (thegrowthanddecayofdifferentlasermodes)insuchawaythatanaccuratesimulationof thesystemusingonlyelementsonthemacroscaleispossible.Anotherexampleisthescale separationbetweenelectriccurrentsandlogicoperationsinacomputer,whichenablesbit basedemulation.Whenthereisnoscaleseparation(suchasinfluidturbulence)macroscale predictionsbecomeimpossiblewithoutsimulatingtheentiremicroscale. Animportantissuetobedeterminediswhethersuchacutoffexistsinthecaseofthehuman brainand,ifitdoesexist,atwhatlevel.Whilethispaperphrasesitintermsof simulation/emulation,itisencounteredinarangeoffields(AI,cognitiveneuroscience, philosophyofmind)inotherforms:whatleveloforganisationisnecessaryforintelligent, personal,orconsciousbehaviour? AkeyassumptionofWBEisthat,atsomeintermediarylevelofsimulationresolution betweentheatomicandthemacroscopic,thereexistsatleastonecutoffsuchthatmeeting criteria1aand1batthislevelofresolutionalsoenablesthehighercriteriatobemet. Atsuchaspatial,temporal,ororganisationalscale,thedynamicsonthelarger/slowerscaleis notfunctionallysensitivetothedynamicsofthesmaller/fasterscale.Suchscaleseparation mightoccuratthesynapticscale,wherethedetailedchemicaldynamicsunderlyingsynaptic functioncouldbereplacedbyasimplifiedqualitativemodelofitseffectsonsignalsand synapticstrengths.Anotherpossiblescaleseparationlevelmightoccurbetweenindividual moleculesandmolecularconcentrationscales:moleculardynamicscouldbereplacedwith massactioninteractionsofconcentrations.Aperhapslesslikely separationcouldalsooccuronhigherlevelsifwhatmattersisthe activityofcorticalminicolumnsratherthanindividualneurons.A finallikelybutcomputationallydemandingscaleorseparation wouldbetheatomicscale,treatingthebrainemulationasaNbody systemofatoms. Conversely,ifitcouldbedemonstratedthatthereisnosuchscale,it woulddemonstratetheinfeasibilityofwholebrainemulation.Due tocausallyimportantinfluencefromsmallerscalesinthiscase,a simulationataparticularscalecannotbecomeanemulation.The causaldynamicsofthesimulationisnotinternallyconstrained,soit isnota1to1modeloftherelevantdynamics.Biologically interestingsimulationsmightstillbepossible,buttheywouldbe localtoparticularscalesandphenomena,andtheywouldnotfully reproducetheinternalcausalstructureofthewholebrain. Figure3:Sizescalesin thenervoussystem.

12

Simulation scales
Thewidelyreproduceddiagramfrom(ChurchlandandSejnowski,1992)inFigure3depicts thevariouslevelsoforganisationinthenervoussystemorderedbysizescale,runningfrom themolecularleveltotheentiresystem.Simulations(andpossiblyemulations)canoccuron alllevels: Molecularsimulation(individualmolecules) Chemistrysimulation(concentrations,lawofmassaction) Geneticexpression Compartmentmodels(subcellularvolumes) Wholecellmodels(individualneurons) Localnetworkmodels(replacesneuronswithnetworkmodulessuchas minicolumns) Systemmodels AnotherhierarchywasintroducedbyJohnFialaduringtheworkshop,andwillbeusedwith someadaptationsinthisdocument. Table2:Levelsofemulation
Level 1 2 3 Computational module Brainregion connectivity Analognetwork populationmodel ClassicAI,highlevelrepresentationsofinformationandinformation processing. Eacharearepresentsafunctionalmodule,connectedtoothersaccording toa(speciesuniversal)connectome(Sporns,Tononietal.,2005). Neuronspopulationsandtheirconnectivity.Activityandstatesof neuronsorgroupsofneuronsarerepresentedastheirtimeaverages.This issimilartoconnectionistmodelsusingANNs,ratemodelneural simulationsandcascademodels. Asabove,plusfiringproperties,firingstateanddynamicalsynapticstates. Integrateandfiremodels,reducedsinglecompartmentmodels(butalso someminicolumnmodels,e.g.(JohanssonandLansner,2007)). Asabove,plusmembranestates(ionchanneltypes,properties,state),ion concentrations,currents,voltagesandmodulationstates.Compartment modelsimulations. Asabove,plusconcentrationsofmetabolitesandneurotransmittersin compartments. Asabove,plusconcentrationsofproteinsandgeneexpressionlevels. Asabove,plusquaternaryproteinstructure. Asabove,pluslocomeinformationandinternalcellulargeometry. Asaboveplusmoleculepositions,oramolecularmechanicsmodelofthe entirebrain. Quantuminteractionsinandbetweenmolecules.

Spikingneural network Electrophysiology

6 7 8 9 10 11

Metabolome Proteome Statesofprotein complexes Distributionof complexes Stochasticbehaviour ofsinglemolecules Quantum

Theamountofunderstandingneededtoaccuratelysimulatetherelevantobjectstendsto increaseradicallyforhigher(here,lownumbered)levels:whilebasicmechanicsiswell understood,membranebiophysicsiscomplex,andthecomputationalfunctionsofbrainareas arelikelyexceedinglymultifaceted.Conversely,theamountofcomputingpowerneeded increasesrapidlyaswedescendtowardslowerlevelsofsimulation,andmaybecome fundamentallyinfeasibleonlevel112.Theamountandtypedataneededtofullyspecifya

Butseealsothefinalchapterof(Hameroff,1987).Themainstumblingblockoflevel11simulationmaynotbe computinghardwareorunderstandingbutfundamentalquantumlimitationsonscanning.
2

13

modelalsochangescharacterbetweenthedifferentlevels.Lowlevelsimulationsrequire massivequantitiesofsimpleinformation(molecularpositionsandtypes)whereashigher levelsrequireasmalleramountofverycomplexinformation(contentofmentalprocesses). Eachlevelhasitsowncharacteristicsizeandtimescale,restrictingtherequiredimaging resolutionandsimulationtimestep.Forexample,synapticspinenecksandthethinnestaxons canbeontheorderof50nmorsmaller,requiringimagingontheorderofthe5nanometer scaletoresolvethem. Aninformalpollamongworkshopattendeesproducedarangeofestimatesoftherequired resolutionforWBEis.Theconsensusappearedtobelevel46.Twoparticipantsweremore optimisticabouthighlevelmodels,whiletwosuggestedthatelementsonlevel89maybe necessaryatleastinitially(butthatthebulkofmatureemulation,oncethebasicswere understood,couldoccuronlevel45).Toachieveemulationonthislevel,theconsensuswas that5550nmscanningresolutionwouldbeneeded.Thisroadmapwillhencefocusonlevel 46models,whilebeingopenforthatdeeperlevelsmayturnouttobeneeded. AsnotedbyFiala,WBElikelyrequiresatleastlevel4tocapturethespecificityofindividual brains,butprobablyrequirescomplexityatlevel6orlowertofullycapturethe computationalcontributionsofionchannels,secondmessengers,proteinleveladaptation, andstochasticsynaptictransmission.Otherparticipantsthoughtthatatleastlevel5wouldbe neededforindividualbrainproperties.

Forecasting
Analysingtherequirementsforemulation(intermsofscanningmethod,numberofentities tosimulate,resultingstoragerequirementsandcomputationaldemands)ateachofthelevels providesawayofboundingprogresstowardsWBE.Giventheseestimatesandscenariosof futureprogressinscanningandcomputingitispossibletocalculatetheearliestpointintime wherethereisenoughresourcestoproduceaWBEonagivenlevelatacertainprice.As betterinformationbecomesavailablesuchestimatescanberefined. Althoughanytimeestimateswillbesubjecttostronguncertainties,theycanbehelpfulin estimatinghowfarawayWBEisfrompolicyrelevanttimescales,aswellaslikelytimeframes forearlysmallscaleemulations.Theyalsoallowcomparisonstoothertechnologyforecasts, enablingestimationofthechancesforsynergies(e.g.thedevelopmentofmolecular nanotechnology,whichwouldaccelerateWBEprogress),reliability(e.g.thefurtherintothe future,themoreunlikelyMooreslawistohold),andtheriskofothertechnologies overtakingWBE(e.g.artificialintelligence). EarlyWBEmayrequirelowerlevelsimulationthanlaterforms,astheremightnotyetbe enoughexperience(andtestsystems)todeterminewhichsimulationelementsarestrictly necessaryforsuccess.ThemainconcerninthisdocumentisestimatingwhenandhowWBE willfirstbeachievedratherthanitseventualmatureorbestform.

14

WBE assumptions
Philosophicalassumptions
Physicalism(everythingsupervenesonthephysical)isaconvenientbutnotnecessary assumption,sincesomenonphysicalisttheoriesofmentalpropertiescouldallowthemto appearinthecaseofWBE.Successcriterion6bemulationassumesmultiplerealizability (thatthesamementalproperty,state,oreventcanbeimplementedbydifferentphysical properties,states,andevents).SufficientapparentsuccesswithWBEwouldprovide persuasiveevidenceformultiplerealizability.Generally,emulationuptoandincludinglevel 6adoesnotappeartodependonanystrongmetaphysicalassumptions.

Computationalassumptions
Computability:brainactivityisTuringcomputable,orifitisuncomputable,the uncomputableaspectshavenofunctionallyrelevanteffectsonactualbehaviour. Nonorganicism:totalunderstandingofthebrainisnotrequired,justcomponentpartsand theirfunctionalinteractions. Scaleseparation:atsomeintermediarylevelofsimulationresolutionbetweentheatomicand themacroscopicthereexistsone(ormore)cutoffssuchthatmeetingcriterion2atthislevelis sufficientformeetingoneormoreofthehighercriteria. Componenttractability:theactualbraincomponentsatthelowestemulatedlevelcanbe understoodwellenoughtoenableaccuratesimulation. Simulationtractability:simulationofthelowestemulatedleveliscomputationallytractable withapracticallyrealizablecomputer.

Neuroscienceassumptions
Braincenteredness:inordertoproduceaccuratebehaviouronlythebrainandsomepartsof thebodyneedtobesimulated,nottheentirebody. WBEappearstobeawayoftestingmanyoftheseassumptionsexperimentally.Inacquiring accuratedataaboutthestructureandfunctionofthebrainandrepresentingitasemulations itshouldbepossibletofindmajordiscrepanciesif,forexample,Computabilityisnottrue.

15

Roadmap
Requirements
WBErequiresthreemaincapabilities:theabilitytophysicallyscanbrainsinordertoacquire thenecessaryinformation,theabilitytointerpretthescanneddatatobuildasoftwaremodel, andtheabilitytosimulatethisverylargemodel.Theseinturnrequireanumberof subcapabilities(Table3:CapabilitiesneededforWBE). Plausiblescanningmethodsrequirewaysofpreparingthebrains,inparticularseparation fromothertissue,fixationandpossiblydyeing.Thereisalsoaneedformethodsofphysically handlingandstoringpiecesoftissue:sincemostscanningmethodscannotimagelarge volumesthebrainswillhavetobesectionedintomanageablepieces.Thismustallow correspondingcellsanddendritestobeidentifiedonbothsides.Whilefixationand sectioningmethodsarecommonlyusedinneuroscience,thedemandsforwholebrain emulationarestricter:muchlargervolumesmustbehandledwithfarlesstolerancefor damage. Imagingmethodsarediscussedinmoredetailinthechapteronscanning.Thethreekey issuesareachievingthenecessaryresolutiontoimagethesmallestsystemsneededforan emulation,theabilitytoimage(notnecessarilysimultaneously)theentirebrain,andthe abilitytoacquirethefunctionallyrelevantinformation. Translatingthedatafromtheimagingprocessintosoftwarerequiressophisticatedimage processing,theabilitytointerprettheimageryintosimulationrelevantparameters,and havingacomputationalneurosciencemodelofsufficientprecision.Theimageprocessingwill havetodealwiththeunavoidableartefactsfromscanningsuchasdistortionsandnoise,as wellasoccasionallostdata.Itwilllikelyincludemethodsofconvertingdirectscandatainto morecompressedforms,suchastracedstructures,inordertoavoidexcessivestorageneeds. Thescaninterpretationprocessmakesuseofthisdatatoestimatetheconnectivity,andto identifysynapticconnections,celltypesandsimulationparameters.Itthenplacesthis informationinaninventorydatabasefortheemulation.Thesestepsarediscussedinthe imageprocessingchapter. Thesoftwaremodelrequiresbothamathematicalmodelofneuralactivityandwaysof efficientlyimplementingsuchmodelsoncomputers(discussedinthechapteronneural simulation).Computationalneuroscienceaimsatmodellingthebehaviourofneuralentities suchasnetworks,neurons,synapsesandlearningprocesses.ForWBE,itneedstohave sufficientlygoodmodelsofallrelevantkindsofsubsystems,alongwiththerelevant parameterssetfromscandatainordertoconstructacomputationalmodeloftheactualbrain thatwasscanned. Toemulateabrain,weneedenoughcomputingpowertorunthebasicemulationsoftware,a sufficientlyrealisticbodysimulation,andpossiblyasimulatedenvironment.Thekey demandsareformemorystoragetoholdtheinformationandprocessorpowertorunitata suitablespeed.Themassiveparallelismoftheproblemwillputsomesignificantdemandson theinternalbandwidthofthecomputingsystem. Inaddition,WBElikelyrequiresthedevelopmentofthreesupportingtechnologyareas,with whichithasasymbioticrelationship.First,validationmethodstocheckthatothersteps

16

produceaccuratedataandmodels.Thisincludesvalidationofscanning,validationofscan interpretation,validationofneurosciencemodels,validationsofimplementation,andwaysof testingthesuccessofWBE.Whileordinaryneuroscienceresearchcertainlyaimsatvalidation, itdoesnotsystematizeit.ForacomplexmultistepresearcheffortlikeWBE,integrated validationislikelynecessarytoensurethatbaddataormethodsdonotconfuselaterstepsin theprocess.Second,WBErequiressignificantlowlevelunderstandingofneurosciencein ordertoconstructthenecessarycomputationalmodelsandscaninterpretationmethods.This isessentiallyacontinuationandstrengtheningofsystemsbiologyandcomputational neuroscienceaimingataverycompletedescriptionofthebrainonsomesizeorfunctional scale.Third,WBEislargescaleneuroscience,requiringmethodsofautomating neuroscientificinformationgatheringandexperimentation.Thiswillreducecostsand increasethroughput,andisnecessaryinordertohandlethehugevolumesofdataneeded. Largescale/industrialneuroscienceisclearlyrelevantforotherneuroscienceprojectstoo.

17

Preparation

Scanning

Physical handling

Resolution

Imaging

Volume

Functional information

Geometric adjustment Image processing Data interpolation

Noise removal

Tracing

Whole brain emulation

Mathematical model Translation Software model of neural system Efficient implementation

Parameter estimation Synapse identification Scan interpretation Connectivity identification Cell type identification Databasing

Environment simulation CPU Simulation Body Simulation Bandwidth Storage

Figure4:TechnologicalcapabilitiesneededforWBE.

18

 Table3:CapabilitiesneededforWBE
Preprocessing/fixation Preparingbrainsappropriately, retainingrelevant microstructureandstate Methodsofmanipulatingfixed brainsandtissuepiecesbefore, during,andafterscanning Capabilitytoscanentirebrain volumesinreasonabletime andexpense. Scanningatenoughresolution toenablereconstruction Scanningisabletodetectthe functionallyrelevantproperties oftissue Handlingdistortionsdueto scanningimperfection Handlingmissingdata Improvingscanquality Detectingstructureand processingitintoaconsistent 3Dmodelofthetissue Identifyingcelltypes Identifyingsynapsesandtheir connectivity Estimatingfunctionally relevantparametersofcells, synapses,andotherentities Storingtheresultinginventory inanefficientway Modelofentitiesandtheir behaviour Implementationofmodel Storageoforiginalmodeland currentstate Efficientinterprocessor communication Processorpowertorun simulation Simulationofbodyenabling interactionwithvirtual environmentorthroughrobot Virtualenvironmentforvirtual body

Physicalhandling Scanning

Volume Imaging Resolution Functionalinformation Geometricadjustment Imageprocessing Datainterpolation Noiseremoval Tracing Celltypeidentification

Translation Scaninterpretation

Synapseidentification Parameterestimation Databasing Softwaremodelofneural system Storage Bandwidth Mathematicalmodel Efficientimplementation

Simulation

CPU Bodysimulation Environmentsimulation

Linkages
MostofthecapabilitiesneededforWBEareindependentofeachother,orformsynergistic clusters.Clustersoftechnologiesdeveloptogether,supportingandmotivatingeachother withtheiroutput.Atypicalexampleisbettermathematicalmodelsstimulatinganeedfor betterimplementationsandcomputingcapacity,whileimprovementsinthelattertwo stimulateinterestinmodelling.Anotherkeyclusteris3Dmicroscopyandimageprocessing, whereimprovementsinonemakestheothermoreuseful. Therearefewclearcaseswhereacapabilityneedsacompletedearliercapabilityinorderto begindevelopment.Currentfixationandhandlingmethodsarelikelyunabletomeetthe

19

demandsofWBElevel3Dmicroscopy,butaregoodenoughtoenableearlydevelopmentfor certainsmallsystems.Scaninterpretationneedsenoughscandatatodevelopmethods,but givencurrentresearchthebottleneckappearstobemoreontheimageprocessingand interpretationsidethandataavailability.Achievinglargevolumescanningrequires parallelizationandscalinguppreviousscanningmethods,forexamplebyusingroboticwork andparallelmicroscopy.Thisrequiresresearchersthinkingintermsof,andhaving experiencewith,anindustrialapproachtodatacollection. Thisinterlinkednatureofthefieldavoidsanyobvioustechnologythresholdsand bottlenecks.Thereisnoonetechnologythatmustbedevelopedbeforeothertechnologiescan advance.Developmentcanoccuronabroadfrontsimultaneously,andrapidprogressina fieldcanpromotefeedbackprogressinrelatedfields.Unfortunately,italsomeansthatslow progressinoneareamayholdbackotherareas,notjustduetolackofresultsbutbyreduced demandfortheirfindings,reducedfunding,andfocusonresearchthatdoesnotleadinthe directionofWBE.

Roadmap
Basedontheseconsiderationswecansketchoutaroadmapwithmilestones,required technologies,keyuncertaintiesandexternaltechnologyinteractions. ApproachtoWBEhastwophases.Thefirstphaseconsistsofdevelopingthebasiccapabilities andsettlingkeyresearchquestionsthatdeterminethefeasibility,requiredlevelofdetailand optimaltechniques.Thisphasemainlyinvolvespartialscans,simulationsandintegrationof theresearchmodalities. Thesecondphasebeginsoncethecoremethodshavebeendevelopedandanautomated scaninterpretationsimulatepipelinehasbeenachieved.Atthispointthefirstemulations becomepossible.Ifthedevelopedmethodsprovetobescalabletheycanthenbeappliedto increasinglycomplexbrains.Herethemainissueisscalinguptechniquesthathavealready beenprovenonthesmallscale.

20

Human emulation

Large mammal emulation

Validation methods

Ground truth models

Appropriate level Small mammal emulation

Deducing function Invertebrate emulation Scanning development Complete inventory Partial emulations Interpretation development Automated pipeline Eutelic organism emulation

Low-level neuroscience

Organism simulation

Full cell simulation

Body simulation

Simulation hardware

Figure5:WBEroadmap. Thekeymilestonesare: Groundtruthmodels:asetofcaseswherethebiologicalgroundtruthisknownandcanbe comparedtoscans,interpretationsandsimulationsinordertodeterminetheiraccuracy. Determiningappropriatelevelofsimulation:thisincludesdeterminingwhetherthereexists anysuitablescaleseparationinbrains(ifnot,theWBEeffortmaybeseverelylimited),andif so,onwhatlevel.Thiswouldthenbetherelevantscaleforscanningandsimulation. Fullcellsimulation:acompletesimulationofacellorsimilarlycomplexbiologicalsystem. WhilestrictlynotnecessaryforWBEitwouldbeatestcaseforlargescalesimulations.

21

Bodysimulation:anadequatesimulationforthemodelanimalsbody(andenvironment). Ideallydemonstratedbyfoolingarealanimalconnectedtoit. Simulationhardware:specialpurposesimulation/emulationcomputerhardwaremaybe foundtobenecessaryoreffective. Organismsimulation:asimulationofanentireorganismintermsofneuralcontrol,body stateandenvironmentalinteraction.Thiswouldnotbeatrueemulationsinceitisnotbased onanyindividualbutratherknownphysiologicaldataforthespecies.Thiswouldenable morerealisticandindividualmodelsasscans,modelsandcomputerpowerimproves. Demonstrationoffunctiondeduction:demonstratingthatallrelevantfunctionalproperties onalevelcanbededucedfromscandata. Completeinventory:acompletedatabaseofentitiesatsomelevelofresolutionforaneural system,e.g.notjusttheconnectivityoftheC.elegansnervoussystembutalsothe electrophysiologyofthecellsandsynapses.Thiswouldenablefullemulationifalltheupdate rulesareknown.Itdemonstratesthatthescanningandtranslationmethodshavematured. Automatedpipeline:asystemabletoproduceasimulationbasedonaninputtissuesample, goingthroughthescan,interpretationandsimulationstepswithoutmajorhuman intervention.Theresultingsimulationwouldbebasedontheparticulartissueratherthan beingagenericmodel. Partialemulation:Acompleteemulationofneuralsystemsuchastheretina,invertebrate gangliaoraV1circuitbasedonscannedandinterpreteddatafromabrainratherthanspecies data.Thiswoulddemonstratethefeasibilityofdatadrivenbrainemulation. Eutelicorganismemulation:acompleteemulationofasimpleorganism,suchasC.elegansor anothereutelic(fixednervoussystem)organismusingdatafrompipelinescanning.Itmay turnoutthatitisunnecessarytostartwithaeutelicorganismandthefirstorganism emulationwouldbeamorecomplexinvertebrate. InvertebrateWBE:Emulationofaninvertebratesuchasasnailoraninsect,withlearning. ThiswouldtestwhethertheWBEapproachcanproduceappropriatebehaviours.Ifthe scannedindividualwastrainedbeforescanning,retentionoftrainedresponsescanbe checked. SmallmammalWBE:DemonstrationofWBEinmiceorrats,provingthattheapproachcan handlemammalianneuroanatomy. LargemammalWBE:Demonstrationinhighermammals,givingfurtherinformationabout howwellindividuality,memoryandskillsarepreservedaswellasinvestigationofsafety concerns. HumanWBE:Demonstrationofaninteractivehumanemulation.

22

Technology drivers
Preparation

Scanning

Physical handling

Resolution

Imaging

Volume

Functional information

Geometric adjustment Image processing Data interpolation

Noise removal

Tracing

Whole brain emulation

Mathematical model Translation Software model of neural system Efficient implementation

Parameter estimation Synapse identification Scan interpretation Connectivity identification Cell type identification Databasing

Environment simulation CPU Simulation Body Simulation Bandwidth Storage

Moores law driven

Commercial drivers

Research drivers

WBE specific?

Figure6:TechnologydriversforWBEnecessarytechnologies. Differentrequiredtechnologieshavedifferentsupportanddriversfordevelopment.

23

 Computersaredevelopedindependentlyofanyemulationgoal,drivenbymassmarket forcesandtheneedforspecialhighperformancehardware.Mooreslawandrelated exponentialtrendsappearlikelytocontinuesomedistanceintothefuture,andthefeedback loopspoweringthemareunlikelytorapidlydisappear(seefurtherdiscussioninAppendixB: ComputerPerformanceDevelopment).Thereisindependent(andoftensizeable)investment intocomputergames,virtualreality,physicssimulationandmedicalsimulations.Like computers,thesefieldsproducetheirownrevenuestreamsanddonotrequireWBEspecific orscientificencouragement. Alargenumberoftheothertechnologies,suchasmicroscopy,imageprocessing,and computationalneurosciencearedrivenbyresearchandnicheapplications.Thismeansless funding,morevariabilityofthefunding,anddependenceonsmallergroupsdeveloping them.Scanningtechnologiesaretiedtohowmuchmoneythereisinresearch(including brainemulationresearch)unlessmedicalorotherapplicationscanbefound.Validation techniquesarenotwidelyusedinneuroscienceyet,butcould(andshould)becomestandard assystemsbiologybecomesmorecommonandwidelyapplied. FinallythereareafewareasrelativelyspecifictoWBE:largescaleneuroscience,physical handlingoflargeamountsoftissueblocks,achievinghighscanningvolumes,measuring functionalinformationfromtheimages,automatedidentificationofcelltypes,synapses, connectivityandparameters.Theseareasaretheonesthatneedmostsupportinorderto enableWBE. Thelattergroupisalsothehardesttoforecast,sinceithasweakdriversandasmallnumber ofresearchers.Thefirstgroupiseasiertoextrapolatebyusingcurrenttrends,withthe assumptionthattheyremainunbrokensufficientlyfarintothefuture.

Uncertainties and alternatives

Themainuncertaintiesintheroadmapare: DoesscaleseparationenablingWBEoccur?Thisisabasicsciencequestion,andifscale separationdoesnotoccuratasufficientlyhighscalelevelthenWBEwouldbeseverely limitedorinfeasible.Itispossiblethatprogressinunderstandingcomplexsystemsingeneral willhelpclarifythesituation.Thequestionofwhichcomplexsystemsaresimulableunder whichconditionsisofgeneralinterestformanyfields.However,inrelationtoWBEthe answerseemsmostlikelytocomefromadvancesincomputationalneuroscienceandfrom trialanderror.Byexperimentingwithvariousspecializedneuralemulations,atdifferent levelsofresolution,andcomparingthefunctionallyrelevantcomputationalpropertiesofthe emulationwiththoseoftheemulatedsubsystem,wecantestwhetheragivenemulationis successful.Ifso,wecaninferthatasufficientscaleseparationforthatsubsystemexistsat(or above)thescalelevel(granularity)usedintheemulation.Weemphasizethatasuccessful emulationneednotpredictalldetailsoftheoriginalbehavioroftheemulatedsystem;itneedonly replicatecomputationallyrelevantfunctionalityatthedesiredlevelofemulation Whatlevelsarepossible/mostappropriateforemulation?Thiswilldetermineboththe requirementsforscanningandemulationsoftware.Inordertodiscoverit,smallscale scanningandsimulationprojectsneedtobeundertaken,developingskillsandmethods. Later,fullscaleemulationsofsmallsystemswilltestwhethertheestimateshold.Ifanearly answercanbefound,effortscanfocusonthislevel;otherwisetheWBEresearchfrontwould havetoworkonmultiplelevels.

24

 Howmuchofthefunctionallyrelevantinformationcanbededucedfromscanningina particularmodality(e.g.electronmicroscopy)?Atpresent,electronmicroscopyappearsto betheonlyscanningmethodthathastherightresolutiontoreachsynapticconnectivity,but itislimitedinwhatchemicalstateinformationitcanreveal.Ifitispossibletodeducethe functionofaneuron,synapseorotherstructurethroughimageinterpretationmethods,then scanningwouldbefarsimplerthanifthisisnot(inwhichcasesomeformofhybridmethod orentirelynewscanningmodalitywouldhavetobedeveloped).Thisissueappearstoforma potentiallywelldefinedresearchquestionthatcouldbepursued.Answeringitwouldrequire findingasuitablemodelsystemforwhichgroundtruth(thecomputationalfunctionalityof targetsystem)wasknown,usingthescanningmodalitytoproduceimageryandthentesting outvariousformsofinterpretationonthedata.
Cellular data and hypotheses Qualitative modelling Quantitative modelling

Wet experiments

Large-scale scanning

Cell programming

Simulation Interpretation Analysis Large-scale simulation

Figure7:WBEcomputationalbiologyresearchcycle(basedon(Takahashi,Yugietal. 2002)).WBEintroduceslargescalescanningandsimulationintothecycle. Developingtherightresearchcycle.Thecomputationalbiologyresearchcycletoday involveswetexperimentsprovidingcellulardataandhypotheses,whichdrivequalitative modelling.Thismodellinginturnisusedinquantitativemodelling,whichusingsimulations generatedatathatcanbeanalysedandemployedtorefinethemodels,comparewiththe experiments,andsuggestnewexperiments(Takahashi,Yugietal.,2002).TheWBEparadigm incorporatesthisresearchcycle(especiallyinthesoftwaremodellingpart),butincludestwo newfactors.Oneislargescalescanningandprocessingofbraintissue,providingmassive amountsofdataasinputtothecycle,butalsorequiringthemodelstoguidethedevelopment ofscanningmethods.Thesecondisthelargescaledatadrivensimulationsthatdonotaimat producingjusthypothesistestingandmodelrefinement,butalsoataccuratelymimickingthe wetsystem.Bothfactorswillincreaseandchangethedemandsfordatamanagement, hypothesissearchingandsimulationanalysis/interpretation.Theywillalsointroduce sociologicalandinterdisciplinaryfactors,asdifferentacademicdisciplineswithverydifferent methodologieswillhavetolearnhowtocommunicateandcooperate.Inordertobeviable, fieldresearchmethodsoftesting,datasharing,validation,andstandardsforwhatconstitutes aresultmustbedevelopedsothattheextendedcycleprovidesincentivesforallparticipants tocooperateandpushthetechnologyforward.Thisiscloselylinkedtothelikelymove

25

towardslargescaleneuroscience,whereautomatedmethodswillplayanincreasingly prominentroleastheyhavedoneingenomics.

Figure8:Caenorhabditiselegans,apopularmodelorganismwithafullymapped302 neuronnervoussystem. Selectionofsuitablemodelsystems.Selectingtherighttargetsforscanningandmodelling willhavetotakeintoaccountexistingknowledge,existingresearchcommunities,likelihood offundingandacademicimpactaswellaspracticalfactors.WhiletheC.elegansnervous systemhasbeencompletelymapped(White,Southgateetal.,1986),westilllackdetailed electrophysiology,likelybecauseofthedifficultyofinvestigatingthesmallneurons.Animals withlargerneuronsmayprovelessrestrictiveforfunctionalandscanninginvestigationbut maylacksizeableresearchcommunities.Molluscssuchasfreshwatersnailsandinsectssuch asfruitfliesmaybepromisingtargets.Theyhavewellcharacterisedbrains,existingresearch communitiesandneuralnetworkswellwithincurrentcomputationalcapabilities. Similarly,theselectionofsubsystemsofthebraintostudyrequirescarefulconsideration. Someneuralsystemsareheavilystudied(corticalcolumns,thevisualsystem,the hippocampus)andbetterdataaboutthemwouldbewarmlyreceivedbytheresearch community,yetthelackofcharacterizationoftheirinputs,outputsandactualfunctionmay makedevelopmentofemulationmethodsveryhard.Onesystemthatmaybeverypromising istheretina,whichhasanaccessiblegeometry,iswellstudiedandsomewhatwell understood,isnotexcessivelycomplex,andbetterinsightsintowhichwouldbeusefultoa wideresearchcommunity.Buildingonretinalmodels,modelsofthelateralgeniculate nucleusandvisualcortexmaybeparticularlyuseful,sincetheywouldbothhaverelatively welldefinedinputsfromthepreviousstages. AtwhatpointwillthepotentialbeclearenoughtobringmajoreconomicactorsintoWBE? GiventhepotentiallyhugeeconomicimpactofhumanWBE(Hanson,1994,2008a,2004, 2008b),ifthefieldshowssufficientpromise,majoreconomicactorswillbecomeinterestedin fundinganddrivingtheresearchasaninvestment.Itisunclearhowfaradvancedthefield wouldneedtobeinordertogarnerthisattention.Soliddemonstrationsofkeytechnologies arelikelyrequired,aswellasaplausiblepathtowardsprofitableWBE.Theimpactoffunding onprogresswilldependonthemainremainingbottlenecksandtheirfundingelasticity.If scanningthroughputorcomputerpoweristhelimitingfactor,extrafundingcanrelatively easilyscaleupfacilities.Bycontrast,limitationsinneuroscienceunderstandingareless responsivetoinvestment.Iffundingarriveslate,whenthefundamentalproblemshave alreadybeensolved,theamplifiedresourceswouldbeusedtoscaleuppreexistingsmall scaledemonstrationprojects.

26

Intellectualpropertyconstitutesanotherimportantconsiderationforcommercialfunding: whatcouldearlydevelopersown,howsecureandlongtermwouldtheirinvestmentbe? Withoutsolidprospectsofhavingpreferentialownershipofwhatitdevelops,afirmis unlikelytopursuetheproject.

Alternative pathways
SpecialhardwareforWBE.ItispossiblethatWBEcanbeachievedmoreefficientlyusing dedicatedhardwareratherthangenerichardware.Suchperformancegainsarepossibleif,for example,thereisaclosemappingbetweenthehardwareandthebrain,orifthefunctionsof theemulationsoftwarecouldbeimplementedefficientlyphysically.Developingsuch dedicatedhardwarewouldbecostlyunlessotherapplicationsexisted(whichiswhyinterest indedicatedneuralnetworkchipspeakedintheearly1990s). Dedicatedneuralnetworkchipshavereachedupto1.7billionsynapticupdates(and337 millionsynapticadjustments)persecondforANNmodels(Kondo,Koshibaetal.,1996), whichisapproachingcurrentsupercomputingspeedsformorecomplexmodels.Recently, therehasbeensomedevelopmentofFPGAs(FieldProgrammableGateArrays)forrunning complexneuronsimulations,producinganorderofmagnitudefastersimulationfora motorneuronthanasoftwareimplementation(fourtimesrealtime,8Mcompartments/s) (WeinsteinandLee,2005).AFPGAimplementationhastheadvantageofbeing programmable,notrequiringWBEspecialpurposehardware.Anotheradvantageinclude thataslongasthereischipspace,morecomplexmodelsdonotrequiremoreprocessingtime andthatprecisioncanbeadjustedtosuitthemodelandreducespacerequirements. However,scalinguptolargeanddenselyinterconnectednetworkswillrequiredeveloping newtechniques(WeinsteinandLee,2006).Abetterunderstandingoftheneocortical architecturemayservetoproducehardwarearchitecturesthatfititwell(Daisyproject,2008). IthasbeensuggestedthatusingFPGAscouldincreasecomputationalspeedsinnetwork simulationsbyuptotwoordersofmagnitude,andinturnenabletestinggroundsfor developingspecialpurposeWBEchips(Markram,2006). Itmayalsobepossibletouseembeddedprocessortechnologytomanufacturelargeamounts ofdedicatedhardwarerelativelycheaply.Astudyofhighresolutionclimatemodellinginthe petafloprangefounda24to34foldreductionofcostandabouttwoordersofmagnitude smallerpowerrequirementsusingacustomvariantofembeddedprocessorchips(Wehner, Olikeretal.,2008). Thisroadmapisroughlycentredontheassumptionthatscanningtechnologywillbesimilar tocurrentmicroscopy,developedforlargescaleneuroscience,automatedsectioningof fixatedtissueandlocalimagetomodelconversion.Forreasonsdiscussedinthescanning section,nondestructivescanningoflivingbrainsappearstobehardcomparedtotheslice anddiceapproachwherewehavevarioussmallscaleexistenceproofs.However,aspointed outbyRobertFreitasJr.,nanomedicaltechniquescouldpossiblyenablenondestructive scanningbyuseofinvasivemeasurementdevices.Evenifsuchdevicesproveinfeasible, molecularnanotechnologycouldlikelyprovidemanynewscanningmethodologiesaswell asradicalimprovementofneuroscientificresearchmethodsandtheefficiencyofmany roadmaptechnologies.Evenfarmoremodestdevelopmentssuchassinglemoleculeanalysis, nanosensors,artificialantibodiesandnanoparticlesforimaging(whichareexpectedtobein useby2015(NanoroadmapProject,2006))wouldhaveanimportantimpact.Henceearlyor verysuccessfulnanotechnologywouldofferfasterandalternativeroutestoachievethe

27

roadmap.Analysingthelikelihood,timeframe,andabilitiesofsuchnanomedicineisoutside thescopeofthisdocument. Onepossiblescanningalternativenotexaminedmuchhereishighresolutionscanningof frozenbrainsusingMRI.Thismightbeacomplementtodestructivescanning,butcould possiblygainenoughinformationtoenableWBE.However,wecurrentlyhavelittle informationonthelimitsandpossibilitiesofthetechnique(seediscussioninAppendixE: Nondestructiveandgradualreplacement). Asdiscussedintheoverview,WBEdoesnotassumeanyneedforhighlevelunderstanding ofthebrainormind.Infact,shouldsuchunderstandingbereacheditislikelythatitcouldbe usedtoproduceartificialintelligence(AI).HumanlevelAI(orsuperintelligentAI)would notnecessarilyprecludeWBE,butsomeofthescientificandeconomicreasonswouldvanish, possiblymakingthefieldlessrelevant.Ontheotherhand,powerfulAIcouldgreatly accelerateneuroscienceadvancesandperhapshelpdevelopWBEforotherpurposes. Conversely,successinsomepartsoftheWBEendeavourcouldhelpAI,forexampleif corticalmicrocircuitryandlearningrulescouldbesimulatedefficientlyasageneral learning/behavioursystem. TheimpactanddevelopmentofWBEwilldependonwhichofthemaincapabilities (scanning,interpretation,simulation)developlast.Iftheydeveloprelativelyindependently itwouldbeunlikelyforallthreetomatureenoughtoenablehumanlevelemulationsatthe sametime.Ifcomputingpoweristhelimitingfactor,increasinglycomplexanimalemulations arelikelytoappear.SocietyhastimetoadapttotheprospectofhumanlevelWBEinthenear future.Ifscanningresolution,imageinterpretation,orneuralsimulationisthelimitingfactor, arelativelysuddenbreakthroughispossible:thereisenoughcomputingpower,scanning technology,andsoftwaretogorapidlyfromsimpletocomplexorganisms,usingrelatively smallcomputersandprojects.Thiscouldleadtoasurprisescenariowhereinsocietyhaslittle timetoconsiderthepossibilityofhumanlevelWBE.Ifcomputingpoweristhelimiting factor,orifscanningisthebottleneckduetolackofthroughput,thenthepaceof developmentwouldlikelybeeconomicallydetermined:ifenoughinvestmentweremade, WBEcouldbeachievedrapidly.ThiswouldplaceWBEenablementunderpoliticalor economicalcontroltoagreaterdegreethaninthealternativescenarios.

Related technologies and spin-offs

ThetechnologiesneededtoachieveWBEincludetheabilitytoscanorganictissueonalow level,interpretthefindingsintofunctionalmodels,andrunextremelylargescale simulations.WBEalsorequiressufficientknowledgeoflowlevelneuralfunction. Thedesireforrunningextremelylargesimulationshasbeenastrongmotivatorfor supercomputing.Applicationsinnanotechnology,virtualreality,cryptography,signal processing,mathematics,genomics,andsimulationoftransportation,societies,business, physics,biology,andclimatesciencewillrequirepetaflopsperformanceinthenextdecade.It isunlikelythatthiswillbetheend,andexaflopperformanceisalreadybeingdiscussedinthe supercomputingcommunity.However,scalingupcurrentarchitecturestotheexascalemay beproblematic,andmayrequirenewwaysofthinkingabouthowtomanagecomplex concurrentsystems.Thesewilltoalargedegreebeshapedbytheproblemsthecomputers areintendedtosolve(andtherelativerankingoftheseproblemsbysociety,affecting

28

funding)aswellasbytradeoffsbetweenperformancewithprice,energyrequirements3,and otherconstraints.Therelatedareaofverylargescaleinformationstorageisalsoakeyissue forWBE. Anobviousrelatedtechnologyisthecreationofvirtualbodymodelsforuseinmedicine. Theycanbeusedastrainingandstudyobjects,or,atamoreadvancedstage,aspersonal medicalmodels.Suchmodelsmightenabledoctorstoinvestigatethecurrentstateofthe patient,compareittopreviousdata,testoroptimisevariousformsofsimulatedtreatment, trainparticularsurgicalapproaches,etc.Thisisatechnologyofsignificantpractical importancethatisdrivenbycurrentadvancesinmedicalimaging,thepersonalisationof healthcare,andimprovingphysiologicalmodels.Whilemostcurrentmodelsareeitherstatic datasetsorhighlevelphysiologicalmodels,personalisationrequiresdevelopingmethodsof physiologicalparameterestimationfromthedata.Simulationwillpromotethedevelopment ofmorerealisticbodymodelsusableforWBE.Conversely,theWBEfocusondatadriven bottomupsimulationappearstofitinwithdevelopingpersonalisedbiochemicalandtissue models. Virtualtestanimals,iftheycouldbedevelopedtoasufficientdegreeofrealism,wouldbe highindemandasfasterandlessexpensivetestinggroundsforbiomedicalhypotheses (MichelsonandCole,2007;Zheng,Kreuweletal.,2007).Theymayperhapsalsobeawayof avoidingtheethicalcontroversiessurroundinganimaltesting(althoughitisnot inconceivablethatconcernsaboutanimalwelfarewouldintimebeextendedtoemulated animals).Thiscouldprovideanimpetusforthedevelopmentoforganismemulation techniquesandespeciallyvalidationmethodsthathelpguaranteethatthevirtualanimals havethesamepropertiesasrealanimalswould. Overall,thereisincreasinginterestandcapabilityinquantitativemodellingofbiology(Di Ventura,Lemerleetal.,2006).Whilemuchefforthasgoneintometabolicorcontrolnetworks ofparticularinterest,thereisapushtowardscellmodelsencompassingmetabolism,the genomeandproteomics(Ortoleva,Berryetal.,2003;Tomita,2001;Schaff,Slepchenkoetal., 2001;Tyson,2001).Thereisalsointerestinbuildingsimulatorsformodellingselfassemblyin subcellularsystems(Ortoleva,Berryetal.,2003).Besidespredictingbiologicalresponsesand helpingtounderstandbiology,modellingwilllikelybecomeimportantforsyntheticbiology (Serrano,2007). Inordertoproducerealisticvirtualstimuliforabrainemulation,accuratesimulationsofthe sensoryinputtothebrainareneeded.Thesametechniquesusedtoinvestigateneural functioncanbeappliedtothesenses.Ithasalsobeenproposedthattechnologytorecord normalneuralactivityalongthebrainnervesandspinalcordcouldbehelpful.Suchdata,for examplerecordedusingmassivelyparallelelectrodearraysornanoimplants,wouldprovide goodtestdatatovalidateabodymodel.Recordedsensorydatawouldalsoberepeatable, enablingcomparisonsbetweendifferentvariantsofthesameemulation.Neuralinterfacingis alsousefulfordevelopingbetterroboticbodymodels.Currently,mostinterestinneural interfacesisfocusedonhelpingpeoplewithdisabilitiesusesensoryormotoricprosthetics. Whileneuralinterfacingforenhancementpurposesispossible,itisunlikelytobecomea significantdriveruntilprostheticsystemshavebecomecheap,safe,andveryeffective.
3

Anexascalecomputerusing2008technologywouldrequiretensofmegawatts(SandiaNationalLaboratories,2008).

29

Issues
Emulation systems
Afunctioningbrainemulationwillinclude,in additiontothebrainmodel(themainpart), somewayforthebrainmodeltoexperience bodilyinteractionswithanenvironment.There aretwodifferentwaysinwhichthiscouldbe accomplished:viaasimulatedvirtualbody inhabitingavirtualreality(whichcanbelinked totheoutsideworld);orviaahardwarebody connectedtothebrainmodelviaabody interfacemodule. EntrylevelWBEdoesnotrequirethecapacityto accommodatealloftheoriginalsensory modalitiesortoprovideafullynaturalisticbody experience.Simulatedbodiesandworlds,or hardwarebodyinterfaces,aswellas Figure9:Emulationsystemforcompletely communicationswiththeoutsideworld,arenot virtualemulation. necessaryperseforbrainemulationexcept insofartheyareneededtomaintainshorttermfunctionofthebrain.Forlongtermfunction, especiallyofhumanmindemulations,embodimentandcommunicationareimportant. Sensoryormotordeprivationappearstoproduceintellectualandperceptualdeficitswithina fewdaystime(ZubekandMacneill,1966). Thebrainemulatorperformstheactual emulationofthebrainandcloselylinked subsystemssuchasbrainchemistry.Theresult ofitsfunctionisaseriesofstatesofemulated brainactivity.Theemulationproducesand receivesneuralsignalscorrespondingtomotor actionsandsensoryinformation(inaddition, somebodystateinformationsuchasglucose levelsmaybeincluded). Thebodysimulatorcontainsamodelofthe bodyanditsinternalstate.Itproducessensory signalsbasedonthestateofthebodymodel andtheenvironment,sendingthemtothe brainemulation.Itconvertsmotorsignalsto musclecontractionsordirectmovementsinthe bodymodel.Thedegreetowhichdifferent Figure10:Emulationsystemforembodied partsofthebodyrequireaccuratesimulationis emulations. likelyvariable. Theenvironmentsimulatormaintainsamodelofthesurroundingenvironment,responding toactionsfromthebodymodelandsendingbacksimulatedsensoryinformation.Thisisalso

30

themostconvenientpointofinteractionwiththeoutsideworld.Externalinformationcanbe projectedintotheenvironmentmodel,virtualobjectswithrealworldaffordancescanbeused totriggersuitableinteractionetc. Theoverallemulationsoftwaresystem(theexoselftoborrowGregEgansterm)would regulatethefunctionofthesimulatorsandemulator,allocatecomputationalresources,collect diagnosticinformation,providesecurity(e.g.backups,firewalls,errordetection,encryption) andsoon.Itcouldprovidesoftwareservicestoemulatedminds(accessedthroughthevirtual environment)and/oroutsideexperimenters. Avariantoftheabovesystemwouldbeanembodiedbrainemulation,inwhichcasethe bodysimulatorwouldmerelycontainthetranslationfunctionsbetweenneuralactivityand physicalsignals,andthesewouldthenbeactuatedusingahardwarebody.Thebodymight becompletelyartificial(inwhichcasemotorsignalshavetobemappedontoappropriate bodybehaviours)orbiologicalbutequippedwithnervecomputerinterfacesenabling sensingandcontrol.Thecomputersystemrunningtheemulationdoesnothavetobe physicallypresentinthebody. Itiscertainlypossibletointroducesignalsfromtheoutsideonhigherlevelsthanina simulatedorrealbody.Itwouldberelativelytrivialtoaddvisualorauditoryinformation directlytothebodymodelandhavethemappearasvirtualoraugmentedreality. Introducingsignalsdirectlyintothebrainemulationwouldrequirethemtomakesenseas neuralsignals(e.g.brainstimulationorsimulateddrugs).Virtualbraincomputerinterfaces withperfectclarityandnoriskofsideeffectscouldbeimplementedasextensionsofthebody simulation/interface. Ifcomputingpowerturnsouttobeabottleneckresource,thenearlyemulationsarelikelyto runmoreslowlythanthebiologicalsystemtheyaimatemulating.Thiswouldlimitthe abilityoftheemulationstointeractinrealtimewiththephysicalworld.Indistributed emulationsdelaysbetweencomputingnodesputastronglimitonhowfasttheycanbecome. Theshortestdelayusing100m/saxonsacrossthebrainisabout1ms.Assuminglightspeed communication,processingnodescannotbefurtherawaythan300kmiflongerdelaysareto beavoided.

Complications and exotica

Besidestraightneuraltransmissionthroughsynapsestheremaybenumerousotherformsof informationprocessinginthebrainthatmayhavetobeemulated.Howimportanttheyare forsuccessinemulationremainsuncertain.Animportantapplicationofearlybrain emulationsandtheirprecursorswillbetoenabletestingoftheirinfluence.

Spinalcord
Whiletraditionallythevertebratespinalcordisoftenregardedaslittlemorethanabundleof motorandsensoraxonstogetherwithacentralcolumnofstereotypicalreflexcircuitsand patterngenerators,thereisevidencethattheprocessingmaybemorecomplex(Berg, Alaburdaetal.,2007)andthatlearningprocessesoccuramongspinalneurons(Crown, Fergusonetal.,2002).Thenetworksresponsibleforstandingandsteppingareextremely flexibleandunlikelytobehardwired(Cai,Courtineetal.,2006).

31

Thismeansthatemulatingjustthebrainpartofthecentralnervoussystemwilllosemuch bodycontrolthathasbeenlearnedandresidesinthenonscannedcord.Ontheotherhand,it ispossiblethatagenericspinalcordnetworkwould,whenattachedtotheemulatedbrain, adapt(requiringonlyscanningandemulatingonespinalcord,aswellasfindingawayof attachingthespinalemulationtothebrainemulation).Butevenifthisistrue,thetimetaken maycorrespondtorehabilitationtimescalesof(subjective)months,duringwhichtimethe simulatedbodywouldbeessentiallyparalysed.Thismightnotbeamajorproblemfor personalidentityinmindemulations(sincepeoplesufferingspinalinjuriesdonotlose personalidentity),butitwouldbeamajorlimitationtotheirusefulnessandmightlimit developmentofanimalmodelsforbrainemulation. Asimilarconcerncouldexistforotherperipheralsystemssuchastheretinaandautonomic nervoussystemganglia. Thehumanspinalcordweighs2.5%ofthebrainandcontainsaround104ofthenumberof neuronsinthebrain(13.5millionneurons).Henceaddingthespinalcordtoanemulation wouldaddanegligibleextrascanandsimulationload.

Synapticadaptation
Synapsesareusuallycharacterizedbytheirstrength,thesizeofthepostsynapticpotential theyproduceinresponsetoagivenmagnitudeofincomingexcitation.Many(most?) synapsesintheCNSalsoexhibitdepressionand/orfacilitation:atemporarychangeinrelease probabilitycausedbyrepeatedactivity(Thomson,2000).Thisrapiddynamicslikelyplaysa roleinavarietyofbrainfunctions,suchastemporalfiltering(FortuneandRose,2001), auditoryprocessing(Macleod,Horiuchietal.,2007)andmotorcontrol(NadimandManor, 2000).Thesechangesoccurontimescaleslongerthanneuralactivity(tensofmilliseconds)but shorterthanlongtermsynapticplasticity(minutestohours).Adaptationhasalreadybeen includedinnumerouscomputationalmodels.Thecomputationalloadisusually13extra statevariablesineachsynapse.

Unknownneurotransmittersandneuromodulators
Notallneuromodulatorsareknown.Atpresentabout10majorneurotransmittersand200+ neuromodulatorsareknown,andthenumberisincreasing.(Thomas,2006)lists272 endogenousextracellularneuroactivesignaltransducerswithknownreceptors,2gases,19 substanceswithputativeorunknownbindingsitesand48endogenoussubstancesthatmay ormaynotbeneuroactivetransducers(manyofthesemaybemoreinvolvedingeneral biochemicalsignallingthanbrainspecificsignals).Plottingtheyearofdiscoveryfordifferent substances(orfamiliesofsubstances)suggestsalinearorpossiblysigmoidalgrowthover time(Figure11).

32

Ghrelin Kisspeptin Orexin NPFF Nociceptin Anandamide Apelin PACAP NO FGF ANF MCH Galanin Bombesin GHRH NPY Dynorphin PDML VIP Endorphin CCK Histamine Neurotensin Enkephalin Somatostatin Substance P GnRH TRH Glycine Dopamine Angiotensin Calcitonin Serotonin Glutamate GABA CRF ACTH Noradrenaline Acetylcholine 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010

Figure11:Timeofdiscoveryoftheneurotransmitterorneuromodulatoractivityofa numberofsubstancesorfamiliesofsubstances.Datatakenfrom(vonBohlenundHalbach andDermietzel2006),likelyunderrepresentingthedevelopmentafter2000. Anupperboundonthenumberofneuromodulatorscanbefoundusinggenomics.About800 Gproteincoupledreceptorscanbefoundinthehumangenome,ofwhichabouthalfwere sensoryreceptors.Manyareorphansthatlackknownligands,andmethodsof deorphanizingreceptorsbyexpressingthemanddeterminingwhattheybindtohavebeen developed.Inthemiddle1990sabout150receptorshadbeenpairedto75transmitters, leavingaround150200orphansin2003(Wise,Jupeetal.,2004).Atpresent,78receptorsare deorphanizedeachyear(vonBohlenundHalbachandDermietzel,2006);atthisrateall orphansshouldbeadoptedwithin20years,leadingtothediscoveryofaround50more transmitters(Civelli,2005). Similarlyguanylylcyclasecoupledreceptors(fourorphans,(WedelandGarbers,1998)), tyrosinekinasecoupledreceptors(<<100,(MullerTidow,Schwableetal.,2004))andcytokine receptorswouldaddafewextratransmitters. However,thereisroomforsomesurprises.Recentlyitwasfoundthatprotonswereusedto signalinC.elegansrhythmicdefecation(Pfeiffer,Johnsonetal.,2008)mediatedusinga Na+/H+exchanger,anditisnotinconceivablethatsimilarmechanismscouldexistinthe brain.Hencetheupperboundonalltransmittersmaybesetbynotjustreceptorsbutalsoby membranetransporterproteins.

33

ForWBEmodellingallmodulatoryinteractionsisprobablycrucial,sinceweknowthat neuromodulationdoeshaveimportanteffectsonmood,consciousness,learningand perception.Thismeansnotjustdetectingtheirexistencebuttocreatequantitativemodelsof theseinteractions,asizeablechallengeforexperimentalandcomputationalneuroscience.

Unknownionchannels
Similartoreceptors,therearelikelyunknownionchannelsthataffectneurondynamics. TheLigandGatedIonChannelDatabasecurrentlycontains554entrieswith71designatedas channelsubunitsfromHomosapiens(EMBLEBI,2008;Donizelli,Djiteetal.,2006).Voltage gatedionchannelsformasuperfamilywithatleast143genes(Yu,YarovYarovoyetal., 2005).Thisdiversityisincreasedbymultimerization(combinationsofdifferentsubunits), modifiersubunitsthatdonotformchannelsontheirownbutaffectthefunctionofchannels theyareincorporatedinto,accessoryproteinsaswellasalternatemRNAsplicingandpost translationalmodification(Gutman,Chandyetal.,2005).Thiswouldenableatleastanorder ofmagnitudemorevariants. Ionchanneldiversityincreasesthediversityofpossibleneuronelectrophysiology,butnot necessarilyinalinearmanner.Seethediscussionofinferringelectrophysiologyfromgene transcriptsintheinterpretationchapter.

Volumetransmission
Surroundingthecellsofthebrainistheextracellularspace,onaverage200acrossand correspondingto20%ofbrainvolume(Nicholson,2001).Ittransportsnutrientsandbuffers ions,butmayalsoenablevolumetransmissionofsignallingmolecules. Volumetransmissionofsmallmoleculesappearsfairlywellestablished.Nitrousoxideis hydrophobicandhaslowmolecularweightandcanhencediffuserelativelyfreelythrough membranes:itcanreachupto0.10.2mmawayfromareleasepointunderphysiological conditions(Malinski,Tahaetal.,1993;SchumanandMadison,1994;WoodandGarthwaite, 1994).Whilemainlybelievedtobeimportantforautoregulationofbloodsupply,itmayalso havearoleinmemory(Ledo,Fradeetal.,2004).ThismightexplainhowLTP(LongTerm Potentiation)caninducecrosstalkthatreducesLTPinductionthresholdsoveraspanof10 mandtenminutes(HarveyandSvoboda,2007). Signalsubstancessuchasdopamineexhibitvolumetransmission(Rice,2000)andthismay haveeffectforpotentiationofnearbysynapsesduringlearning:simulationsshowthata singlesynapticreleasecanbedetectedupto20mawayandwitha100mshalflife(Cragg, Nicholsonetal.,2001).Largermoleculeshavetheirrelativediffusionspeedreducedbythe limitedgeometryoftheextracellularspace,bothintermsofitstortuosityanditsanisotropy (Nicholson,2001).AssuggestedbyRobertFreitas,theremayalsoexistactiveextracellular transportmodes.DiffusionratesarealsoaffectedbylocalflowoftheCSFandcandifferfrom regiontoregion(FenstermacherandKaye,1988);ifthisisrelevantthenlocaldiffusionand flowmeasurementsmaybeneededtodevelopatleastageneralbraindiffusionmodel.The geometricpartofsuchdatacouldberelativelyeasilygainedfromthehighresolution3D scansneededforotherWBEsubproblems.

34

Rapidandbroadvolumetransmissionsuchasfromnitrousoxidecanbesimulatedusinga relativelycoarsespatiotemporalgridsize,whilelocaltransmissionrequiresagridwitha spatialscaleclosetotheneuralscaleifdiffusionisseverelyhindered. Forconstrainingbrainemulationitmightbeusefultoanalysetheexpecteddiffusionand detectiondistancesofthe200knownchemicalsignallingmoleculesbasedontheirmolecular weight,diffusionconstantanduptake(fordifferentlocalneuralgeometriesandsource/sink distributions).Thiswouldprovideinformationondiffusiontimesthatconstrainthediffusion partoftheemulationandpossiblyshowwhichchemicalspeciesneedtobespatially modelled.

Bodychemicalenvironment
Thebodyactsasaninput/outputunitthatinteractswithourperceptionandmotoractivity.It alsoactsasachemicalenvironmentthataffectsthebrainthroughnutrients,hormones, salinity,dissolvedgases,andpossiblyimmunesignals.Mostofthesechemicalsignalsoccur onasubconsciouslevelandonlybecomeapparentwhentheyinfluencee.g.hypothalamusto producehungerorthirstsensations.Forbrainemulation,someaspectsofthischemical environmenthastobesimulated. Thiswouldrequiremappingthehumanmetabolome,atleastinregardstosubstancesthat crossthebloodbrainbarrier.Themetabolomeislikelyontheorderof2,0002,500 compounds(Beecher,2003;Wishart,Tzuretal.,2007)andlargelydoesnotchangemore rapidlythanonthesecondtimescale.Thissuggeststhatcomparedtothedemandsofthe WBE,thebodychemistrymodel,whileinvolved,wouldberelativelysimple. Ifaproteininteractionmodelisneededratherthanmetabolism,thencomplexityincreases. Accordingtooneestimatethehumaninteractomeisaround650,000proteinprotein interactions(Stumpf,Thorneetal.,2008).

Neurogenesisandremodelling
Recentresultsshowthatneurogenesispersistsinsomebrainregionsinadulthood,andmight havenontrivialfunctionalconsequences(Saxe,Malleretetal.,2007).Duringneurite outgrowth,andpossiblyafterwards,celladhesionproteinscanaffectgeneexpressionand possibleneuronfunctionbyaffectingsecondmessengersystemsandcalciumlevels(Crossin andKrushel,2000).However,neurogenesisismainlyconfinedtodiscreteregionsofthebrain anddoesnotoccurtoagreatextentinadultneocortex(Bhardwaj,Curtisetal.,2006). Sinceneurogenesisoccursonfairlyslowtimescales(>1week)comparedtobrainactivityand normalplasticity,itcouldprobablybeignoredinbrainemulationifthegoalisanemulation thatisintendedtofunctionfaithfullyforonlyafewdaysandnottoexhibittrulylongterm memoryconsolidationoradaptation. Arelatedissueisremodellingofdendritesandsynapses.Overthespanofmonthsdendrites cangrow,retractandaddnewbranchtipsinacelltypespecificmanner(Lee,Huangetal., 2006).Similarlysynapticspinesintheadultbraincanchangewithinhourstodays,although themajorityremainstableovermultimonthtimespans(Grutzendler,Kasthurietal.,2002; Holtmaat,Trachtenbergetal.,2005;Zuo,Linetal.,2005).Evenifneurogenesisisignoredand theemulationisofanadultbrain,itislikelythatsuchremodellingisimportanttolearning andadaptation.

35

 Simulatingstemcellproliferationwouldrequiredatastructuresrepresentingdifferentcells andtheirdifferentiationstatus,dataonwhattriggersneurogenesis,andmodelsallowingfor thegradualintegrationofthecellsintothenetwork.Suchasimulationwouldinvolve modellingthegeometryandmechanicsofcells,possiblyeventissuedifferentiation.Dendritic andsynapticremodellingwouldalsorequireageometryandmechanicsmodel.While technicallyinvolvedandrequiringatleastageometrymodelforeachdendriticcompartment thecomputationaldemandsappearsmallcomparedtoneuralactivity.

Gliacells
Gliacellshavetraditionallybeenregardedasmerelysupportingactorstotheneurons,but recentresultssuggestthattheymayplayafairlyactiveroleinneuralactivity.Besidethe importantroleofmyelinizationforincreasingneuraltransmissionspeed,attheveryleast theyhavestrongeffectsonthelocalchemicalenvironmentoftheextracellularspace surroundingneuronsandsynapses. Glialcellsexhibitcalciumwavesthatspreadalongglialnetworksandaffectnearbyneurons (NewmanandZahs,1998).Theycanbothexciteandinhibitnearbyneuronsthrough neurotransmitters(Kozlov,Anguloetal.,2006).Conversely,thecalciumconcentrationofglial cellsisaffectedbythepresenceofspecificneuromodulators(PereaandAraque,2005).This suggeststhattheglialcellsactsasaninformationprocessingnetworkintegratedwiththe neurons(FellinandCarmignoto,2004).Onerolecouldbeinregulatinglocalenergyand oxygensupply. Ifglialprocessingturnsouttobesignificantandfinegrained,brainemulationwouldhaveto emulatethegliacellsinthesamewayasneurons,increasingthestoragedemandsbyatleast oneorderofmagnitude.However,thetimeconstantsforglialcalciumdynamicsisgenerally farslowerthanthedynamicsofactionpotentials(ontheorderofsecondsormore), suggestingthatthetimeresolutionwouldnothavetobeasfine,makingthecomputational demandsincreasefarlesssteeply.

Ephapticeffects
Electricaleffectsmayalsoplayaroleviasocalledephaptictransmission.Inahigh resistanceenvironment,currentsfromactionpotentialsareforcedtoflowthrough neighbouringneurons,changingtheirexcitability.Ithasbeenclaimedthatthisprocess constitutesaformofcommunicationinthebrain,inparticularthehippocampus(Krnjevic, 1986).However,inmostpartsofthebrainthereisalargeextracellularspaceandblocking myelin,soevenifephapticinteractionsplayarole,theydosoonlylocally,e.g.inthe olfactorysystem(Bokil,Laarisetal.,2001),densedemyelinatednervebundles(Reutskiy, Rossonietal.,2003),ortrigeminalpainsyndromes(LoveandCoakham,2001).Itshouldbe notedthatthenervoussystemappearsrelativelyinsensitivetoeverydayexternalelectric fields(Valberg,Kavetetal.,1997;SwansonandKheifets,2006). Ifephapticeffectswereimportant,theemulationwouldneedtotakethelocallyinduced electromagneticfieldsintoaccount.Thiswouldplausiblyinvolvedividingtheextracellular space(possiblyalsotheintracellularspace)intofiniteelementswherethefieldcanbe assumedtobeconstant,linearorotherwiseeasilyapproximable.Thecorticalextracellular lengthconstantisonorderof100m(GardnerMedwin,1983),whichwouldnecessitateon theorderof1.41012suchcompartmentsifeachcompartmentis1/10ofthelengthconstant.

36

Eachcompartmentwouldneedatleasttwovectorstatevariablesand6componentsofa conductivitytensor;assumingonebyteforeach,thetotalmemoryrequirementswouldbeon theorderof10terabytes.Comparedtoestimatesofneuralsimulationcomplexity,thisis relativelymanageable.Theprocessingneededtoupdatethesecompartmentswouldbeonthe sameorderasadetailedcompartmentmodelofeveryneuronandgliacell.

Dynamicalstate
Themethodsforcreatingthenecessarydataforbrainemulationdiscussedinthispaperdeal withjustthephysicalstructureofthebraintissue,notitsstateofactivity.Someinformation suchasworkingmemorymaybestoredjustasongoingpatternsofneuralexcitationand wouldbelost.Similarly,informationincalciumconcentrations,synapticvesicledepletion, anddiffusingneuromodulatorsmaybelostduringscanning.Alikelyconsequencewouldbe amnesiaofthetimeclosesttothescanning. However,lossofbrainactivitydoesnotseemtopreventthereturnoffunctionandpersonal identity.Thisisdemonstratedbythereawakeningofcomapatients,andbycoldwaternear drowningcasesinwhichbrainactivitytemporarilyceasedduetohypothermia(Elixson, 1991).

Quantumcomputation
Whilepracticallyallneuroscientistssubscribetothedogmathatneuralactivityisa phenomenonthatoccursonaclassicalscale,therehavebeenproposals(mainlyfrom physicists)thatquantumeffectsplayanimportantroleinthefunctionofthebrain(Penrose, 1989;Hameroff,1987).Sofarthereisnoevidenceforquantumeffectsinthebrainbeyond quantumchemistry,andnoevidencethatsucheffectsplayanimportantroleforintelligence orconsciousness(Litt,Eliasmithetal.,2006).Thereisnolackofpossiblecomputational primitivesinneurobiologynoranyphenomenathatappearunexplainableintermsof classicalcomputations(KochandHepp,2006).Quantitativeestimatesfordecoherencetimes forionsduringactionpotentialsandmicrotubulessuggestthattheydecohereonatimescale of10201013s,abouttenordersofmagnitudefasterthanthenormalneuralactivity timescales.Hencequantumeffectsareunlikelytopersistlongenoughtoaffectprocessing (Tegmark,2000).This,however,hasnotdeterredsupportersofquantumconsciousness,who arguethattheremaybemechanismsprotectingquantumsuperpositionsoversignificant periods(RosaandFaber,2004;Hagan,Hameroffetal.,2002). Ifthesequantummindhypothesesweretrue,brainemulationwouldbesignificantlymore complex,butnotimpossiblegiventheright(quantum)computer.In(Hameroff,1987)mind emulationisconsideredbasedonquantumcellularautomata,whichinturnarebasedonthe microtubulenetworkthattheauthorsuggestsunderliesconsciousness. Assuming7.1microtubulespersquaremand768.9minaveragelength(Cash,Alievetal., 2003)andthat1/30ofbrainvolumeisneurons(althoughgiventhatmicotubulinetworks occursinallcells,gliaandanyothercelltype!maycounttoo)gives1016microtubules.If eachstoresjustasinglequantumbitthiswouldcorrespondtoa1016qubitsystem,requiringa physicallyintractable210^16bitclassicalcomputertoemulate.Ifonlythemicrotubulesinsidea cellactasaquantumcomputingnetwork,theemulationwouldhavetoinclude1011 connected130,000qubitquantumcomputers.Anothercalculation,assumingmerelyclassical computationinmicrotubules,suggests1019bytesperbrainoperatingat1028FLOPS (Tuszynski,2006).Oneproblemwiththesecalculationsisthattheyimputesuchaprofoundly

37

largecomputationalcapacityatasubneurallevelthatamacroscopicbrainseemsunnecessary (especiallysinceneuronsaremetabolicallycostly).

Analogcomputation
Asurprisinglycommondoubtexpressedaboutthepossibilityofsimulatingevensimple neuralsystemsisthattheyareanalogratherthandigital.Thedoubtisbasedonthe assumptionthatthereisanimportantqualitativedifferencebetweencontinuousanddiscrete variables. Ifcomputationsinthebrainmakeuseofthefullpowerofcontinuousvariablesthebrainmay essentiallybeabletoachievehypercomputation,enablingittocalculatethingsanordinary Turingmachinecannot(Ord,2006;SiegelmannandSontag,1995).See(ZenilandHernandez Quiroz,2007)forareviewofhowdifferentbraincomputationalarchitectureswouldenable differentlevelsofcomputationalpower,andtherequirementsforneuralnetworksto simulatethese. However,brainsaremadeofimperfectstructureswhichare,inturn,madeofdiscreteatoms obeyingquantummechanicalrulesforcingthemintodiscreteenergystates,possiblyalso limitedbyaspacetimethatisdiscreteonthePlanckscale(aswellasnoise,seebelow)andso itisunlikelythatthehighprecisionrequiredofhypercomputationcanbephysicallyrealized (Eliasmith,2001).Evenifhypercomputationwerephysicallypossible,itwouldbynomeans becertainthatitisusedinthebrain,anditmightevenbedifficulttodetectifitwere(the continualandotherwisehardtoexplainfailureofWBEwouldbesomeevidenceinthis direction).However,findingclearexamplesofnonTuringcomputableabilitiesofthemind wouldbeawayofrulingoutTuringemulation. Adiscreteapproximationofananalogsystemcanbemadearbitrarilyexactbyrefiningthe resolution.IfanMbitvalueisusedtorepresentacontinuoussignal,thesignaltonoiseratio isapproximately20log10(2M)dB(assuminguniformdistributionofdiscretizationerrors, whichislikelyforlargeM).Thiscanrelativelyeasilybemadesmallerthanthenaturalnoise sourcessuchasunreliablesynapses,thermal,orelectricalnoise.Thethermalnoiseisonthe orderof4.21021J,whichsuggeststhatenergydifferencessmallerthanthiscanbeignored unlesstheyoccurinisolatedsubsystemsorontimescalesfastenoughtonotthermalize.Field potentialrecordingscommonlyhavefluctuationsontheorderofmillivoltsduetoneuron firingandabackgroundnoiseontheorderoftensofmicrovolts.Againthissuggestsalimit tothenecessaryprecisionofsimulationvariables4.

Determinism
Asomewhatrelatedcriticismistheassumeddeterminismofcomputers,whilethebrainis assumedeithertocontaintruerandomnessoraphysicallyindeterministicelement(often declaredtobefreewill). Therandomnessversionofthedeterminismcriticismcanbemetbyincludingsufficientnoise inthesimulation.Randomeventsmayplayaroleinthefunctionofthenervoussystem.In particular,thenumberofindividualmoleculesinvolvedintranscriptionregulationofsome proteinsandinthefunctionofsynapticspinesislowenoughthatindividualrandom interactionscanaffectwhetherageneisswitchedonorwhetheraphosphorylationcascade
4

Analogcomputationmaystillbeausefulhardwareparadigmundersomeconditions.

38

happens.Thisrandomnessmayhavebiologicalimportance,andissometimesincludedin biophysicalmodels.Anotherpossibleroleofnoisemightbestochasticresonance,wherenoise actstoincreasethesignaltonoiseratioinnonlinear(andinformationtheoretically suboptimal)systemsattemptingtodetectaweakperiodicinput(Gammaitoni,Hanggietal., 1998).Ithasbeendemonstratedinmechanoreceptors(DouglassandWilkens,1998)and variousotherformsofsensorysystems(DouglassandWilkens,1998)asawayofdetecting faintsignals.However,whiletheexactspectraldistributionandpowermaymatterforan emulation,itdoesnotseemthattheresonanceeffectwouldbedisruptedbyusing pseudorandomnoiseiftherecurrencetimeislongenough.Unlesstherearesomeimportant hiddenvariablesinthenoiseofthebrain,noisecanbeeasilyapproximatedusingasuitably longperiodicrandomnumbergenerator(Tegmark,2000)orevenbymeansofanattached physicalrandomnumbergeneratorusingquantummechanics(Stefanov,Gisinetal.,2000). RandomnessisthereforehighlyunlikelytoposeamajorobstacletoWBE. Hiddenvariablesorindeterministicfreewillappeartohavethesamestatusasquantum consciousness:whilenotinanyobviouswaydirectlyruledoutbycurrentobservations,there isnoevidencethattheyoccurorarenecessarytoexplainobservedphenomena.

Summary
Table4showsanoverviewofinformalestimatesofthelikeliehoodthatcertainfeaturesare neededforWBEandwhethertheywouldposeseriousimplementationproblemsifneeded. Table4:Likeliehoodestimatesofmodellingcomplications.
Spinalcord Synapticadaptation LikeliehoodneededforWBE Likely Verylikely Implementationproblems Minor.Wouldrequirescanning someextraneuraltissue. Minor.Introducesextrastate variablesandparametersthatneed tobeset. Minor.Similartoknown transmittersandmodulators. Minor.Similartoknownion channels. Medium.Requiresdiffusionmodels andmicroscalegeometry. Medium.Requiresmetabolomic modelsanddata. Medium.Requirescellmechanics andgrowthmodels. Minor.Wouldrequiremore simulationcompartments,butlikely runningonaslowertimescale. Medium.Wouldrequirerelatively finegrainedEMsimulation. Profound.Wouldprecludemost proposedscanningmethods. Profound.Wouldprecludecurrently conceivedscanningmethodsand wouldrequirequantumcomputing. Profound.Wouldrequireanalog computerhardware. Mediumtoprofound,dependingon whethermerelytruerandomnoise orhiddenvariablesareneeded.

Currentlyunknown neurotransmittersand neuromodulators Currentlyunknownionchannels Volumetransmission Bodychemicalenvironment Neurogenesisandremodelling Gliacells

Verylikely

Verylikely Somewhatlikely Somewhatlikely Somewhatlikely Possible

Ephapticeffects Dynamicalstate Quantumcomputation

Possible Veryunlikely Veryunlikely

Analogcomputation Truerandomness

Veryunlikely Veryunlikely

39

Scanning
Thefirststepinbrainemulationistoacquirethenecessaryinformationfromaphysicalbrain. Wewillcallthisstepscanning. Brainemulationforcompartmentmodelsofneuronactivityneedstoacquirebothgeometric dataaboutthelocalizationandmorphologyofthenervousconnectionsand functional/chemicaldataabouttheirnaturesuchaswhationchannels,receptors,and neurotransmittersarepresent,thepresenceofelectricalsynapses,electricalmembrane properties,phosphorylationstatesofsynapses,andgeneticexpressionstates.Thismustbe doneatasufficientresolution.Itmaybepossibletoinfersomefunctionalpropertiessuchas whetherasynapseisexcitatoryorinhibitorypurelyfromgeometry(e.g.,asynapsefroma smoothneuronwithaparticularmorphologyislikelyinhibitory).Yetitremainsunclearhow muchinformationaboutsynapticstrengthandneuromodulatortypecanbeinferredfrom puregeometryatagivenlevelofresolution. Thereareseveralpotentialapproachestoscanning.Herewefocusondestructivescanning,in whichthebrainisdestructivelydisassembledduringtheemulationprocess.Theprocess couldbeappliedimmediatelyafterdeathoroncryogenicallypreservedbraintissue.Thisis thetechnologicallysimplestapproach.Destructivescanninghasgreaterfreedombothin physicaleffectsused,energylevels,andfixatingthebrainthroughfreezingand/orchemical fixation.Possiblealternativesincludenondestructivescanningandgradualreplacement; thesearediscussedinAppendixE. Mostmethodsdiscussedinthissectionrequirethesectioningofthebrainintosmallerpieces thatcanbeaccuratelyscanned.Thesectioningmethodsandthehandlingofthetissuemay posesignificantproblemsandrequirethedevelopmentofspecialhandlingtechnology.In particular,sampledistortionanddriftarekeyproblems;thefirstrequiringverycareful sectioningmethods,thesecondeitherhighprecisionequipmentorembeddingoffiducial markers.

MRImicroscopy
AlthoughMRIimagingmaynotbesuitableforscanningentirebrainsatsufficientresolution, MRImicroscopymightbesuitableforscanningpartsofthemifwaterdiffusionisstoppedby freezingorfixation. 3DMRmicroscopyhasachievedresolutionsontheorderof3.73.33.3m(Ciobanu,Seeber etal.,2002)butislimitedbythesmallfieldofviewandlongexposuretimes.Onewayof improvingthefieldofviewmaybeparallelism,whereanarrayofMRIcoilsismovedover thesurface(McDougallandWright,2007). AcombinationofMRIandAFMismagneticresonanceforcemicroscopy(MRFM)wherea magneticbeadisplacedonanultrathincantileverandmovedoverthesample(orthe reverse).BygeneratingRFmagneticfieldsthespinsofnucleiwithintheresonantslice beneaththebeadcanbeflipped,producingforcesonthecantileverthatcanbedetected.This wouldenableidentificationofthemoleculespresentnearthesurface.Currentresolutions achievedare80nmvoxelsinascanvolumeof0.5m3(Chao,Doughertyetal.,2004)and singlespindetectionwith25nmresolutioninonedimension(Rugar,Budakianetal.,2004).

40

Whetherthiscanbescaleduptodetectinge.g.thepresenceofcellmembranesorparticular neurotransmittersremainstobeseen.

Opticalmethods

Figure12:Confocalmicroscopypictureofvisualcortexneurons.StainedwithGABA channelantibody(red)andcontainingneuronsexpressinggreenfluorescentprotein.The whitescalebaris100m.(Lee,Huangetal.,2006). Opticalmicroscopymethodsarelimitedbytheneedforstainingtissuestomakerelevant detailsstandoutandthediffractionlimitssetbythewavelengthoflight(0.2m).Themain benefitisthattheygowelltogetherwithvariousspectrometricmethods(seebelow)for determiningthecompositionoftissues. Subdiffractionopticalmicroscopyispossible,iflimited.Variousfluorescencebasedmethods havebeendevelopedthatcouldbeapplicableiffluorophorescouldbeattachedtothebrain tissueinawaythatprovidedtherelevantinformation.Structuredilluminationtechniques usepatternedilluminationandpostcollectionanalysisoftheinterferencefringesbetweenthe illuminationandsampleimagetogetherwithopticalnonlinearitiestobreakthediffraction limit.Thisway,50nmresolvingpowercanbeachievedinawidefield,atthepriceof photodamageduetothehighpowerlevels(Gustafsson,2005).Nearfieldscanningoptical microscopy(NSOM)usesamultinanometreopticfibertoscanthesubstrateusingnearfield optics,gainingresolution(downtothemultinanometerscale)andfreedomfromusing

41

fluorescentmarkersattheexpenseofspeedanddepthoffield.Itcanalsobeextendedinto nearfieldspectroscopy. Confocalmicroscopysuffersfromhavingtoscanthroughtheentireregionofinterestand qualitydegradesawayfromthefocalplane.Usinginversescatteringmethodsdepth independentfocuscanbeachieved(Ralston,Marksetal.,2007). Opticalhistologyanddissection Allopticalhistologyusesfemtosecondlaserpulsestoablatetissuesamples,avoidingthe needformechanicalremovalofthesurfacelayer(Tsai,Friedmanetal.,2003).Thistreatment appearstochangethetissue210mfromthesurface.However,Tsaietal.wereoptimistic aboutbeingabletoscanafluorescencelabelledentiremousebraininto2terapixelsatthe diffractionlimitofspatialresolution. Anotherinterestingapplicationoffemtosecondlaserpulsesismicrodissection(Sakakura, Kajiyamaetal.,2007;Colombelli,Grilletal.,2004).Thelaserwasabletoremove100m samplesfromplantandanimalmaterial,modifyinga~10mborder.Thisformofoptical dissectionmightbeanimportantcomplementforEMmethods,inthat,afterscanningthe geometryofthetissueatahighresolution,relevantpiecescanberemovedandanalyzed microchemically.ThiscouldenablegainingboththeEMconnectivitydataanddetailed biochemistryinformation.Platformsalreadyexistthatcanbothinjectbiomoleculesinto individualcells,performmicrodissection,isolateandcollectindividualcellsusinglaser catapulting,andsetupcomplexopticalforcepatterns(Stuhrmann,Jahnkeetal.,2006). KnifeEdgeScanningMicroscopy(KESM)

Figure13:KnifeEdgeScanningMicroscope.Microscopeobjective(left),diamondknife (right)cuttingthespecimeninthecenter.Notetheilluminationrefractedthroughthe knife(CopyrightBrainNetworksLaboratory,TexasA&MUniversity).

42

 KESMisamethodforstainingandimaginglargevolumesathighresolutionsbyintegrating thesectioningandimagingstep.Itwasdevelopedforreconstructionandmodellingofthe threedimensionalanatomicalstructureofindividualcellsinsitu.Whilehavinglower resolutionthanSBFSEM(seebelow)itenablestheimagingofanentiremacroscopictissue volumesuchasamousebraininreasonabletime(McCormick,2002a;McCormick,2002b). TheKESMactssimultaneouslyasamicrotomeandamicroscope.Adiamondknifeisused forthecuttingandillumination,movingtogetherwiththemicroscopeacrossastationary tissuespecimenembeddedinplastic.Imagingoccursofthejustseparatedsectionalonga1D strip.Imagingat0.3mresolutionof0.5mthicksectionshasbeendemonstrated, producingdatasetswheremicrovasculatureandindividualcellscanbetracedusing contouringalgorithms.Theinstrumentscansatratesupto200Mpixels/s.Thiswouldenable scanningamousebrain(1cm3)inahundredhours,givinga15terabytedatasetof uncompresseddata(Mayerich,Abbottetal.,2008).Thewidthoffieldis2.5mmfor64m resolutionand0.625mmfor32mresolution(McCormick,Kohetal.,2004).Inorderto reducejitteringandusingkniveslargerthanthemicroscopefieldofview,thesampleiscutin astairstepmanner(KohandMcCormick,2003). Themainlimitationistheneedforstaininginsideavolume.Atpresentthenumberofsuch stainsislimited.Usingtransgenicanimalsexpressingstainsorfluorescencecouldenable mappingofdifferentneuralsystems.

XRaymicroscopy

Figure14:Xraymicrotomographystereoimageofthesupraesophagealganglionin Drosophilia,showingindividualneurons.Scale50m.(Mizutani,Takeuchietal.,2007) Xraymicroscopyisintermediatebetweenopticalmicroscopyandelectronmicroscopyin termsofresolution. HardXraymicrotomographyhasbeenusedtoimagethelarvalDrosophilabrain(Mizutani, Takeuchietal.,2007).Thiswasachievedbystainingneuronswithmetalandthenacquiring imagesrotatedby0.12with300msexposure,withatotalacquisitiontimeof1800seconds. Thefieldofviewwasontheorderof0.5mm2withapixelresolutionof0.47x0.47m;the resultingtomographicalresolutionwasabout1mineachdirection.Individualneurons

43

couldbeidentified,andtheauthorsestimatethatthesetupcouldinprinciplescana1mm3 blockofmammaliannervetissuewith103104neurons.However,theresolutionislikelytoo crudetoidentifysynapsesandconnectivity. ByusingsoftXrayswithwavelengthsinthewaterwindow(betweentheabsorptionedges ofoxygenandcarbonat2.34.4nm)wherecarbonabsorbstentimesasstronglyaswater, organicstructurescanbeimagedwithgoodcontrastdowntoatleast30nmresolution. Advantagesincludethatitcanbedonewithoutspecialsamplepreparationproceduresand canbeappliedtohydratedcells,itpermitsthickerspecimensthanEM(upto10m),Xray absorptionspectraoflocalizedregionscanberecordedbyusingdifferentXrayenergies,and potentiallythreedimensionalimagingcouldbepossibleusingmultiplebeams(Yamamoto andShinohara,2002). AmajordisadvantageisthattheXrayscausetissuedamage.WhenthetotalXrayfluxgoes above4105photonsperm2,myofibrilslosetheircontractility;andabove106photonsper m2,yeastcellslosetheirdyeexclusionability(asignthattheyaredeadorpunctured).This islowerthanthefluxesofXraymicroscopeswith50nmresolution,makingscanningof tissuewhileretainingitsfunctionimpossible(Fujisaki,Takahashietal.,1996).Onewayof avoidingdegradationoftheimageistousearapidpulseofrays.Thisalsoavoidsmotion blurringduetodiffusioninthecaseofhydratedtissue:toimagefreelydiffusingobjectsof size10nmexposureshorterthan0.1msisneeded,whileacellularorganellewouldrequire between14ms1.4sdependingonsize(ItoandShinohara,1992).Usingvitrifiedtissuethe radiationdosecanbeincreasedthousandfoldwithoutstructuralchanges,andthediffusion issuedisappears(Methe,Springetal.,1997).TheseconcernsmakeXrayimagingunlikelyto workfornondestructivescanning. OfparticularinterestforWBEisthepossibilityofdoingspectromicrosopytodeterminethe localchemicalenvironment.Xrayabsorptionedgesareaffectedbythechemicalbindingstate oftheatomprobed.Differentaminoacidshavedistinguishablefingerprintsthatare relativelyunaffectedbypeptidebonding.Inprinciple,thespectraofproteinscouldbe predictedbasedontheirsequences.ScanningXraymicroscopeshavelongexposuretimes (minutes),althoughtheydeposit510timeslessradiationinthesample(Jacobsen,1999). WhilecurrentlyfartooslowforWBEpurposes,findingwaysofspeedingupthisprocess maymakeitrelevantforWBE.

Atomicbeammicroscopy
Insteadofphotonsorelectrons,neutralatomscouldbeusedtoimageasample.Sincethede Brogliewavelengthofthermalatomsissubnanometer,theresolutioncouldbeveryhigh.By usingunchargedandinertatomslikehelium,thebeamwouldalsobenondestructive(Holst andAllison,1997).Heliumatomscatteringhasalargecrosssectionwithhydrogen,which mightmakeitpossibletodetectmembranesinunstainedtissue. Atpresent,imagingathighresolutionusingatomicbeammicroscopyhasnotbeenachieved (lowerresolutionhasbeenachieved(Doak,Grisentietal.,1999)).Developmentofridged atomicmirrorshasenabledfocusingneutralatombeams(Oberst,Kouznetsovetal.,2005; ShimizuandFujita,2002)andwouldinprincipleallowfocusingthebeamtoaspotsizeof tensofnanometers(Kouznetsov,Oberstetal.,2006);byscanningthespotacrossthesample animagecouldbebuiltuplikeinotherformsofscanningmicroscopy.

44

Electronmicroscopy

Figure15:EMpictureofrathippocampusneuropil.Dmarksadendriteofapyramidalcell. Severalsynapsescanbeseentotheleft,noticeablebythepresenceofsmallspherical vesiclesonthepresynapticsideandadarkpostsynapticdensityonthereceivingside.The scalebaris1m.(CopyrightJSpacek,SynapseWeb) Electronmicroscopycanresolvethefinedetailsofaxonsanddendritesindenseneuraltissue. Imagescanbecreatedthroughtransmissionelectronmicroscopy(TEM),whereelectronsare sentthroughtissue,orscanningelectronmicroscopy(SEM)whereelectronsarescattered fromthesurface:bothmethodsrequirefixingthesamplebyfreezingand/orembeddingitin polymer.TEMhasachieved0.1nmimaging(Nellist,Chisholmetal.,2004).However,the mainchallengeistoautomatesectioningandacquisitionofdata.Thethreecurrentmain methodsareserialsectionelectrontomography(SSET),serialsectiontransmissionelectron microscopy(SSTEM)andserialblockfacescanningelectronmicroscopy(SBFSEM) (BriggmanandDenk,2006).TwonewmethodsthatmaybeusefulareFEIsDualBeam FocusedIonBeamSEM(FIBSEM)andautomaticultrathinsectioningandSEMimaging (ATLUM). SSET:HighresolutionTEM3Dimagescanbecreatedusingtiltseriesbasedtomography wherethepreparationistiltedrelativetotheelectronbeam,enablingtherecordingofdepth information(Frank,1992;Penczek,Markoetal.,1995).Thismethodappearssuitedmainlyfor localscanning(suchasimagingcellularorganelles)andcannotpenetrateverydeepintothe surface(around1m)(Lui,Frsteretal.,2005). SSTEM:CreatingultrathinslicesforTEMisanotherpossibility.(Tsang,2005)createdathree dimensionalmodeloftheneuromuscularjunctionthroughserialTEMof50nmsections createdusinganultramicrotome.(White,Southgateetal.,1986)usedserialsectionsto

45

reconstructtheC.elegansnervoussystem.However,sectioningisphysicallytrickyandlabor intensive. SBFSEM:Onewayofreducingtheproblemsofsectioningistoplacethemicrotomeinside themicroscopechamber(Leighton,1981);forfurthercontrast,plasmaetchingwasused (KuzirianandLeighton,1983).(DenkandHorstmann,2004)demonstratedthat backscatteringcontrastcouldbeusedinsteadinaSEM,simplifyingthetechnique.They producedstacksof5070nmthicksectionsusinganautomatedmicrotomeinthemicroscope chamber,withlateraljitterlessthan10nm.Theresolutionandfieldsizewaslimitedbythe commerciallyavailablesystem.Theyestimatedthattracingofaxonswith20nmresolution andS/Nratioofabout10withina200mcubecouldtakeaboutaday(while10nmx10nm x50nmvoxelsatS/N100wouldrequireascantimeontheorderofayear). Reconstructingvolumesfromultrathinsectionsfacesmanypracticalchallenges.Current electronmicroscopescannothandlesectionswiderthan12mm.Longseriesofsectionsare neededbuttheriskoferrorsordamageincreasewiththelength,andthenumberofspecimen holdinggridsbecomesexcessive(unlesssectioningoccursinsidethemicroscope(Kuzirian andLeighton,1983)).Currentstateoftheartforpracticalreconstructionfromtissueblocksis about0.1mm3,containingabout107108synapses(Fiala,2002). FIBSEM5:Thesemiconductorindustryhaslongusedfocusedionbeams(FIB)(usually acceleratedbeamsofgalliumions)toverypreciselycutawaypartsofintegratedcircuitchips forfailureanalysis.ResearchersatFEIhaverecentlydemonstratedthatthistechniqueand instrumentcanalsobeappliedtoplasticembeddedneuraltissueinamannersimilartothe SBFSEMabove(Mulders,Knottetal.,2006).IntheFIBSEM,thetop3050nmlayerofablock oftissue(havingblockfacedimensionsofapproximately100x100m)isablatedawayusing thefocusedionbeam.TheresultingblockfaceisthenimagedwiththeSEMusingthe backscattersignaljustasisdoneintheSBFSEM,andthisprocessisrepeatedtoimage hundredsofsuccessivelayers. BecausetheFIBSEMablatesawaytheblockstoplayerusinganionbeam(insteadofa diamondknife),thecookingoftheblocksurfacethatoccursinSBFSEMduetothehigh beamcurrentisnotaproblem.Usinglongerintegrationtimes,FIBSEMimageshave demonstratedlateralresolutionsof5nmorbetterandmuchimprovedsignaltonoiseratios. ATLUM:TheaboveSSETandSSTEMapproachesusesectionsobtainedbydiamondknife sectioningonatraditionalsemiautomatedultramicrotome.Thisprocessisonlysemi automatedbecausethecollectionofthesesectionsrequiresmanuallyscoopingfreefloating sectionsfromtheknifeswaterboatontoTEMslotgridsusinganeyelash.Thisisan inherentlyunreliableprocessforallbutthesmallestvolumes. TheAutomaticTapeCollectingLatheUltramicrotome(ATLUM)isaprototypemachine recentlydevelopedatHarvardUniversitythatfullyautomatestheprocessofultrathin sectioningandcollection(Hayworth,Kashturietal.,2006;Hayworth,2007).IntheATLUMa tissuesample(typically12mminwidth,10mmlong,and0.5mmdeep)ismountedonasteel axleandisrotatedcontinuouslyviaanultrapreciseairbearingwhileapiezodrivendiamond knifeslicesoffoneultrathinsectionperrevolution.Afeedbackloopemployingcapacitive sensorsmaintainsthepositionofthekniferelativetotheaxletowithinapproximately10nm allowingevenlargeareatissuesections(manysquaremillimeters)tobecuttothicknesses
5

TextderivedfromKennethHayworth.

46

below40nm.Eachsectioncomesstreamingofftheknifesedgeontothesurfaceofawater boatattachedbehindtheknife.Intraditionalultramicrotomythisfragilefloatingsection wouldhavetobecollectedmanually(viadexteroususeofaneyelash)ontoaTEMslotgrid,a notoriouslyunreliableprocess.Incontrast,intheATLUMeachsectionisimmediately collectedbythemechanismontoasubmergedconveyorbeltofcarboncoatedMylartape(100 mthickand8mmwide).Inthisway,hundredsoflargeareaultrathinsectionsare automaticallysecuredontoameterslongMylartape.

Figure16:OverviewofATLUMoperation.(CopyrightLichtmanLab,HarvardUniversity) ThistapeissubsequentlystainedwithheavymetalsandimagedinaSEM.Imagingviathe SEMbackscattersignal(liketheSBFSEMandFIBSEMapproachesabove)allowstheATLUM todispensewithfragileTEMslotgrids(whichalsoremovestheirwidthofsection limitation).TheMylartapeprovidesasturdysubstrateforhandlingandstoragewhileits carboncoatingpreventschargingandbeamdamageduringSEMimaging. ImagesobtainedfromATLUMtissuetapesareofequivalentqualitytotraditionalTEM imagesshowinglateralresolutionbetterthan5nanometers.Recenttestshavealsoshownthat atomographictiltseries(asintheSSETtechnique)maybeperformedonATLUMcollected sectionstoobtainadditionaldepthinformation.Thisisperformedbytiltingthesectionat variousangleswithrespecttotheSEMselectronbeamthusprovidingZresolutionseven finerthanthesectionthicknessitself.Becausethesectionsarestoredforlaterimagingthey canbepoststainedwithheavymetals(producingimageswithsuperiorsignaltonoiseratios andmuchfasterimagingtimesthantheSBFSEMandFIBSEMblockfaceimagers)and,if needed,theycanbepoststainedwithaseriesofotherstainsforoverlayingchemicalanalysis mapsonthehighresolutionSEMstructuralimages.

47

Comparisonoftechniques
Resolution WiththeexceptionoftheSBFSEM,allofthesetechniqueshavealreadydemonstrated sufficientresolutiontomaptheexactpointtopointconnectivityofbraintissuedowntothe levelofcountingsynapticvesiclesinindividualsynapses.Withadditionalmodificationsit maybepossiblefortheSBFSEMtoalsoreachtheseresolutions. Reliabilityandrobustness TransmissionEMtechniquesliketheSSTEMandSSETrequiresectionstobemountedonslot grids.Theprocessofcollectingsectionsonslotgridshasnotyetbeensuccessfullyautomated, andsectionsmountedonslotgridsarethemselvesextremelyfragile(becausetheyare essentiallyfreelysupportedstructureslessthanamicronthick).Thisraisesseriousdoubtsas tothefeasibilityofscalingthesetechniquesuptoverylargevolumesofneuraltissue (althoughmachinesliketheATLUMmaybemodifiabletoallowautomatedcollectionof sectionsontapeswithprefabricatedslots,oralternativelyslotscanbecreatedinthetapeafter collectioniscompleted).SEMbackscatterimagingseemstoofferequivalentresolutionand imagequalitywhileavoidingthesepitfallsintrinsictoTEMapproaches. OneremainingpossibleadvantageofTEMapproachesisimagingspeed.Imagesincurrent SEMsarebuiltuponepixelatatimeastheelectronbeamscansacrossthesamplessurface. Incontrast,thepixelsinTEMimagesarecapturedinparallelbyaCCD+scintillatormounted belowthesection.Thisfactcould,inprinciple,allowimagingtimesordersofmagnitude fasterthanSEMimaging.However,manytechnicalissuescomplicatethiscomparison.In practice,currentTEMsimagesectionsonlyslightlyfasterthanSEMsdo.Inaddition, multibeamSEMs(describedbelow)arecurrentlybeingdevelopedthatwillmassively parallelize,andthusspeedup,SEMimageacquisition Blockfaceapproaches(SBFSEMandFIBSEM)haveinherentreliabilitysincetheyavoidthe perilousstepofcollectingultrathinsectionsbysimplydestroyingtheminsituafterhaving alreadyimagedthem.TheFIBSEMtechniqueisadditionally,atleastinprinciple,robustto smalldifferencesinembeddingquality.Alloftheothertechniquesusediamondknife sectioningwhichrequiresgooduniformityofresininfiltrationandresinhardnessthroughout thetissueblocktosectionsmoothly.Ofcoursetissuesamplesforallthetechniquesmustbe infiltratedcorrectlyforproperultrastructuralpreservation WhiletheATLUMisnecessarilylessreliablethanblockfaceapproaches(sinceitcollects ultrathinsectionsforlaterimaging)itstillhasthepotentialforextremereliabilityoverlarge volumes.Thisisbecausetheultrathinsections,withintheATLUMmechanism,aresecuredto thesturdyMylartapealmostimmediatelyaftertheyaresectionedbythediamondknife.In fact,theleadingedgeofeachsectionissecuredtothecollectiontapewhilethetrailedgeis stillbeingsectionedbytheknife.Inthiswayeachsectionisalwaysundercompletecontrolof themechanism. Imagingtime Forconcreteness,letsoutlineareasonableneartermneuronalcircuitmappinggoal necessarytosupportasetofpartialbraincircuitemulationexperiments.Wecanthen comparetheimagingtimes,andthusthefeasibility,ofthedifferentmethods.Oneofthemost interestingandbeststudiedpiecesofbraincircuitryisthatunderlyingorientationtuningin theprimaryvisualcortex(V1)inthecatandintheprimate.Thiscircuitscrudefunctional properties(orientationtunedsimpleandcomplexcells)havebeenknownfordecadesyet thecircuitryunderlyingthesefunctionalresponsesremainsatopicofheateddebate

48

generatingdozensofjournalarticleseveryyear.Moreimportantly,ourlackofadetailed understandingoftheneuronalcircuitryhasblockedprogressintoV1smoresubtle computationalpropertiessuchasitsroleinperceptualgrouping. Atrialbrainemulationexperimentmightsetasagoalthemodellingoforientationtuning responsesofafewcellsinV1(possiblycorrelatedwithprevioustwophotonrecordingof activitywithinthesametissue).Thistaskwouldnotrequireentirevolumesofcomplex neuropilbetracedintotal.Itwouldrequire,however,imagingtheconnectivityofatleast severaldozenneuronsinthevariouslayersofV1,andtheirconnectivitybothwitheachother andwithmanydozenaxonalprojectionstoV1fromthelateralgeniculatenucleus(LGN).In addition,theseaxonalprojectionsfromtheLGNwouldneedtobetracedbacktotheirorigins todeterminetheirrelativepositionsandthustoinfertheirownreceptivefieldproperties.In whole,suchastudywouldrequirenanoresolutiontracingofpartsofoveronehundred neuronsandthousandsofsynapticconnectionsspanningavolumeontheorderof10mm3 (ThisisassumingthatthemyelinatedprojectionsbetweenLGNandV1canbetracedatmuch lowerresolution,excludingitfromthevolumeestimate).Becauseofthenatureofthestudy, suchneuronscouldnotbeselectivelylabelledbeforehandandsotheelectronmicroscopic methodsdescribedinthissection(whichcanimageanyarbitraryneuronalcellorprocess)are theonlycurrenttechniqueswhichmaybecapableofperformingthetask. SSETandSSTEM:Asdiscussedabove,themanualcollectionofTEMreadysections(for techniquesliketheSSETandSSTEM)isoutofthequestionforvolumesaslargeas10mm3. SBFSEMandFIBSEM:Sofarthelargestvolumessectionedinthesedeviceshavebeenless than0.01mm3,withblockfacedimensionstypically200x200morless.Imagingrateon thesedevicesisaround100kHz(10microsecondsperpixel).Ingeneral,blockfaceimaging approachesnecessitatesuchslowimagingrates(toobtainadequatesignaltonoiseratios) sincethematerialcanonlybelightlystainedwhileinblockform. OurV1circuittracingscenariodoesnotstrictlyrequiretheentire10mm3volumebeimaged athighestresolution;however,astheneuronalcircuitstobetracedwanderrandomly throughouttheentirevolume,onedoesnotknowaprioriwhichpartstoimageathigh resolution.Because,intheseblockfaceimagingdevices,eachsectionisdestroyedinsitu, thereisonlyonechancetoimageitandthusineffectallpartsofthevolumemustbeimaged atsufficientresolutiontotracethefinestneuronalprocesses.Assuming5nmlateral resolutionand50nmsections(typicalvaluesforneuropiltracingstudies)the10mm3block consistsof81015voxelsandwouldrequireontheorderof2,000yearstoimage!Suchlong imagingtimemakesthistracingstudyinfeasibleonthesemachines. ATLUM:ThelargestvolumesectionedontheATLUMsofaris0.1mm3withblockface dimensionsof1.4mmx4.5mmandsectionthicknessof45nm(totalof400sections).ATLUM cuttingspeedisaround0.03mm/secandeachoftheseultrathinsections(6.3mm2inarea)was producedatarateofapproximately4minutespersection.Ifsucharatecouldbereliably sustaineditwouldtakeabout3monthsforanATLUMlikedevicetoreducea10mm3tissue blocktoaseriesof50nmsectionseachsecurelymountedonalongMylartapereadyfor imaging. BecausetheATLUMcollectssectionsforlaterimaging,itssectionscanbepoststainedwith heavymetals.Thisallowsadequatesignaltonoiseratioswithshorterimageacquisition times.Imagingrateisaround1MHz(1.0microsecondperpixel)sobulkimagingtheentire 10mm3wouldrequireontheorderof200years.Thisstillwouldrendertheprojectinfeasible;

49

however,unliketheblockfacetechniques,theATLUMtechniquecantakeadvantageof directedhighresolutionimagingtodramaticallyshortenthisimagingtime. Asstatedabove,atcurrentsectioningspeeds,theATLUMcouldreducea10mm3blockof braintissuetoatapeofultrathinsectionsinjustafewmonths.Suchatapewouldconstitutea permanentultrathinsectionlibraryoftheentirevolume,andanypartofitcouldbe randomlyimaged(andreimaged)atwhateverresolutiondesired.Aresearchercould coarselyimagetheentirevolumeinafewdays,findatargetneuronwithinthevolume,and thensubsequentlydirecthighresolutionimagingexactlywhereitisneededtotracethat neuronsprocessesanditsconnectionswithadditionalneurons.Inthisway,amultineuron circuit,stretchingthroughouttheentire10mm3volume,couldbemappedwithsynaptic precisionwhileonlyimagingperhapsoneonethousandthofthewholevolume.Inthis fashion,theATLUMcanpotentiallyreducethetimeneededtotraceacompleteneuralcircuit fromhundredsofyearstojustafewmonths,makingtheV1trialemulationexperiment potentiallyfeasibleinthenearterm. Parenthetically,wecanalsonotethattheneartermgoaloftracingandemulatingcircuitsofa fewhundredneuronsdrasticallyreducestheimageprocessingandcomputersimulation requirements.Insteadofrequiringmillionsofneuronswithbillionsofsynapsestobetraced andidentified,onlythelimitedsubsetunderstudyneedbetracedintheimagedata.The successfulcompletionofpartialbrainemulationstudies(suchasthehypotheticalV1study above)isprobablyanecessaryprecursortospurthetypeofresearchandinvestmentin sectioning,imaging,tracing,andemulationstechnologiesneededforeventuallyscalingupto wholebrainemulationlevels. PossibilitiesforincreasingSEMimagingspeed Fromtheabovediscussionitisclearthatlongimagingtimesconstituteamajorbarrierto wholebrainemulationusingSEMtechniques.However,thereiscurrentlyamajorresearch pushtowardmassivelyparallelmultibeamSEMswhichhasthepotentialtospeedupSEM imagingbymanyordersofmagnitude.Thisresearchpushisbeingdrivenbythe semiconductorindustryaspartofitsefforttoreducefeaturesizesoncomputerchipsbelow thelevelthattraditionalphotolithographycanproduce. Thecircuitrypatternswithincomputerchipsareproducedthroughaseriesofetchingand dopingsteps.Eachofthesestepsmustaffectonlyselectedpartsofthechip,soareastobeleft unaffectedaretemporallycoveredbyathinlayerofpolymerwhichispatternedinexquisite detailtomatchthesubmicronfeaturesofthedesiredcircuitry.Forcurrentmassproduction ofchipsthispolymerlayerispatternedbyshiningultravioletlightthroughamaskontothe surfaceofthesiliconwaferwhichhasbeencoveredwiththephotopolymerinliquidform. Thisselectivelycuresonlythedesiredpartsofthephotopolymer.Toobtainsmallerfeatures thanUVlightcanallow,electronbeams(justasinaSEM)mustinsteadbeusedtoselectively curethephotopolymer.Thisprocessiscalledebeamlithography.Becausetheelectronbeam mustberasteredacrossthewafersurface(insteadoffloodilluminatingitasinlight lithography)theprocessiscurrentlymuchtooslowforproductionlevelruns. Severalresearchgroupsandcompaniesarecurrentlyaddressingthisspeedproblemby developingmultibeamebeamlithographysystems(Kruit,1998;vanBruggen,vanSomeren etal.,2005;vanSomeren,vanBruggenetal.,2006;ArradianceInc).Inthesesystems, hundredstothousandsofelectronbeamsrasteracrossawaferssurfacesimultaneously writingthecircuitrypatterns.ThesemultibeamsystemsareessentiallySEMs,anditshould beastraightforwardtasktomodifythemtoallowmassivelyparallelscanningaswell

50

(Pickard,Grovesetal.,2003).Forbackscatterimaging(asintheSBFSEM,FIBSEM,and ATLUMtechnologies)thismightinvolvemountingascintillatorwithagridofholes(onefor eachebeam)veryclosetothesurfaceofthetissuebeingimaged.Inthiswaytheinteractions ofeachebeamwiththetissuecanbereadoffindependentlyandsimultaneously. ItisdifficulttopredicthowfasttheseSEMsmayeventuallyget.A1,000beamSEMwhere eachindividualbeammaintainsthecurrent1MHzacquisitionrateforstainedsections appearsreachablewithinthenexttenyears.WecanverytentativelyapplythisprojectedSEM speeduptoaskhowlongimagingahumanbrainwouldtake.First,assumeabrainwere slicedinto50nmsectionsonATLUMlikedevices(anenormousfeatwhichwoulditselftake approximately1,000machineseachoperatingat10xthecurrentsectioningrateatotalof 3.5yearstoaccomplish).Thismassiveultrathinsectionlibrarywouldcontaintheequivalent of1.11021voxels(at5550nmpervoxel).Assumingjudicioususeofdirectedimaging withinthisultrathinsectionlibraryonly1/10mayhavetobeimagedatthisextremelyhigh resolution(usingmuchlower,andthusfaster,imagingonwhitematertracts,cellbody interiorsetc.).Thisleavesroughly1.11020voxelstobeimagedathighresolution.If1,000 SEMseachutilizing1,000beamletsweretotacklethisimagingjobinparalleltheircombined dataacquisitionratewouldbe11012voxelspersecond.Atthisratetheentireimagingtask couldbecompletedinlessthan4years.

Nanodisassembly
Themostcompleteapproachwouldbetopickthebrainapartatombyatomormoleculeby molecule,recordingtheirpositionandtypeforfurtheranalysis.Thescenarioin(Morevec, 1988)canalsobedescribedasnanodisassembly(inanunfixatedbrain,withonthefly emulation)workingonaslightlylargersizescale.(Merkle,1994)describesarelatively detailedproposalwherethebrainisdividedinto3.210150.4mcubeswhereeachcube wouldbedisassembledatomically(andatom/moleculepositionsrecorded)byadisassembler nanodevice(Drexler,1986)overathreeyearperiod. Ithasbeenpointedoutthatmedicalnanoroboticsistheformofnanotechnologybestusedfor nondestructivescanning6.Giventhatnodetailedproposalforananodisassemblerhasbeen madeitishardtoevaluatethechancesofnanodisassembly.Itwouldhavetoactatalow temperaturetopreventmoleculesinthesamplefrommovingaround,removingsurface moleculesonebyone,identifyingthemandtransmittingtheposition,orientationandtypeto secondlinedataaggregators.Clearchallengesaretheconstructionoftooltipsthatcanextract arbitrarymoleculesordetectmoleculartypeforfurtherhandlingwithspecializedtooltips,as wellashandlingmacromoleculesandfragilemolecularstructures.Macromoleculescanlikely notbepulledoutinonepiece.Atomicdisassemblywouldavoidthecomplicationsof moleculesforthegreatersimplicityofahandfulofatomtypes,atthepriceofneedingto breakmolecularbonds,theriskofensuingrearrangementsandthepossibilityofcreating reactivefreeradicals.Matureproductivenanosystemswouldappeartobeanecessary precursortechnologyforthisapproach(probablyfortheproductionofthemassivelyparallel disassemblysystemrequiredforcontrolledatomicdisassemblyofamacroscopicobject) (ForesightNanotechInstituteandBatelleMemorialInstitute,2007).Itappearslikelythatthe kindoftechnologyneededfordisassemblywouldbesignificantlymorecomplexthantheone neededforatomicallypreciseassembly.

RobertFreitasJr.,personalcommunication.

51

Chemicalanalysis
Akeychallengeistodetectthechemicalstateandtypeofcellularcomponents.Normallythis isdonebystainingwithdyesorquantumdotsthatbindtotherighttarget,followedby readoutusingopticalmethods.Besidetheneedfordiffusingdyesthroughthesamples,each dyeisonlyselectiveforacertaintargetorgroupoftargets,necessitatingmultipledyesfor identifyingallrelevantcomponents.Ifthenumberofchemicalsthathavetobeidentifiedis large,thiswouldmakedyeingineffective. OnepossibleapproachisRamanmicrospectroscopy(Krafft,2004;Krafft,Knetschkeetal., 2003),wherenearinfraredscatteringisusedtoimagethevibrationspectrumofthechemical components(mainlymacromolecules)oftissue.Theresolutionfornearinfraredspectroscopy isabout1m(limitedbydiffraction)andconfocalmethodscanbeusedfor3Dimaging. Recordingtimesareverylong,ontheorderofminutesforindividualpixels;inordertobe usefulforWBEthishastobespeededupsignificantlyorparallelized.Usingshorter wavelengthsappearstoinducetissuedamage(Puppels,Olminkhofetal.,1991),whichmay beoflittleconcernfordestructivescanning.Ultravioletresonancemicrospectroscopyhasalso beenused,enablingselectiveprobingofcertainmacromolecules(Pajcini,Munroetal.,1997; Hanlon,Manoharanetal.,2000).Insomecasesnativefluorescencecanenableimagingby triggeringitwithUVlight,laserinducednativefluorescence,LINF,suchasinthecaseof serotonin(Tan,Parpuraetal.,1995;Parpura,Tongetal.,1998)andpossiblydopamine (Mabuchi,Shimadaetal.,2001). Anewmethodwithgreatpromiseisarraytomography,ahybridbetweenoptical fluorescenceimagingandelectronmicroscopy(MichevaandSmith,2007).Samplesarecut intoultrathin(50200nmthick)sectionsforminganorderedarrayonaglassslide.Thearray isthenlabelledwithfluorescentantibodiesorotherstainsandimaged,generatinga3D reconstructionwithadepthresolutionsetbythesectioningthickness.Afterthisimagingthe arraycanbeelutedtoremovethestain,restainedwithanewstain,imagedandsoon.Finally itcanbestainedwithmetalandimagedinascanningelectronmicroscope.Thedifferent imagestackscanthenbecombined,givingahighresolution3Dreconstructionwithchemical information. Atpresentitlooksuncertainhowmuchfunctionallyrelevantinformationcanbedetermined fromspectra.Ifthenumberofneurontypesisrelativelylowandchemicallydistinct,itmight beenoughtorecognizetheirindividualprofiles.Addingdyestailoredtodisambiguate otherwiseindistinguishablecasesmayalsohelp.

Embedding, fixation and staining techniques

Mostformsofdestructivescanningrequirefixationandembeddingthebraintobescanned sothatitcanbehandled,sectioned,andimagedwell.Thisprocessmustpreserve ultrastructureandnecessarychemicalinformation. Currentelectronmicroscopyoftenreliesonosmiumtetroxideandotherheavymetalstainsto staincellmembranesforimprovedcontrast(Palay,McGeeRusselletal.,1962).Itappears likelythatunlesspuremorphologycanbeusedtodeducefunctionalproperties,improved stainingmethodsareneededtocreatealinkbetweenwhatcanbeimagedandwhatis functionallyrelevant.Thismightbeanareawherenanoparticles(andlater,moreadvanced nanodevices)willbeimportant.

52

Conclusion
ItislikelythatthescanningmethodsusedtoproviderawdataforWBEwillbe,atleastinthe earlydays,destructivemethodsmakinguseofsectioningandscanning. Assumingthatimagingthefinestaxonalprocessesandsynapticspinesarerequired(50nm), thissetsaresolutionrequirementontheorderof5nmatleastintwodirections.Comparing themethodsmentionedandtheirresolutionwegetthefollowingcategorization: Table5:Scanningmethods
Method MRI Resolution >5.5m(nonfrozen) Doesnotrequiresectioning,may achievebetterresolutiononvitrified brains. Spectromicroscopypossible? Notimplementedyet. Requiresfluorescentmarkers, spectroscopypossible. Enablesmultiplestaining Basic2Dmethod,mustbecombined withsectioningortomographyfor 3Dimaging.Damagefromhigh energyelectronsathighresolutions.

MRImicroscopy NIRmicrospectroscopy Allopticalhistology KESM Xraymicrotomography MRFM SI Xraymicroscopy SBFSEM FIBSEM ATLUM SSET Atomicbeammicroscopy NSOM SEM Arraytomography TEM

3m 1m 0.7m 0.3mx0.5m 0.47m 80nm 50nm 30nm Requiredresolution? 5070nmx120nm 3050nmx120nm 40nmx5nm 50nmx1nm 10nm 5nm? 120nm 120nmSEM,50x200x200nm fluorescencestains <1nm

A5550nmresolutionbrainscanrequires1.41021voxels,alargeamountofrawdata(see nextsection).Destructivescanningworkingonfixatedbrainsmayavoidhavingtostorethe entiredatasetbyonlystoringthemostrecentslicesofbrainandperformimageprocessingon these.Insteadoffirstscanningtheentirebrain,andthenprocessingtheimagedatatoextract therelevantfeaturesandneuronalnetwork,theextractioncanproceedpiecemealand concurrentlyasnewimagingdataiscollected.Oncerelevantinformationhasbeenextracted fromadatabatch,therawvisualdataforthatbatchcanbediscarded. However,ideassuchasthesinglebrainphysicallibrary(Hayworth,2002)demonstratethat thescanningdoesnothavetobesingleshot:abrainissectionedandfixedinasuitable mannerforinteractivescanning(possiblyusingseveralmodalities)andretrieval.Thisalso getsaroundthedatastorageproblembyonlyneedingtoscanregionsofinterestandstore theirdata(possiblytemporarily,sincetheycanberescannedifneededlater).Inearly neuroinformaticsapplications,thismightincludemappingoutthelongrangeconnectivityof asubsetofrepresentativeneuronsinordertofindtheirmorphologyandconnectivity,only requiringimagingaverysmallsubsetofthebrainsvolume(0.01%formappingout100,000 neuronsinahumanbrain).Itmayalsobepossibletocombinethisapproachwithmethods likearraytomography(MichevaandSmith,2007)forcombinedchemicalandstructural maps.

53

 Lowresolutionmethodssuchasopticalmicroscopyareanimportanttestofmanyaspectsof theWBEresearchendeavour,suchaslargescaledatamanagement,inferringneuralfunction frommorphology(possiblyusingspectroscopyorothermethodsofestimatingchemical states),andprovidingearlytestdatafortracingandconnectivityreconstructionalgorithms. Iftherequiredlevelofsimulationisatlevel5orabove(detailedcellularelectrophysiology andneuronconnectivity:seeTable2)weappeartohavealreadyachievedtheresolution requirementsandtheremainingproblemisindata/tissuemanagementandscalingup methodstohandlelargebrains.UsingKESMorATLUMitshouldbepossibleinthevery nearfuturetoconstructadetailedconnectomeofthebrain,inparticulartheexact interconnectionsofcorticalminicolumns. IftherequiredlevelforWBEisbelowlevel5,increasesofimagingresolutionareofrelatively limiteduse.Rather,weneedmodalitiesthatenablemappingtheproteins,possiblytheir statesandthepresenceofmetabolitesorRNAtranscripts.Thiswouldposeamajorresearch challenge,althoughitisinlinewithmuchresearchinteresttodayexaminingthebiophysics ofcells.

54

Imageprocessingandscaninterpretation
Thedatafromthescanningmustbepostprocessedandinterpretedinordertobecomeuseful forbrainemulation(orotherresearch).Cellmembranesmustbetraced,synapsesidentified, neuronvolumessegmented,distributionofsynapses,organelles,celltypesandother anatomicaldetails(bloodvessels,glia)identified.Currentlythisislargelydonemanually: cellularmembranescanbeidentifiedandhandtracedatarateof12hours/m3(Fialaand Harris,2001),fartooslowforevensmallcorticalvolumes. Softwareneededincludes(after(Fiala,2002)): Geometricadjustment(aligningsections,handlingshrinkage,distortions) Noiseremoval Datainterpolation(replacinglostorcorruptedscandata) Cellmembranetracing(segmentation,tracingin2Dand3D) Synapseidentification Identificationofcelltypes Estimationofparametersforemulation Connectivityidentification Databasing Datahandlingisatpresentabottleneck.0.1mm3at400pixels/mresolutionand50nm sectionthicknesswould(compressed)contain73terabytesofrawdata.Afullbrainatthis resolutionwouldrequire109terabytes(Fiala,2002).Whileextremelylarge,eventhismight onedayberegardedasfeasible(seeAppendixB).However,asmentionedintheprevious chapter,foremulationpurposesitmaybefarmorepracticaltoperformasequenceof scanningandinterpretationstepssothatonlyasmallerbufferofhighresolutiondatais necessary. Ifthebrainisdividedintosmallblocks,eachblockcouldbeanalyzedindependently,atleast intermsoflowlevelimageprocessingandpreliminarytracing.Datafromneighbouring blockswouldneedtobecomparedbutthereisnoneedfortheanalysisofmoreremote blocks(withthepossibleexceptionofinterpolatinglostdata).Thissuggeststhatitcanbe parallelizedtoagreatdegree.Asthescanprogresses,lowleveldatacanbediscardedto makeroomforthecompressedhighlevelrepresentationneededtobuildtheemulation7. ThissectiondealswiththeassumptionthatWBEisachievedusingimagebasedmethods ratherthancorrelationanalysismethodswheree.g.nanomachinesrecordlocalneuralactivity andestimateconnectivityfromthecorrelationsintheactivitypattern.Correlationmethods wouldhavecorrespondingdemandsforsignalprocessingandinference,butinthetime domainratherthanthespacedomain.

Geometric adjustment
Variousmethodsforachievingautomaticregistration(correctingdifferencesinalignment)of imagestacksarebeingdeveloped.Atitssimplest,registrationinvolvesfindingacombination oftranslation,scaling,androtationthatmakessubsequentimagesmatchbest.However, skewingandnonlineardistortionscanoccur,requiringmorecomplexmethods.Combining thiswithoptimizationmethodsandanelasticmodeltocorrectforshapedistortionproduced
7Thislossofdatamaybeofconcernsinceinitialscansarelikelytobeexpensive(andhencehardtorepeat),andfor humanbrains(whichareindividuallyvaluable).SomeformsofscanningsuchasATLUMmayproducestorable librariesofslicesthatcanberetainedforrescanningorfurtheranalysis(Hayworth,2002).

55

goodresultswithmacroscopicstacksofratandhumanbrains(Schmitt,Modersitzkietal., 2007).

Noise removal
Removingnoisefromimagesispartofstandardimageprocessing,withanextensive literatureandstrongresearchinterest.Ingeneral,noiseremovalissimplifiedwhenthekinds ofnoiseintroducedbyscanningartefactsandthenatureofthesystemareknown.Asan example,see(Mayerich,McCormicketal.,2007)wherelightvariationsandknifechatter noisewereremovedfromKESMdata.

Data interpolation
Lost/corrupteddatamustbereplacedwithprobabilisticinterpolations.Thismightrequire feedbackfromlaterstagestofindthemostlikelyinterpretationorguess,constrainedbywhat makessensegivenknowndata. Forlargelostvolumes,genericneuronsandconnectivitymighthavetobegeneratedbased onmodelsofmorphologyandconnectivity.Thegoalwouldbetoavoidchangingthe functionalityofthetotalnetwork.EarlyresultsinWBEandstatisticsfromsuccessfulimaging volumesshouldprovidemuchusefulinputindevelopingthis. Sectioningthebrainintoindividuallyscannablechunkswillintroducedatalossandpossible misalignmentalongtheedges(forexample,theKESMsuffersdamageupto5minwidth betweendifferentcolumns(Kwon,Mayerichetal.,2008)).Thismayprovetobethemajor sourceoflostdatainWBE.Sincethiscouldcausemistracingoflongaxonscrossing numerouschunkboundaries,solvingthisissueisahighpriority.Alignmentcanprobablybe achievedreliablyifthelostzoneissufficientlysmallerthanthecorrelationlengthinthe surroundingimages.

56

Cell tracing

Figure17:BloodvesselreconstructionfromKESMNissldata.(CopyrightbrainNetworks Laboratory,TexasA&MUniversity) Automatedtracingofneuronsimagedusingconfocalmicroscopyhasbeenattemptedusinga varietyofmethods.Evenifthescanningmethodusedwillbeadifferentapproachitseems likelythatknowledgegainedfromthesereconstructionmethodswillbeuseful. Oneapproachistoenhanceedgesandfindtheoptimaljoiningofedgepixels/voxelstodetect contoursofobjects.Anotherisskeletonization.Forexample,(Urban,OMalleyetal.,2006) thresholdedneuronimages(afterimageprocessingtoremovenoiseandartefacts),extracting themedialaxistree.(Dima,Scholzetal.,2002)employeda3Dwavelettransformtoperform amultiscalevalidationofdendriteboundaries,inturnproducinganestimateofaskeleton. Athirdapproachisexploratoryalgorithms,wherethealgorithmstartsatapointanduses imagecoherencytotracethecellfromthere.Thisavoidshavingtoprocessallvoxels,but riskslosingpartsoftheneuroniftheimagesaredegradedorunclear.(AlKofahi,Laseketal., 2002)usedirectionalkernelsactingontheintensitydatatofollowcylindricalobjects. (MayerichandKeyser,2008)useasimilarmethodforKESMdata,acceleratingthekernel calculationbyusinggraphicshardware.(Uehara,Colbertetal.,2004)calculatesthe probabilityofeachvoxelbelongingtoacylindricalstructure,andthenpropagatesdendrite pathsthroughit. Oneweaknessofthesemethodsisthattheyassumecylindricalshapesofdendritesandthe lackofadjoiningstructures(suchasdendriticspines).Byusingsupportvectormachinesthat aretrainedonrealdataamorerobustreconstructioncanbeachieved(SantamaraPang, Bildeaetal.,2006). Overall,tracingofbranchingtubularstructuresisamajorinterestinmedicalcomputing.A

57

surveyofvesselextractiontechniqueslisted14majorapproaches,withseveralexamplesof each(KirbasandQuek,2004).Thesuccessofdifferentmethodsismodalitydependent.

Figure18:3DvisualizationofGolgistainedcellreconstructedfromKESMdata. (CopyrightBrainNetworksLaboratory,TexasA&MUniversity

Figure19:3Dreconstructionofacube(2mside)ofneuropilfromrathippocampus. Axonsaregreen,dendritesochre,astrocytespaleblue,myelindarkblue.(CopyrightJ Spacek,SynapseWeb)

58

Synapse identification

Figure20:ReconstructedspinydendriteofCA1pyramidalcell,renderedfromdatain (HarrisandStevens,1989).(CopyrightSynapseWeb) Inelectronmicrographs,synapsesarecurrentlyrecognizedusingthecriteriathatwithina structuretherearesynapticvesiclesadjacenttoapresynapticdensity,asynapticdensitywith electrondensematerialinthecleftanddensitiesonthecytoplasmicfacesinthepreand postsynapticmembranes(Colonnier,1981;PetersandPalay,1996). OneofthemajorunresolvedissuesforWBEiswhetheritispossibletoidentifythefunctional characteristicsofsynapses,inparticularsynapticstrengthandneurotransmittercontent,from theirmorphology. Ingeneral,corticalsynapsestendtobeeitherasymmetricaltypeIsynapses(7595%)or symmetricaltypeIIsynapses(525%),basedonhavingaprominentorthinpostsynaptic density.TypeIIsynapsesappeartobeinhibitory,whiletypeIsynapsesaremainlyexcitatory (butthereareexceptions)(PetersandPalay,1996).Thisallowsatleastsomeinferenceof functionfrommorphology. Theshapeandtypeofvesiclesmayalsoprovidecluesaboutfunction.Small,clearvesicles appeartomainlycontainsmallmoleculeneurotransmitters;largevesicles(60nmdiameter) withdensecoresappeartocontainnoradrenaline,dopamineor5HT;andlargevesicles(up to100nm)with5070nmdensecorescontainneuropeptides(Hokfelt,Brobergeretal.,2000; Salio,Lossietal.,2006).Unfortunatelytheredoesnotappeartobeanyfurtherdistinctiveness ofvesiclemorphologytosignalneurotransmittertype.

59

Identification of cell types

Distinguishingneuronsfromgliaandidentifyingtheirfunctionaltyperequiresother advancesinimagerecognition. Thedefinitionofneurontypesisdebated,aswellasthenumberoftypes.Theremightbeas manyas10,000types,generatedthroughaninterplayofgenetic,posttranscriptional, epigenetic,andenvironmentalinteractions(MuotriandGage,2006).Therearesome30+ namedneurontypes,mostlycategorizedbasedonchemistryandmorphology(e.g.shape,the presenceofsynapticspines,whethertheytargetsomataordistaldendrites).Distinguishing morphologicallydifferentgroupsappearfeasibleusinggeometricalanalysis(Jelinekand Fernandez,1998). Intermsofelectrophysiology,excitatoryneuronsaretypicallyclassifiedintoregularspiking, intrinsicbursting,andchattering,whileinterneuronsareclassifiedintofastspiking,burst spiking,latespikingandregularspiking.However,alternateclassificationsexist.(Gupta, Wangetal.,2000)examinedneocorticalinhibitoryneuronsandfoundthreedifferentkindsof GABAergicsynapses,threemainelectrophysiologicalclassesdividedintoeightsubclasses, andfiveanatomicalclasses,producing15+observedcombinations.Examiningthesubgroup ofsomatostatinexpressinginhibitoryneuronsproducedthreedistinctgroupsintermsof layerlocationandelectrophysiology(Ma,Huetal.,2006)withapparentlydifferentfunctions. Inprefrontalcortexlayer2/3inhibitoryneuronsmorphologyandelectrophysiologyalso producedclustersofdistincttypes(Krimer,Zaitsevetal.,2005). Overall,itappearsthatthereexistdistinctclassesofneuronsintermsofneurotransmitter, neuropeptideexpression,proteinexpression(e.g.calciumbindingproteins),andoverall electrophysiologicalbehaviour.Morphologyoftenshowsclustering,buttheremayexist intermediateforms.Similarly,detailsofelectrophysiologymayshowoverlapbetween classes,buthavedifferentpopulationmeans. Somefunctionalneurontypesarereadilydistinguishedfrommorphology(suchasthefive typesofthecerebellarcortex).Akeyproblemisthatwhiledifferingmorphologieslikely impliesdifferingfunctionalproperties,thereversemaynotbetrue.Someclassesofneurons appeartoshowastronglinkbetweenelectrophysiologyandmorphology(Krimer,Zaitsevet al.,2005)thatwouldenableinferenceofatleastfunctionaltypejustfromgeometry.Inthe caseoflayer5pyramidalcells,somestudieshavefoundalinkbetweenmorphologyand firingpattern(Kasper,Larkmanetal.,1994;MasonandLarkman,1990),whileothershave not(ChangandLuebke,2007).Itisquitepossiblethatdifferentclassesaredifferently identifiable,andthatthemorphologyfunctionlinkcouldvarybetweenspecies. Inmanyspeciesthereexistidentifiableneurons,neuronsthatcanbedistinguishedfromother neuronsinthesameanimalandidentifiedacrossindividuals,andsetsofequivalentcellsthat aremutuallyindistinguishable(butmayhavedifferentreceptivefields)(Bullock,2000). Whilerelativelycommoninsmallandsimpleanimals,identifiableneuronsappeartobea minorityinlargerbrains.Earlyanimalbrainemulationsmaymakeuseoftheequivalenceby usingdatafromseveralindividuals,butasthebrainsbecomelargeritislikelythatall neuronshavetobetreatedasindividualandunique.

60

Estimation of parameters for emulation

(Markram,2006)liststhefollowingnecessarybuildingblocksforreconstructingneural microcircuits: Table6:Necessarydataforreconstructingneuralmicrocircuits
Network Totalnumberofneurons Geneexpressionprofiles Neurons Electrophysiological profiles Morphologicalprofiles Totalnumberofboutons Totalnumberoftargets Terminalsperconnection Presynapticinnervationpattern Totalsynapsenumbers Synapsesperconnection Postsynapticinnervationpatterns Transmissionstatistics Connectionconductances Synapseconductances Neurotransmitterreceptors Ionicpermeabilities Reversalpotentials Probabilitiesofrelease Depressiontimeconstants Facilitationtimeconstants Longtermplasticities Morphoelectrophysiological types Ionchannel compositions Ionchannel distributions Frequenciesof occurrence

Presynaptic Structural connectivity Postsynaptic

Synapticbiophysics Functional connectivity Synapticdynamics

Manyoftheseareinturnmultivariate,suchasionicpermeabilitiesorthelistof conductances. NeurondataintheNeocorticalMicrocircuitDatabase(Markram,2005)include135 electrophysiologicalparametersderivedfromresponsestostimuli,234geometricparameters basedonneuralreconstructionand51geneticpropertiesfromsinglecellRTPCRdata.For synapsesdataoninvolvedcellidentities,anatomicalproperties(15parameters), physiologicalproperties(reversalpotentialandpharmacologicalblock),synapticdynamics(5 parameters)andkinetics(14parameters).

Electrophysiology
Givenelectrophysiologicaldataofacellsresponsestosimplecurrentstimuliitispossibleto replicatetheresponseinacompartmentmodel(inparticularthedensityofionicchannels) throughautomatedmethodssuchasgeneticalgorithmsandsimulatedannealing (Druckmann,Banittetal.,2007;Keren,Peledetal.,2005;VanierandBower,1999;VanGeit, Achardetal.,2007).However,fittingexperimentshaveshownthatthesameneuralactivity canbeproducedbydifferentsetsofconductances(AchardandDeSchutter,2006).Thismay suggestlessneedtodetectallionicconductancesifdonebyothermeans,buttheresultsalso suggestthatthesetofgoodmodelsformrelativelyisolatedhyperplanes:relativelyprecise fittingofallparametersisneededtogetgoodperformance.

61

Geneexpression
Differentinhibitoryinterneuronsshowdifferentgeneexpressionfordifferentreceptors,ion channelsandgapjunctionformingproteins(Blatow,Caputietal.,2005).Strongcorrelations betweengeneexpressionofcertainionchannelsandconductanceshavebeenobserved suggestingthatfunctioncanbeinferredfromgeneticdata(ToledoRodriguez,Blumenfeldet al.,2004).Differentneuronsshowvariableexpressionlevels,yetdifferentneurontypeshave uniquepatterns(Schulz,Goaillardetal.,2006;Schulz,Goaillardetal.,2007).Differencesin alternatesplicinghavebeenfoundinindividualcellswithinaneuronpopulation,butthere arealsodifferencesinsplicingindifferentbrainregionssuggestingthereareoverall regulatorymechanisms(WangandGrabowski,1996). IfscanningcandetectrelevantmRNAlevels(andthelinkbetweentheseandfunctional propertieshavebeenfoundusinghighthroughputanalysisofallkindsofneurons)itwould likelybepossibletodeducefunctionaltype.

Detectingsynapticefficacies
Normallysynapticpropertiesaredeterminedelectrophysiologicallybytriggeringaction potentialsinthepresynapticneuronandmeasuringtheresponseinthepostsynapticneuron. Thiscandetectnotjustthegeneralstrengthofthesynapsebutalsoadaptationproperties. Therearecaseswhereelectrophysiologicalpropertiesappeartobelinkedtodetectable differencesinsynapticmorphology,inparticulartheappearanceofvesicleribbons(Fields andEllisman,1985;Hull,Studholmeetal.,2006).LTPlikelyaffectssynapticcurvature,size andperforationsofthepostsynapticdensityinatimedependentmanner(MarroneandPetit, 2002;Marrone,2007).Itiscommonlyassumedthatthis,aswellasmoreshortterm electrophysiologicalchanges,involvesremodellingofthecytoskeletoninresponseto plasticity(DillonandGoda,2005;Chen,Rexetal.,2007).Particularproteins,suchasprofilin, aregulatorofactinpolymerisation,areactivatedbyLTPinducingstimuliandmayindicate synapsesthatarepotentiating(AckermannandMatus,2003). Manysynapsesaresilent,lackingmembraneAMPAreceptors;stimulationcanproducea transitiontoanactivevocalstate(Kullmann,2003).Silentsynapsesdonotappeardifferent inmicrographs,suggestinganegativeanswertothequestionofwhetherfunctioncanbe inferredfromEMstructure(AtwoodandWojtowicz,1999).ThismeansthatforWBE detectingatleastAMPAmaybenecessary;thiscouldlikelybeachievedusingarray tomography.

Connectivity identification
Thisstepassignssynapticconnectionsbetweenneurons. Atpresent,statisticalconnectivityrulesareusedbasedonproximity,thePetersrule,in whichsynapticconnectionsareassumedwhereaxonswithboutonsoverlapwithdendrites (BraitenbergandSchuz,1998;Peters,1979).Thiscanbeusedtoestimatethestatisticsof synapticconnectivity(Binzegger,Douglasetal.,2004;Shepherd,Stepanyantsetal.,2005; Kalisman,Silberbergetal.,2003).However,neuralgeometrycannotpredictthestrengthof functionalconnectionsreliably,perhapsbecausesynapticplasticitychangesthestrengthof geometricallygivensynapses(Shepherd,Stepanyantsetal.,2005).

62

Itisnotpossibletorelyonsynapsesonlyoccurringfromaxonstodendrites;axosomatic, axoaxonic,anddendrodendritic(whichmaybeonewayorreciprocal)synapseshavebeen observed.Occasionally,severalsynapsescoincide,suchasserialaxoaxodendriticsynapses andsynapticglomeruliwhereanaxonsynapsesontotwodendrites,oneofwhichalso synapsesontheother. Onepossibilityisthatthereexistsenoughstereotypy(repeatingpatternsofstructuraland functionalfeatures)inthebraintosimplifyconnectivityidentification(andpossibly interpolation)(Silberberg,Guptaetal.,2002).Suchinformationwouldconstrainthe interpretationprocessbyrulingoutcertainpossibilities,whichwouldattheveryleastenable aspeedup.Acquiringthepotentiallyverycomplexinformationaboutstereotyped arrangementswouldbeoneoftheearliestapplicationsandbenefitsofdetailedscanningand massiveneuroinformaticsdatabases.However,deviationsfromstereotypymaybeof particularimportanceinachievingpersonWBEsincetheycanrepresentindividualvariations orcharacteristics. Gapjunctions,wherethepreandpostsynapticcellsareelectricallylinkedbyconnexon channelsthroughthecellmembrane,canbeidentifiedbymembranesremainingparallelwith just2nmseparationandagridofconnexons.Theyappeartoberelativelyrareinmammals, butoccuratleastintheretina,inferioroliveandlateralvestibularnucleus(PetersandPalay, 1996). AtleastinsimplenervoussystemssuchasC.elegansgeneexpressioncontainssignificant informationaboutitsconnectivitysignature(Kaufman,Droretal.,2006).Attheveryleast, theavailabilityofthiskindofinformationcouldbeusedtoconstrainneurontypesand connectivity.

Conclusion
Imageprocessingisamatureareawithmanycommercialandscientificapplications. Developmentofbasicprocessingforhandlingthescandatadoesnotappeartoposemany problemsbeyondtheneedforextremelyhighthroughputsignalandimageprocessing,and perhapsthedatamanagementissuesoftherawscans. Theneuroinformatics/WBErelatedfurtherprocessingstepsposearesearchchallenge. Identifyingcellularobjects,inparticularconnectivityandsynapses,isanontrivialimage interpretationproblemthatiscurrentlybeingstudied.Basiccontouringalgorithmsappearto workreasonablywell,anditislikely,givenotherresultsincurrentimagerecognition,that synapsedetectorscouldbeconstructed.Imageinterpretationistraditionallya computationallycostlyoperation,andmayconceivablybeabottleneckindevelopingearly WBEmodels(inthelongrun,assumingimprovementsincomputerhardwarenecessary anywayforlargescaleemulation,thisbottleneckisunlikelytoremain). Thehardestandcurrentlyleastunderstoodissueisestimatingemulationparametersfrom imagery(ordevelopingscanningmethodsthatcangaintheseparameters).Thisrepresents oneofthekeyresearchissuesWBEneedtoanswer(ifonlytentatively)inordertoassessits viabilityasaresearchprogram.

63

Neuralsimulation
TheareaofneuralsimulationbeganwiththeclassicHodgkinandHuxleymodeloftheaction potential(HodgkinandHuxley,1952).Atthetime,calculatingasingleactionpotentialusing amanuallycrankedcalculatortook8hoursofhardmanuallabour.Sincethentheabilityto computeneuralactivityacrosslargenetworkshasgrownenormously(MooreandHines, 1994).

How much neuron detail is needed?

Itisknownthatthemorphologyofneuronsaffectstheirspikingbehaviour(Ascoli,1999), whichsuggeststhatneuronscannotsimplybesimulatedasfeaturelesscellbodies.Insome casessimplificationsofmorphologycanbedonebasedonelectricalproperties(Rall,1962), butitisunlikelythisisgeneric. Anissuethathasbeendebatedextensivelyisthenatureofneuralcodingandespecially whetherneuronsmainlymakeuseofaratecode(wherefiringfrequencycontainsthesignal) ortheexacttimingofspikesmatter(Rieke,Warlandetal.,1996).Whileratecodes transmittinginformationhavebeenobserved,therealsoexistfastcognitiveprocesses(suchas visualrecognition)thatoccurontimescalesshorterthanthenecessarytemporalaveragingfor ratecodes.Neuralrecordingshavedemonstratedbothprecisetemporalcorrelationsbetween neurons(Lestienne,1996)andstimulusdependentsynchronization(Gray,Konigetal.,1989). Atpresent,theevidencethatspiketimingisessentialisincomplete,buttheredoesnotappear tobeanyshortageofknownneurophysiologicalphenomenathatcouldbesensitivetoit.In particular,spiketimingdependentplasticity(STDP)allowssynapticconnectionstobe strengthenedorweakeneddependingontheexactorderofspikeswithaprecision<5ms (Markram,Lubkeetal.,1997;BiandPoo,1998).Henceitisprobablyconservativetoassume thatbrainemulationneedsatimeresolutionsmallerthan0.41.4ms(Lestienne,1996)in orderfullytocapturespiketiming. Thetimeresolutionisconstrainedbytheabovespiketimingrequirementaswellasnumeric considerationsinmodellingactionpotentials(wherethedynamicsbecomeverystiff).Ion channelsopenin0.1ms,suggestingthatemulationsthatdonotdependonindividual detailedionchanneldynamicsmightneedthistimeresolution.Iftheyalsotakeindividual ionchannelsandenzymesintoaccount,theywouldhavetobe13ordersofmagnitudemore detailed. Oneofthemostimportantrealizationsofcomputationalneuroscienceinrecentyearsisthat neuronsinthemselvesholdsignificantcomputationalresources.Dendriticcomputing involvesnonlinearinteractionsinthedendritictree,allowingpartsofneuronstoactas artificialneuralnetworksontheirown(SingleandBorst,1998;LondonandHausser,2005; Sidiropoulou,Pissadakietal.,2006).Itappearspossiblethatdendriticcomputationisa significantfunctionthatcannotbereducedintoapointcellmodelbutrequirescalculationof atleastsomeneuronsubsystems.

InternalChemistry
Brainemulationneedstotakechemistrymoreintoaccountthancommonlyoccursincurrent computationalmodels(Thagard,2002).Chemicalprocessesinsideneuronshave

64

computationalpowerontheirownandoccuronavastrangeoftimescales(fromsub millisecondtoweeks).Neuromodulatorsandhormonescanchangethecausalstructureof neuralnetworks,e.g.byshiftingfiringpatternsbetweendifferentattractorstates. About200chemicalspecieshavebeenidentifiedasinvolvedinsynapticplasticity,forminga complexchemicalnetwork.However,muchofthecomplexitymayberedundantparallel implementationsofafewcorefunctionssuchasinduction,patternselectivity,expressionof change,andmaintenanceofchange(wheretheredundancyimprovesrobustnessandoffers thepossibilityoffinetuning)(AjayandBhalla,2006). Attheverylownumbersofmoleculesfoundinsynapticspines,chemicalnoisebecomesa significantfactor,makingchemicalnetworksthatarebistableatlargervolumesunstable belowthefemtoliterlevelandreducingpatternselection(Bhalla,2004b,a).Itislikelythat complexformationoractivityconstrainedbymembranesisessentialforthereliabilityof synapses.However,thismaynotnecessarilyrequiredetailedmodellingofthecomplexes themselves,justoftheirstatistics. Proteomicsmethodsarebeingappliedtosynapses,potentiallyidentifyingallpresent proteins(Li,2007).TheSynapseproteinDataBasecontainsabout3,000humansynapse relatedproteins(Zhang,Zhangetal.,2007),althoughitislikelythatmanyofthesearenot involvedintheactualsynapticprocessing. Oftheproteinscodedbythehumangenome,around3.2%(988)havebeenpredictedtobe regulatorymolecules,2.8%(868)kinases,5.0%(1543)receptors,1.2%(376)signalling molecules,and1.3%(406)ionchannels.Iftherelativeproportionsarethesameamongthe unknownproteins(41.7%,12,809),thenumbersshouldbescaledupby1.4(VenterAdamset al.,2001).Posttranslationalmodificationsproduceonaverage36differentproteinspergene (Wilkins,Sanchezetal.,1996).Thisplacesanupperlimitonthenumberofproteintypes directlyinvolvedinneuralsignallingandinternalsignaltransductionontheorderof35,000. Ifallgeneswereinvolvedweshouldexpecttheproteometobearound158,000proteins. Laterestimateshaverunupallthewayto1millionproteins(and600,000immunoglobulins varyinginepitopebinding,likelyirrelevantforWBE)(HumpherySmith,2004),although mostestimatesappeartotendtowardsthe100,000range. IftherequirementsforWBEareonthekineomelevel,thenforeachinvolvedproteina numberofspeciesmaybeneeded,dependingonthenumberofpossiblephosphorylation, dimerization,carboxylation,orotheralteredstatesthatexist.Inprinciple,thiscouldleadtoa combinatorialexplosionoftypes,requiringstoringdataforindividualproteinmoleculesif thenumberoffunctionallyrelevantkindsisverylarge. ItiscommontosimulatecellularregulatorynetworksusingmassactionlawsandMichaelis Mentenkinetics,althoughthisassumesfreediffusionandrandomcollisions.Thisassumption doesnotalwayshold,sincemolecularmobilityinsidecellsislimitedbythepropertiesofthe cytoplasm,compartmentalization,andproteinanchoringtosurfaces.Thiscanbeatleast partiallytakenintoaccountbyadjustingtheequationsforfractalkinetics.Whetherfractal kineticsormassactionisvaliddependsmainlyontheprobabilityofreactions(Grimaand Schnell,2006;SchnellandTurner,2004;XuandDing,2007). Itmightalsobenecessarytomodelgeneexpressionandotherepigeneticmechanisms.Long termplasticityinneuronsrequireschangesingeneexpression,bothasresponsetoplasticity inducinginputandinregulatingoverallplasticity.Inaddition,geneexpressionisknownto

65

drivethecircadianrhythm,whichaffectsmanyaspectsofbrainfunction(Levensonand Sweatt,2005;MillerandSweatt,2007;McClungandNestler,2008).Whetherallformsof plasticityorotherrelevantchangescanbesimulatedwithsufficientdetailwithoutsimulating thegeneexpressionnetworkiscurrentlyunknown.Ifnot,thenthebrainemulationwouldat theveryleasthavetosimulatethecontributinggeneregulationnetworkandatmosta sizeablepartofthetotalnetwork.Geneexpressiontimescalesareontheorderofminutesto hours,suggestingthatthetimestepofexpressionsimulationdoesnothavetobeasshortas forneuronsifitisnotdoneonanindividualproteinlevel.

Learningrulesandsynapticadaptation
AWBEwithoutanysynapticchangewouldlikelycorrespondtoaseverelyconfusedmind, trappedinanterogradeamnesia.Whileworkingmemorymaybebasedonattractorstatesof neuralactivityintheprefrontalcortexandsomeformsofprimingandhabituationplausibly arebasedonsynapticadaptationandcalciumbuildup,longtermmemoryformation requireschangesinthestrength(andpossiblyconnectivity)ofsynapses. Synapticlearninghasbeenextensivelystudiedincomputationalneuroscience,startingwith DonaldHebbs1949suggestionthatcooccurringneuralfiringinitiatedlongtermchange. Sincethenmodelsonalllevelsofabstractionhavebeenstudied,rangingfromthecompletely abstracttodetailedsynapticbiochemistry. Manynetworkmodelsincludevarioussynapticlearningrules.Thereisawidevarietyof rulesused,rangingfromsimpleHebbianruleswithnostatebeyondthecurrentweight,over theBCMruleaccountingforLTP/LTDeffectsinratecodingneurons(Bienenstock,Cooper etal.,1982)andSTDPthatcountstimedifferencesbetweenspikes(Senn,Markrametal., 2001),todetailedsignaltransductionmodelsthatinclude30+substances(Kikuchi,Fujimoto etal.,2003;BhallaandIyengar,1999). Synapsesalsoshowvariousformsofadaptationtorepeatedfiring,includingbothfacilitation anddepression,whichinturncanaffectthenetworkdynamicsinwaysthatappearlikelyto havebehaviourallyrelevantconsequences(Thomson,2000).Variousmodelshavebeen constructedofhowsynapticresourcesaredepletedandreplenished(Tsodyks,Pawelziket al.,1998).Suchmodelsusuallyincludeafewextrastatevariablesinthesynapsesandtime constantsforthem. ForWBEitisimportanttokeepthenumberofstatevariablesandlocalparameterslowin synapses,sincetheydominatethestoragedemandsforneural/compartmentmodelsofthe brain.Atpresent,wedonothaveafirmestimateofhowcomplexsynapsesneedtobein ordertoreproducethefullrangeofplasticityobserved.Itisverylikelythatsimpleweight basedmodelsaretoosimpleandthatmodelswithahandfulofstatevariablescovermost observedphenomena.Whetherthefulltransductioncascadeneedstobesimulatedisunclear. Ontheplusside,thisisanareathathasbeensubjecttointensemodellingandresearchon multiplescalesforseveraldecadesandveryaccessibletoexperimentaltesting.Progressin synapticmodellingmaybeagoodindicatorofneuroscienceunderstanding.

Neural models
ThefirstneuralmodelwastheMcCullochPittsneuron,essentiallybinaryunitssumming weightedinputsandfiring(i.e.sending1ratherthan0asoutput)ifthesumwaslargerthana threshold(Hayman,1999;McCullochandPitts,1943).Thismodelanditscontinuousstate

66

successorsformthebasisofmostartificialneuralnetworkmodels.Theydonothaveany internalstateexceptthefiringlevel.Theirlinktorealbiologyissomewhattenuous,although asanabstractiontheyhavebeenveryfruitful. Morerealisticmodelssuchasintegrateandfiresumsynapticpotentialsandproduce spikes. Singlecellmodellingcanbedoneonroughlyfivelevelsofabstraction(Herz,Gollischetal., 2006):asblackboxmodulesthatgenerateprobabilisticresponsestostimuliaccordingtosome probabilitydistribution,asaseriesoflinearornonlinearfiltersofsignals,asasingle compartmentwithionicconductances,asareducedcompartmentmodel(fewcompartments) orasadetailedcompartmentalmodel.Themoreabstractmodelsaretheoreticallyand computationallytractableandhavefewerdegreesoffreedom,whilethemoredetailed modelsareclosertobiologicalrealismandcanbelinkedtoempiricaldata.Accordingto (Herz,Gollischetal.,2006),mosttaskspecificcomputationofknowndirectbiological relevancecanbeachievedbyreducedcompartmentalmodels,withthepossibleexceptionof someformsofnonlineardendriticprocessing.

Figure21:Compartmentmodelsofneuronsusuallybeginwithanelectrophysiologically characterizedneuron(A).Thiscanberegardedasanetworkofelectricalcableswith individualproperties(B).Thesearethensubdividedintoisopotentialcompartments.The systemcanthenbemodeledasanequivalentelectronicnetwork(C).Eachionchanneltype correspondstoapairofparallelconnectedpotentialsandvariableresistancesper compartment.Compartmentsimulationsnumericallysimulatetheequivalentnetwork8. Imagefrom(BowerandBeeman,1998) Conductancebasedmodelsarethesimplestbiophysicalrepresentationofneurons, representingthecellmembraneasacapacitorandthedifferentionchannelsas(variable) resistances.Neuronsorpartsofneuronsarereplacedbytheirequivalentcircuits,whichare

Seehttp://www.brainsmindsmedia.org/archive/222foratutorialinthismethodology.

67

thensimulatedusingordinarydifferentialequations9.Besidethemembranepotentialthey have(atleast)twogatingvariablesforeachmembranecurrentasdynamicalvariables. Thecoreassumptionsinconductancebasedmodelsarethatdifferentionchannelsare independentofeachother;thegatingvariablesareindependentofeachother,depending onlyonvoltage(orotherfactorssuchascalcium),firstorderkineticsinthegatingvariables; andthattheregionbeingsimulatedisisopotential. Morecomplexionchannelmodelswithinternalstatesinthechannelshavebeendeveloped, aswellasmodelsincludingcalciumdynamics(possiblywithseveralformsofcalcium buffering). Insimpleneuronmodels,theneuronic(firingrateupdate)equationscanbeuncoupled fromthemnemonic(synapticweightupdate)equations,theadiabaticlearning hypothesis(Caianiello,1961).However,realisticmodelsoftenincludeacomplexinterplayat synapsesbetweenmembranepotential,calciumlevels,andconductancesthatmakethis uncouplingdifficulttopreserve. Acommonguidelineinmodellingistousecompartments1/10to1/20ofthelengthconstant10 ofthedendrite(oraxon),makingpotentialdifferencesbetweencompartmentsdifferjustby 5%.Typicallengthconstantsareontheorderof25mm,givingcompartmentssmallerthan 200100m.Inheavilybranchingneurons,theshortdistancebetweenbranchesforcesfiner resolution,ontheorderof10morless.Itcanthereforebeexpectedthatthemajorityof compartmentswillbeduetocorticalarborsratherthanthelongaxonsthroughthewhite matter. Thenumberofstatevariablesofaneuronatleastscaleswiththenumberofsynapsessince eachsynapsehasitsowndynamics.Thenumberofsynapsesisalsoaroughestimateofthe numberofcompartmentsneededforanaccuratemorphologicalmodel.Eachcompartment hastostorealistofneighbourcompartments,dynamicalvariables,andlocalparameters. Synapsescanbetreatedasregularcompartmentswithextrainformationaboutweight, neurotransmitters,andinternalchemicalstate.Asynapseresolutioncompartmentmodelof 1011neuronswithontheorderof104compartmentswouldrequire1015compartments. Ifstatesinextracellularspacematter(e.g.duetovolumetransmission,ephapticsignalsor neurogenesis),itmaybenecessarytodividethesimulationintoaspatialgridratherthan compartmentsfollowingtheneurons.Avolumebasedsimulationwherethebrainisdivided intosizervoxelswouldencompass1.4103/r3voxels.Eachvoxelwouldcontaininformation aboutwhichcells,compartments,andotherinformationthatexistedinside,aswellasalistof thedynamicalvariables(localelectricfields,chemicalconcentrations)andlocalparameter values.For10msidevoxelstherewouldbe1.41018voxelsinahumanbrain.

Thismakesthesimplifyingassumptionthattheneuroncanbedividedintoisopotentialcompartments.Amore complexmodelwoulddescribethemembranestatebypartialdifferentialequations;thiswouldbehighly cumbersome.Fortunatelytherearenocurrentlyknownphenomenathatappearstorequiresuchmodels. 10Thelengthconstantdefineshowquicklyanimposedvoltagedecayswithdistanceinapassivecable,V(x)=V0 exp(x/),=((d/4)RM/RA)wheredisthediameter,RMthemembraneresistancepersquaremeterandRAtheaxial resistancepermeter.

9

68

Reducedmodels

Figure22:Computationalcostfordifferentneuronmodelsversusbiologicalplausibility. From(Izhikevich,2004),figure2. (Izhikevich,2004)reviewsbothtypicalneuralfiringpatternsandanumberofcomputational modelsofspikingneurons.Heestimatesboththenumberofpossiblebiologicalfeatures differentmodelscanachieveandhowmanyfloatingpointinstructionsareneededpermsof simulation(onlyassumingasomacurrent,nottakingtheeffectsofdendritesandsynapses intoaccount11): Table7:Neuronmodelcosts

Model Integrateandfire Integrateandfirewithadapt. Integrateandfireorburst Resonateandfire Quadraticintegrateandfire Izikhevich(2003) FitzHughNagumo HindmarshRose MorrisLecar Wilson HodgkinHuxley #ofbiologicalfeatures 3 5 10 12 6 21 11 18 14* 15 19* FLOPS/ms 5 10 13 10 7 13 72 120 600 180 1200

*OnlytheMorrisLecarandHodgkinHuxleymodelsarebiophysicallymeaningfulinthe sensethattheyattemptactuallytomodelrealbiophysics,theothersonlyaimforacorrect phenomenologyofspiking. The(Izhikevich,2003)modelisinterestingsinceitdemonstratesthatitmaybepossibleto improvetheefficiencyofcalculationssignificantly(twoordersofmagnitude)withoutlosing toomanyfeaturesoftheneuronactivity.Themodelitselfisatwovariabledynamicalsystem withtwomodelparameters.ItwasderivedfromtheHodgkinHuxleyequationsusinga bifurcationanalysismethodologykeepingthegeometryofphasespaceintact(Izhikevich, 2007).Whileitisnotdirectlybiophysicallymeaningful,itorsimilarreducedmodelsoffull biophysicsmaybepossiblecomputationalshortcutsinbrainemulation.Whethersuch reductionscanbedonedependsonwhetherornotthedetailsoninternalneuralbiophysics areimportantfornetworkrelevantpropertiessuchasexactspiketiming.Itmayalsobe
ToafirstapproximationeachcompartmentwouldrequirethesamenumberofFLOPSmakinga multicompartmentmodelabout34ordersofmagnitudemoredemanding.
11

69

possibletoapplyreductionmethodsonsubneuralmodels,buttheapproachrequiresan understandingofthegeometryofphasespaceofthesystem.

Simulators
Thereexistnumeroussimulationsystemsatpresent.SomeofthemorecommonareGENESIS (GEneralNEuralSImulationSystem)(Wilson,Bhallaetal.,1989;BowerandBeeman,1998) andNeuron(CarnevaleandHines,2006).Forareviewandcomparisons,see(Brette,Rudolph etal.,2007). Keyissuesforneuralsimulatorsarenumericalstability,extendabilityandparallelizability. Thenumericalmethodsusedtointegrateconductancebasedmodelsneedtobothproduce accurateapproximationofsolutionsofthegoverningequationsandrunfast.Thisismade moreproblematicbythestiffnessofsomeoftheequations.Mostneuralsimulatorshavebeen designedtobeeasytoextendwithnewfunctions,oftenproducingverycomplexsoftware systems.ForWBE,thismightnotbenecessaryoncethebasicneurosciencehasbeen elucidated.Neuralsimulatorsneedtobeabletorunonparallelcomputerstoreachhigh performance(seesectionbelow).ThisisakeyneedforWBE.

Parallel simulation
Networks,neurons,andcompartmentsareingeneraljustlinkedtonearbyentitiesandact simultaneously,makingbrainmodelsnaturallysuitedforparallelsimulations.Themain problemisfindingtherightgranularityofthesimulation(i.e.howmanyandwhichentities toputoneachprocessingnode)sothatcommunicationsoverheadisminimized,orfinding communicationsmethodsthatallowthenodestocommunicateefficiently. Simulationscanbetimedrivenoreventdriven.Atimedrivensimulationadvancesone timestepatatimewhileaneventdrivensimulationkeepsaqueueoffutureevents(suchas synapticspikesarriving)andadvancesdirectlytothenext.Forsomeneuralnetworkmodels, suchasintegrateandfire,thedynamicsbetweenspikescanbecalculatedexactly,allowing thesimulationefficientlyjusttojumpforwardintimetowhenthenextspikeoccurs(Mattia andGiudice,2000).However,forhighlyconnectednetworksthetimesbetweenthearrivals ofspikesbecomeveryshort,andtimedrivensimulationsareequallyefficient.Ontheother hand,thetimestepfortimedrivenmodelsmustbeshortenoughthatthediscretizationof spiketimingtoparticulartimestepsdoesnotdisrupttimingpatterns,orvarioustechniques forkeepingsubtimesteptiminginformationmustbeemployedinthesimulation(Morrison, Mehringetal.,2005). Following(Brette,Rudolphetal.,2007),thecomputationalcostpersecondofbiologicaltime isofordercUN/dt+cPFNpfortimedrivensimulationsand(cU+cS+cQ)FNpforeventdriven simulations.cUisthecostperupdate,cPthecostforonespikepropagation(assumedtobe smallerthantheupdatecost),cQthecostofenqueuing/dequeuingaspike(assumedtobe constant),Nthenumberofneurons,Ftherateoffiring,pthenumberofsynapsesperneuron anddtthetimestep.ForF=1Hz,p=10,000anddt=0.1ms,thetimestepdependenttermofthe firstequationislikelytodominate:smallertimestepscanincreasethecomputationalcost strongly.Inthesecondequationthereisnotimeconstantdependency,butthemultiplicative termislikelytobelargeandcansufferifthequeuemanagementhasanontrivialcost.The effectivetimeconstantinaneventdrivensimulationisgoingtobeontheorderof1/Fp, makingthetimeandeventdrivensimulationsaboutequallyfast.Iftherearegapjunctions

70

ordendrodendriticinteractionsthetimecomplexitybecomesmuchhigher(Brette,Rudolph etal.,2007). Wellimplementedsimulationstendstoscalelinearlywithnumberofprocessors,although variousmemoryandcommunicationsbottlenecksmayoccurandoptimaluseofcachingcan giveevensuperlinearspeedupforsomeproblemsizes(Djurfeldt,Johanssonetal.,2005; Migliore,Canniaetal.,2006).Themainproblemappearstobehighconnectivity,sinceinter processorcommunicationsareamajorbottleneck.Keepingcommunicationstoaminimum, forexamplebyonlysendinginformationaboutwhenandwhereaspikehasoccurred (JohanssonandLansner,2007)orrunningdendriticsubtreesonthesameprocessor(Hines, Markrametal.,2008),improvesperformancesignificantly.Ifbrainemulationrequiresmore informationthanthistoflowbetweenprocessingnodesperformancewillbelowerthanthese examples.

Current large-scale simulations

Representativecurrentlargescalebrainsimulationsincludethefollowing(seealsoAppendix Cforfurtherexamples). Asimulationof16Purkinjecellswith4,500compartmentsreceivinginputfrom244,000 granulecellstook2.5hoursona128processorCray3TE(nominally76.8GFLOPS)to calculate2secondsofsimulatedactivity(Howell,DyhrfjeldJohnsenetal.,2000).Thisimplies around0.3MFLOPSperneuron12andaslowdownfactorof4,500. TheBlueBrainProjectaimsatmodellingmammalianbrainsatvariousscalesandlevels,in particularmodellingacompletecorticalcolumninbiologicaldetail(Markram,2006).The corticalcolumnusesabout10,000morphologicallycomplexneuronsconnectedwith108 synapsesonan8,000processorBlueGenesupercomputer. Thesofar(2006)largestsimulationofafullHodgkinHuxleyneuronnetworkwasperformed ontheIBMWatsonResearchBlueGenesupercomputerusingthesimulatorSPLIT (HammarlundandEkeberg,1998;Djurfeldt,Johanssonetal.,2005).Itwasamodelofcortical minicolumns,consistingof22million6compartmentneuronswith11billionsynapses,with spatialdelayscorrespondingtoa16cm2cortexsurfaceandasimulationlengthofonesecond realtime.Mostofthecomputationalloadwasduetothesynapses,eachholding3state variables13.Theoverallnominalcomputationalcapacityusedwas11.5TFLOPS,giving0.5 MFLOPSperneuronor1045FLOPSpersynapse.Simulatingonesecondofneuralactivity took5,942s14.Thesimulationshowedlinearscalinginperformancewiththenumberof processorsupto4,096butbegantoshowsome(23%)overheadfor8,192processors (Djurfeldt,Lundqvistetal.,2006). Asimulationinvolving8millionsimplespikingIzhikevichneuronswith6,300synapse/cell withalowfiringrateandSTDPachievedaslowdownofjust10ona4,096processorBlue Genesupercomputer.Thiswasachievedbybothoptimizingthesynapticupdatesandby reducingthenumberofinterprocessormessagesbycollectingallspikesfromagivennode thatweresenttoanothernodeintoasinglemessage(Frye,Ananthanarayananetal.,2007).
Thisalsoincludesoverhead.ThePurkinjecellmodelsusedsignificantlymorecomputationsper neuronthanthegranulecellmodels. 13Conductance,depressionandfacilitation. 14MikaelDjurfeldt,personalcommunication.
12

71

 Anevenlargersimulationwith1011neuronsand1015synapseswasdonein2005byEugene M.IzhikevichonaBeowulfclusterwith273GHzprocessors(Izhikevich,2005).Thiswas achievedbynotstoringthesynapticconnectivitybutbygeneratingitwheneveritwas needed,makingthismodelratherillsuitedforbrainemulation.Onesecondofsimulation took50days,givingaslowdownfactorof4.2million. (JohanssonandLansner,2007)estimatethecomputationaldemandsforsimulatingthe mammalianneocortexofdifferentspecies,basedonaparticularmodelofitsfunction. Assumingthatminicolumnsorganizedintohypercolumnsarethecomputationalunitsand thatthesecommunicatethroughspiketrainsandusingaclusterarchitecture,theyshowthat itisfeasibletorunanetworkofintermediatesizebetweenratandcatonacurrent supercomputercluster.Theyclaimthatusingthiskindofhighlevelnetwork(andassuming continuedlinearscaling)amacaquesizedcortexcouldberunonaBlueGene/Linrealtime.

Conclusion
Currentneuralsimulationstendtohaveneuronnumbersandstructuresfittedtothe availablecomputers(inparticularintermsofconnectivity).Thisisimportantforthelinear scalingofspeedwithprocessornumberandgivesaperformanceimprovementatthe expenseofgenerality.WBEscalecomputingwilllikelyhaveacomputationalgranularity fittedtothestructureofthebrain,unlessthecomputationalpoweravailableissolargethat inefficienciesduetoaloosefitareirrelevant. Modelsarecurrentlynotdrivenbyscanneddata,andindividualvariationsinneuron propertiesaregeneratedbydrawingfromapresetrandomdistribution.Whilethisallows modelsthatgenerateneededparametersonlywhenrequiredandhenceavoidstoringthem, thisisrelativelyrareandmostimplementationsdousestoredparameters.Hencetheredoes notseemtobeanydifferencebetweenstoragerequirementsforWBEmodelsandrandom models,assumingthesameunderlyingparametersandstructure. Thespeedupofdifferentnetworksrangesoversixordersofmagnitude,butmostarea hundredfoldtoathousandfoldslowerthanbiology,andafewareclosetorealtime.Thisis likelymoreduetoresearchpracticethananyinherentlimitation:modelsizesareselectedso thattheyfitavailablecomputingpower.Thelengthofthesimulationissettoproducedata comparabletosomebiologicalmeasurement(which,forneuralnetworks,tendtobeafew secondslongforneurocognitivelyinterestingcases),andthelengthofsimulationtime correspondstowhatconstitutesanacceptableturnaroundtimeforacomputercentreoran officecomputer.Smallersimulationscanrunfasteruntiltheyhitbottleneckssetbyinter processorcommunicationsspeed.ItishencelikelythatevenearlyWBEwillbeclosetoreal time,withcomputationalconstraintsratherthanspeedlimitingthelengthofthe simulation. Weclearlytodayhavecomputationalcapabilitiessufficienttosimulatemanyinvertebrate brains(suchassnails,ants,andfruitflies)usingcompartmentmodelsonparallelclustersof modestsize.Smallmammalianbrainsappearcomputationallywithinreach. Giventhedifferencesbetweenmodels,implementation,computersandotherparametersitis hardtoreliablycomparecurrentlargescalemodelstoestimatetrends.Thereisalsoalackof historicaldata,asonlyrecentlylargemodelshavebecomepracticallypossible.Hencea conservativeestimateassumesthatthemodelswillgrowbasedsolelyoncomputerpower

72

andnotonanyalgorithmicimprovement.Ifmodelsreliablyexpressingsublinear computationaldemandsappear,theywouldlikelymakelargescalecomputerpower irrelevantasalimitingfactorforWBE. Themajorcomputationalloadscaleswiththenumberofsynapsesor(forcompartment models)compartments,sincetheyarethemostnumerousentities.Hence,onelikelyuseful metricwouldbethenumberofsynapticupdatespersecond,aswellasthecomputational demandspersynapse.

73

Bodysimulation
Thebodysimulationtranslatesbetweenneuralsignalsandtheenvironment,aswellas maintainsamodelofbodystateasitaffectsthebrainemulation. Howdetailedthebodysimulationneedstobeinordertofunctiondependsonthegoal.An adequatesimulationproducesenoughandtherightkindofinformationfortheemulation tofunctionandact,whileaconvincingsimulationisnearlyorwhollyindistinguishablefrom thefeeloftheoriginalbody. Anumberofrelativelysimplebiomechanicalsimulationsofbodiesconnectedtosimulated nervoussystemshavebeencreatedtostudylocomotion.(Suzuki,Gotoetal.,2005)simulated theC.elegansbodyasamultijointrigidlinkwherethejointswerecontrolledby motorneuronsinasimulatedmotorcontrolnetwork.rjanEkeberghassimulated locomotioninlamprey(EkebergandGrillner,1999),stickinsects(Ekeberg,Blmeletal., 2004),andthehindlegsofcat(EkebergandPearson,2005)wherearigidskeletonismoved bymuscleseithermodeledasspringscontractinglinearlywithneuralsignals,orinthecase ofthecat,amodelfittingobserveddatarelatingneuralstimulation,length,andvelocitywith contractionforce(Brown,Scottetal.,1996).Thesemodelsalsoincludesensoryfeedbackfrom stretchreceptors,enablingmovementstoadapttoenvironmentalforces:locomotioninvolves aninformationloopbetweenneuralactivity,motorresponse,bodydynamics,andsensory feedback(Pearson,Ekebergetal.,2006). Todaybiomechanicalmodelsoftwareenablesfairlydetailedmodelsofmuscles,theskeleton, andthejoints,enablingcalculationofforces,torques,andinteractionwithasimulated environment(BiomechanicsResearchGroupInc,2005).Suchmodelstendtosimplifymuscles aslinesandmakeuseofprerecordedmovementsortensionstogeneratethekinematics. Adetailedmechanicalmodelofhumanwalkinghasbeenconstructedwith23degreesof freedomdrivenby54muscles.However,itwasnotcontrolledbyaneuralnetworkbutrather usedtofindanenergyoptimizinggait(AndersonandPandy,2001).Astateoftheartmodel involving200rigidboneswithover300degreesoffreedom,drivenbymuscularactuators withexcitationcontractiondynamicsandsomeneuralcontrol,hasbeendevelopedfor modellinghumanbodymotioninadynamicenvironment,e.g.forergonomicstesting (IvancevicandBeagley,2004).Thismodelrunsonanormalworkstation,suggestingthat rigidbodysimulationisnotacomputationallyhardproblemincomparisontoWBE. Otherbiomechanicalmodelsarebeingexploredforassessingmusculoskeletalfunctionin human(FernandezandPandy,2006),andcanbevalidatedorindividualizedbyuseofMRI data(Arnold,Salinasetal.,2000)orEMG(LloydandBesier,2003).Itisexpectedthatnear futuremodelswillbebasedonvolumetricmuscleandbonemodelsfoundusingMRI scanning(Blemker,Asakawaetal.,2007;BlemkerandDelp,2005),aswellasconstructionof topologicalmodels(MagnenatThalmannandCordier,2000).Therearealsovarious simulationsofsofttissue(Benham,Wrightetal.,2001),breathing(Zordan,Cellyetal.,2004) andsofttissuedeformationforsurgerysimulation(Cotin,Delingetteetal.,1999). Anothersourceofbodymodelscomesfromcomputergraphics,wheremuchefforthasgone intorenderingrealisticcharacters,includingmodellingmuscles,hairandskin.Theemphasis hasbeenonrealisticappearanceratherthanrealisticphysics(Scheepers,Parentetal.,1997), butincreasinglythemodelsarebecomingbiophysicallyrealisticandoverlappingwith

74

biophysics(ChenandZeltzer,1992;Yucesoy,Koopmanetal.,2002).Forexample,30 contact/collisioncoupledmusclesintheupperlimbwithfasciaandtendonsweregenerated fromthevisiblehumandatasetandthensimulatedusingafinitevolumemethod;this simulation(usingonemillionmeshtetrahedra)ranatarateof240secondsperframeona singleCPUXeon3.06GHz(ontheorderofafewGFLOPS)(Teran,Sifakisetal.,2005).Scaling thisup20timestoencompass600musclesimpliesacomputationalcostontheorderofa hundredTFLOPSforacompletebodysimulation. Physiologicalmodelsareincreasinglyusedinmedicineforeducation,researchandpatient evaluation.Relativelysimplemodelscanaccuratelysimulatebloodoxygenation(Hardman, Bedforthetal.,1998).Forabodysimulationthismightbeenoughtoprovidetheright feedbackbetweenexertionandbrainstate.Similarlysimplenutrientandhormonemodels couldbeusedinsofararealisticresponsetohungerandeatingweredesired.

Conclusion
Simulatingarealistichumanbodyiskinematicallypossibletoday,requiringcomputational powerrangingbetweenworkstationsandmainframes.Forsimplerorganismssuchas nematodesorinsectscorrespondinglysimplermodelscould(andhave)beenused.Sincethe needforearlyWBEismerelyadequatebodysimulation,thebodydoesnotappeartoposea majorbottleneck. However,sincemanymotoractionsinvolvearichinterplaybetweenmechanicsandnerve signalsitshouldnotbesurprisingifrelativelycomplexmodelsareneededforapparently simpleactionssuchasstandingorwalking.Newlystartedemulationsmayneedtimeand efforttolearntousetheirunfamiliarbodies.

75

Environmentsimulation
Theenvironmentsimulationprovidesasimulatedphysicalenvironmentforthebody simulation.Onecanagainmakethedistinctionbetweenanadequateenvironmentsimulation andaconvincingsimulation.Anadequateenvironmentproducesenoughinputtoactivate thebrainemulationandallowittointeractinsuchawaythatitsstateandfunctioncanbe evaluated.Aconvincingsimulationiscloseenoughtorealitythatthekindsofsignalsand interactionthatoccursishard(orimpossible)fortheemulatedbeingtodistinguishfrom reality. Itseemslikelythatwealreadyhavethetoolsformakingadequateenvironmentsintheform ofe.g.game3Drenderingengineswithphysicsmodelsorvirtualenvironmentssuchas SecondLife.Whilenotcoveringmorethansightandsound,theymightbeenoughfortesting anddevelopment.ForemulationsofsimplerbrainssuchasC.eleganssimulationswith simplifiedhydrodynamics(similarto(EkebergandGrillner,1999))maybeenough,possibly extendedwithsimulatedchemicalgradientstoguidebehaviour. Convincingenvironmentsmightbenecessaryonlyifthelongtermmentalwellbeingof emulatedhumans(orothermammals)isatstake.Whileitispossiblethatahumancould adapttoamerelyadequateenvironment,itseemslikelythatitwouldexperiencesuchan environmentasconfiningorlackinginsensorystimulation.Notethateveninaconvincing environmentsimulationnotalldetailshavetofitphysicalrealityperfectly(Bostrom,2003). Plausiblesimulationismoreimportantthanaccuratesimulationinthisdomainandmay actuallyimprovetheperceivedrealism(Barzel,Hughesetal.,1996).Inaddition,humans acceptsurprisinglylargedistortions(20%lengthchangeofobjectswhennotpayingdirect attention,3%whenpayingattention(Harrison,Rensinketal.,2004)),allowingagreatdealof leewayinconstructingaconvincingenvironment. Whatqualityofenvironmentisneededtocompletelyfoolthesenses?Inthefollowingwe willassumethatthebrainemulationrunsinrealtime,i.e.,thatonesecondofsimulationtime correspondstoonesecondofoutsidetime.Forsloweremulations,theenvironmentmodel wouldbeslowedcomparably,andallcomputationaldemandsdividedbythescalefactor. Atthecoreoftheenvironmentmodelwouldbeaphysicsenginesimulatingthemechanical interactionsbetweentheobjectsintheenvironmentandthesimulatedbody.Itwouldnot onlyupdateobjectpositionsdependingonmovementandmaintainaplausiblephysics,it wouldalsoprovidecollisionandcontactinformationneededforsimulatedtouch.Ontopof thisphysicssimulationaseriesofrenderingenginesfordifferentsenseswouldproducethe rawdataforthesensesinthebodymodel.

Vision
Visualphotorealismhasbeensoughtincomputergraphicsforabout30years,andthis appearstobeafairlymatureareaatleastforstaticimagesandscenes.Mucheffortis currentlygoingintosuchtechnology,foruseincomputergamesandmovies. (McGuigan,2006)proposesagraphicsTuringtestandestimatesthatfor30Hzinteractive visualupdates518.41036.8TFLOPSwouldbeenoughforMonteCarloglobalillumination. Thismightactuallybeanoverestimatesinceheassumesgenerationofcompletepictures. Generatingonlythesignalneededfortheretinalreceptors(withhigherresolutionforthe

76

foveathantheperiphery)couldpresumablyreducethedemands.Similarly,moreefficient implementationsoftheilluminationmodel(oracheaperone)wouldalsoreducedemands significantly.

Hearing
Thefullacousticfieldcanbesimulatedoverthefrequencyrangeofhumanhearingby solvingthedifferentialequationsforairvibration(Garriga,Spaetal.,2005).Whileaccurate, thismethodhasacomputationalcostthatscaleswiththevolumesimulated,upto16TFLOPS fora222mroom.Thiscanlikelybereducedbytheuseofadaptivemeshmethods,orray orbeamtracingofsound(Funkhouser,Tsingosetal.,2004). Soundgenerationoccursnotonlyfromsoundsourcessuchasinstruments,loudspeakers,and peoplebutalsofromnormalinteractionsbetweenobjectsintheenvironment.Bysimulating surfacevibrations,realisticsoundscanbegeneratedasobjectscollideandvibrate.Abasic modelwithNsurfacenodesrequires0.5292NGFLOPS,butthiscanbesignificantlyreduced bytakingperceptualshortcuts(RaghuvanshiandLin,2006;RaghuvanshiandLin,2007).This formofvibrationgenerationcanlikelybeusedtosynthesizerealisticvibrationsfortouch.

Smell and Taste

Sofarnoworkhasbeendoneonsimulatedsmellandtasteinvirtualreality,mainlydueto thelackofoutputdevices.Somesimulationsofodorantdiffusionhavebeendonein underwaterenvironments(BairdRC,JohariHetal.,1996)andinthehumanandratnasal cavity(Keyhani,Schereretal.,1997;Zhao,Daltonetal.,2006).Ingeneral,anodorsimulation wouldinvolvemodellingdiffusionandtransportofchemicalsthroughairflow;andthe relativelylowtemporalandspatialresolutionofhumanolfactionwouldlikelyallowafairly simplemodel.Afarmoreinvolvedissueiswhatodorantmoleculestosimulate.Humans have350activeolfactoryreceptorgenes,butwecanlikelydetectmorevariationdueto differentdiffusioninthenasalcavity(Shepherd,2004). Tasteappearsevensimplerinprincipletosimulatesinceitonlycomesintoplaywhenobjects areplacedinthemouthandthenonlythroughahandfulofreceptortypes.However,the tastesensationisacomplexinterplaybetweentaste,smell,andtexture.Itmaybenecessaryto haveparticularlyfinegrainedphysicsmodelsofthemouthcontentsinordertoreproducea plausibleeatingexperience.

Haptics
Thehapticsensesoftouch,proprioception,andbalancearecrucialforperformingskilled actionsinrealandvirtualenvironments(RoblesDeLaTorre,2006). Tactilesensationrelatesbothtotheforcesaffectingtheskin(andhair)andtohowtheyare changingasobjectsorthebodyaremoved.Tosimulatetouch,stimulicollisiondetectionis neededtocalculateforcesontheskin(andpossiblydeformations)aswellasthevibrations whenitismovedoverasurfaceorexploringitwithahardobject(Klatzky,Ledermanetal., 2003).Toachieverealistichapticrendering,updatesinthekilohertzrangemaybenecessary (LinandOtaduy,2005).Inenvironmentswithdeformableobjectsvariousnonlinearitiesin responseandrestitutionhavetobetakenintoaccount(MahvashandHayward,2004).

77

Proprioception,thesenseofhowfarmusclesandtendonsarestretched(andbyinference, limblocation)isimportantformaintainingpostureandorientation.Unliketheothersenses, proprioceptivesignalswouldbegeneratedbythebodymodelinternally.SimulatedGolgi organs,musclespindles,andpulmonarystretchreceptorswouldthenconvertbodystates intonerveimpulses. Thebalancesignalsfromtheinnerearappearsrelativelysimpletosimulate,sinceitisonly dependentonthefluidvelocityandpressureinthesemicircularchannels(whichcanlikely beassumedtobelaminarandhomogeneous)andgravityeffectsontheutricleandsaccule. Comparedtoothersenses,thecomputationaldemandsareminuscule. Thermoreceptioncouldpresumablybesimulatedbygivingeachobjectinthevirtual environmentatemperature,activatingthermoreceptorsincontactwiththeobject. Nocireception(pain)wouldbesimulatedbyactivatingthereceptorsinthepresenceof excessiveforcesortemperatures;theabilitytoexperiencepainfromsimulatedinflammatory responsesmaybeunnecessaryverisimilitude.

Conclusion
Renderingaconvincingenvironmentforallsensesprobablyrequiresontheorderofseveral hundredTFLOPS.Whilesignificantbytodaysstandards,thisrepresentsaminusculefraction ofthecomputationalresourcesneededforbrainemulation,andisnotnecessaryformeeting thebasicsuccesscriteriaofemulation.

78

Computerrequirements
WBErequiressignificantcomputerpowerandstorageforimageprocessingand interpretationduringthescanningprocess,andtoholdandruntheresultingemulation.Both problemsappeartobestronglyparallelizable. Awayofestimatingthedistancebetweencurrentcapabilitiesandthoseneededforhuman WBEistoestimatethenumberofentities/statevariablesneededtospecifytheemulation,the requiredtimeresolution,andcomparethistotrendsincomputerhardware.Thiswilldepend onwhatlevelofdetailaWBEcanbeachievedat;fullmolecularsimulationwouldrequire vastlymorecomputationalpowerthanamodelusingsimplifiedneurons.Foragivenlevelof simulation,wecanthenestimatetheearliestpossibledateassumingthatMooreslaw continuesunchangedwhenthatkindofemulationwillbepossibleforagivenamountof money. AppendixBanalysescurrentcomputingtrendsindetail.Themainconclusionisthatmemory perdollarincreasesoneorderofmagnitudeper4.8yearsandprocessingpowerperdollar increasesoneorderofmagnitudeper3.7,3.9or6.4yearsdependingonwhetheronebasesthe predictiononsupercomputerprice/performance,supercomputerpowerorcommodity computers.Wewillusetheoptimisticsupercomputerestimateandthemorecautious commodityestimatestogetanoverallrange. Itshouldbenotedthattheestimatesbelowmakemerelyorderofmagnitudeestimatesofthe numberofentitiesandcomplexityoftheirstorage;theyarequitedebatable.However,given thatanorderofmagnitudecomplexityincreaseonlyaddscirca5yearstotheestimate,exact numbersarenotnecessary.Wearealsoignoringbodyandenvironmentsimulationbecause, asshowninthenextsections,theylikelyrequireonlyafractionofthebrainemulation computations. Table8:Storagedemands(emulationonly,humanbrain)
Level 1 2 Computational module Brainregion connectivity Analognetwork populationmodel #entities 1001,000? 105regions,107 connections 108populations,1013 connections. Bytesper entity ? 3?(2byte connectivity,1 byteweight) 5(3byte connectivity,1 byteweight,1 byteextrastate variable) 8(4byte connectivity,4 statevariables) 1byteperstate variable 1byteperstate variable 1byteperstate variable 1byteperstate variable 1byteperstate Memory demands(Tb) ? 3105 Earliestyear, $1million ? Present

50

Present

Spikingneural network Electrophysiology Metabolome Proteome

1011neurons,1015 connections. 1015compartmentsx10 statevariables=1016. 1016compartmentsx102 metabolites=1018. 1016compartmentsx103 proteinsandmetabolites =1019. 1016compartmentsx103 proteinsx10states=1020 1016compartmentsx103

8,000

2019

5 6 7

10,000 106 107

2019 2029 2034

8 9

Statesofprotein complexes Distributionof

108 109

2038 2043

79

complexes Full3DEMmap (Fiala,2002) Stochasticbehaviour ofsinglemolecules

proteinsandmetabolites x100states/locations. 50x2.5x2.5nm

variable 1byteper voxel, compressed. 31(2bytes moleculetype, 14bytes position,14 bytesvelocity,1 bytestate) Qbits 109 2043

10

1025molecules

3.11014

2069

11

Quantum

Either1026atoms,or smallernumberof quantumstatecarrying molecules.

ForthecaseofaManhattanprojectspending$109,subtract14.4yearsfromtheseestimates. Table9:Processingdemands(emulationonly,humanbrain)
Level #entities FLOPS per entity Timesteps persecond CPU demand (FLOPS) Earliest year,$1 million (commod ity computer estimate) ? ? Earliest year,$1 million (Super computer estimate) ? ?

1 2

Computational module Brainregion connectivity Analog network population model Spikingneural network Electrophysiol ogy

1001,000? 105regions, 107 connections 108 populations, 1013 connections 1011neurons, 1015 connections 1015 compartments x10state variables= 1016. 1016 compartments x102 metabolites= 1018. 1016 compartments x103proteins and metabolites= 1019. 1016 compartments x103proteins x10states= 1020 1016

? ?

? ?

? ?

102

1015

2023

200815

10

103

1018

2042

2019

Metabolome

103 FLOPS perms per compart ment. 103

104

1022

2068

2033

104

1025

2087

2044

Proteome

103

104

1026

2093

2048

Statesof protein complexes

103

104

1027

2100

2052

9
15

Distributionof

103

106

1030

2119

2063

RoadrunneratLosAlamosNationalLaboratoryachieved1.7petaflopsonMay25,2008.

80

complexes

10

11

Stochastic behaviorof single molecules Quantum

compartments x103proteins and metabolitesx 100statesper location=1021. 1025molecules

103

1015

1043

2201

2111

Either1026 atoms,or smaller numberof quantumstate carrying molecules.

Qbits

10131020

ForthecaseofaManhattanprojectspending$109,subtract19.1/11.1yearsfromthese estimates,respectively. Aroughestimateforsimplerbrainsisthatamacaquebrainhas14%ofthehumansynapses, catbrains3%,rat0.26%,mouse0.1%(JohanssonandLansner,2007).Assumingsimulation demandsscalebysynapsenumber(likelyforcompartmentmodels)thismeansmacaque emulationsonagivenlevelcanbeachieved5.4/3.2yearsearlier,catemulations9.7/5.6years, ratemulations16/9.6yearsandmouseemulations19/11(dependingonwhichcolumnabove isused).Emulationsofhoneybees(950,000neurons)andaplysia(20,000)appearfeasible todayatafairlyhighscale(51/30,61/36yearsearlierrespectively).Aslowerdecadetimeof computerimprovementproducesalongergapbetweenanimalemulationsandhuman emulations.

Conclusions
Itappearsfeasiblewithintheforeseeablefuturetostorethefullconnectivityoreven multistatecompartmentmodelsofallneuronsinthebrainwithintheworkingmemoryofa largecomputingsystem. Achievingtheperformanceneededforrealtimeemulationappearstobeamoreserious computationalproblem.However,theuncertaintiesinthisestimatearealsolargersinceit dependsonthecurrentlyunknownnumberofrequiredstates,thecomputationalcomplexity ofupdatingthem(whichmaybeamenabletodrasticimprovementsifalgorithmicshortcuts canbefound),thepresumedlimitationofcomputerhardwareimprovementstoaMoores lawgrowthrate,andtheinterplaybetweenimprovingprocessorsandimproving parallelism16.Aroughconclusionwouldneverthelessbethatifelectrophysiologicalmodels areenough,fullhumanbrainemulationsshouldbepossiblebeforemidcentury.Animal modelsofsimplemammalswouldbepossibleonetotwodecadesbeforethis.

16ThiscanbeseenintheongoingdebatesaboutwhetherconsumerGPUperformanceshouldberegardedasbeingin the$0.2rangeperGFLOPS;ifWBEcanbemappedtosuchhighperformancecheapspecialpurposehardware severalordersofmagnitudeofimprovementcanbeachieved.

81

Validation
Ascomputersoftwareincreasedincomplexity,previousmethodsofdebuggingbecame insufficient,especiallyassoftwaredevelopmentmovedfromsmallgroupstolargeprojectsin largeorganisations.Thisledtothedevelopmentofanumberofsoftwaretesting methodologiesaimingatimprovingquality(GelperinandHetzel,1988).Currently neuroscienceappearstobeinanearlydebuggingparadigmwheredataandprocedures certainlyaretestedinvariousways,butusuallyjustthroughreplicationorasanadhoc activitywhensomethingunexpectedoccurs.Forlargescaleneurosciencetestingand validationmethodsneedtobeincorporatedintheresearchprocesstoensurethattheprocess works,thatthedataprovidedtootherpartsoftheresearchisaccurateandthatthelink betweenrealityandmodelisfirm. Animportantearly/ongoingresearchgoalwouldbetoquantifytheimportanceofeachlevel ofscaleofphenomenatotheresultantobservedhigherbrainfunctionsofinterest.In particular,itisimportanttoknowtowhatlevelofdetailtheyneedtobesimulatedinorderto achievethesamekindofemergentbehaviouronthenextlevel.Insomecasesitmightbe possibletoprovethisbyperformingsimulations/emulationsatdifferentlevelsof resolutionandcomparingtheirresults.Thiswouldbeanimportantapplicationofearly smallscaleemulationsandwouldhelppavethewayforlargerones. Ideally,agoldstandardmodelatthehighestpossibleresolutioncouldbeusedtotestthe extenttowhichitispossibletodeviatefromthisbeforenoticeableeffectsoccur,andto determinewhichfactorsareirrelevantforhigherlevelphenomena.Someofthisinformation mayalreadyexistintheliterature,someneedstobediscoveredthroughnewcomputational neuroscienceresearch.Explorationofmodelfamilieswithdifferentlevelsofbiologicaldetail isalreadydoneoccasionally. Acomplementaryapproachistodevelopmanufactureddatawherethegroundtruthis known(phantomdatasets),andthenapplyreconstructionmethodsonthisdatatoseehow welltheycandeducethetruenetwork.Forexample,theNETMORPHsystemmodelsneurite outgrowthwhichcreatesdetailedneuralmorphologyandnetworkconnectivity(Koene, 2008).Thiscanthenbeusedtomakevirtualslices.Multipledatasetscanbegeneratedtotest theoverallreliabilityofreconstructionmethods.

82

Discussion
Asthisreviewshows,WBEontheneuronal/synapticlevelrequiresrelativelymodest increasesinmicroscopyresolution,alesstrivialdevelopmentofautomationforscanningand imageprocessing,aresearchpushattheproblemofinferringfunctionalpropertiesof neuronsandsynapses,andrelativelybusinessasusualdevelopmentofcomputational neurosciencemodelsandcomputerhardware.Thisassumesthatthisistheappropriatelevel ofdescriptionofthebrain,andthatwefindwaysofaccuratelysimulatingthesubsystems thatoccuronthislevel.Conversely,pursuingthisresearchagendawillalsohelpdetect whethertherearelowleveleffectsthathavesignificantinfluenceonhigherlevelsystems, requiringanincreaseinsimulationandscanningresolution. Theredonotappeartoexistanyobstaclestoattemptingtoemulateaninvertebrateorganism today.Wearestilllargelyignorantofthenetworksthatmakeupthebrainsofevenmodestly complexorganisms.Obtainingdetailedanatomicalinformationofasmallbrainappears entirelyfeasibleandusefultoneuroscience,andwouldbeacriticalfirststeptowardsWBE. Suchaprojectwouldserveasbothaproofofconceptandatestbedforfurtherdevelopment. IfWBEispursuedsuccessfully,atpresentitlooksliketheneedforrawcomputingpowerfor realtimesimulationandfundingforbuildinglargescaleautomatedscanning/processing facilitiesarethefactorsmostlikelytoholdbacklargescalesimulations.

83

AppendixA:Estimatesofthecomputational capacity/demandsofthehumanbrain
Themostcommonapproachisastraightforwardmultiplicativeestimate:giventhenumberof neurons,theaveragenumberofsynapsesandanassumedamountofinformationper synapseornumberofoperationspersecondpersynapse.Thismultiplicativemethodhas beenappliedtomicrotubuliandproteinstoo. However,itstillmightbenecessarytostoreconcentrationsofseveralchemicalspecies, neurotransmittertypesandotherdataifabiologicallyrealisticmodelisneeded(especially theidentitiesofthepreandpostsynapticneurons).Someestimatesofthestorage requirementsofbrainemulationareincludedinthetablebelow. Otherestimationmethodsarebasedonanalogyorconstraints.(Moravec,1999)suggested exploitingtheknownrequirementsofimageprocessingbyequatingthemwitha correspondingneuralstructure(theretina),andthenscalinguptheresult.(Merkle,1989a) usedenergyconstraintsonelementaryneuraloperations.(Landauer,1986)attemptedan estimationbasedonexperimentalpsychologicalmemoryandsignaltheory. Assumptionontheorderofonebitofinformationpersynapsehassomesupporton theoreticalgrounds.Modelsofassociativeneuralnetworkshaveaninformationstorage capacityslightlyunder1bitpersynapsedependingonwhatkindofinformationisencoded (Nadal,1991;NadalandToulouse,1990).Extendingthedynamicsofsynapsesforstoring sequencedatadoesnotincreasethiscapacity(RehnandLansner,2004).Geometricaland combinatorialconsiderationssuggest35bitspersynapse(Stepanyants,Hofetal.,2002; Kalisman,Silberbergetal.,2005).FittingtheoreticalmodelstoPurkinjecellssuggeststhat theycanreach0.25bits/synapse(Brunel,Hakimetal.,2004). Table10:Estimatesofcomputationalcapacityofthehumanbrain.Unitshavebeen convertedintoFLOPSandbitswheneverpossible.LevelsrefertoTable2.
Source (Leitl,1995) Assumptions Assuming1010neurons,1,000 synapsesperneuron,34bitIDper neuronand8bitrepresentationof dynamicstate,synapticweights anddelays.[Level5] Assumingmicrotubulidimerstates asbitsandoperatingon nanosecondswitchingtimes. [Level10] Basedon100billionneuronswith 1,000connectionsand200 calculationspersecond.[Level4] Arguesthatthenumberof computationalelementsinthe brainisgreaterthanthenumberof neurons,possiblyevenuptothe 1017individualproteinmolecules. [Level8] Assuming2bitslearningper secondduringconscioustime, experimentbased.[Level1] Computational demands Memory 51015bits(butnotes thatthedatacanlikely becompressed). 81019bits

(Tuszynski,2006)

1028FLOPS

(Kurzweil,1999) (Thagard,2002)

21016FLOPS

1012bits

1023FLOPS

(Landauer,1986)

1.5109bits(109bits withloss)

84

(Neumann,1958) (Wang,Liuetal.,2003) (FreitasJr.,1996) (Bostrom,1998)

(Merkle,1989a) (Moravec,1999;Morevec, 1988;Moravec,1998)

(Merkle,1989a) (Dix,2005)

Storingallimpulsesoveralifetime. Memoriesarestoredasrelations betweenneurons. 1010neurons,1,000synapses,firing 10Hz[Level4] 1011neurons,5103synapses,100 Hz,eachsignalworth5bits.[Level 5] EnergyconstraintsonRanvier nodes. Comparesinstructionsneededfor visualprocessingprimitiveswith retina,scalesuptobrainand10 timespersecond.Produces1,000 MIPSneurons.[Level3] Retinascaleup.[Level3] 10billionneurons,10,000synaptic operationspercycle,100Hzcycle time.[Level4] 1010neurons,1,000synapseseach. [Level4] 1014synapses,identitycodedby48 bitsplus2x36bitsforpreand postsynapticneuronid,1byte states.10msupdatetime.[Level4] 50200billionneurons,20,000 sharedsynapsesperneuronwith 256distinguishablelevels,40Hz firing.[Level5] 1011neurons,102104synapses,100 1,000Hzactivity.[level4] 11011neurons,eachwith104 compartmentsrunningthebasic HodgkinHuxleyequationswith 1200FLOPSeach(basedon (Izhikevich,2004)).Each compartmentwouldhave4 dynamicalvariablesand10 parametersdescribedbyonebyte each.

1014bits/second 1017operationsper second 21015operationsper second(10131016ops/s) 108MIPS

1020bits 108432bits(Seefootnote 17)

81014bits.

10121014operationsper second. 1016synapticops/s

41015bits(for structuralinformation) 1013bits 21016bits(for structuralinformation)

(Cherniak,1990) (Fiala,2007)

256,000terabytes/s

(Seitz)

21012synaptic operationspersecon

4101581015bits

(Malickas,1996)

10151018synaptic operationspersecon 1.21018FLOPS

1.121028bits

17ThisinformationdensityisfarlargerthantheBekensteinblackholeentropyboundontheinformation contentinmaterialsystems(Bekenstein,1981).

85

AppendixB:ComputerPerformance Development
Forecastsoffuturecomputerperformanceareneededtomakeroughestimatesofwhenbrain emulationwillbecomefeasibleintermsofcomputerstorageandprocessingcapacity.The followingestimatesare(withtwoexceptions)basedondatafromJohnC.McCallums datasetsofCPUpriceperformance(McCallum,2003),memoryperformance(McCallum, 2007b)anddiskperformance(McCallum,2007a).

Processing Power
Plottingtheperformanceofcomputersystemsgivesanestimateofavailabletotalcomputer powerofunitary,offtheshelfsystems(Figure23).

Figure23:Processingpower(MIPS)overtime. Aleastsquaresfithasbeenmadewithanexponentialcurve(straightinthissemilogplot). Thedecadetime,thetimeittakesforanorderofmagnitudeincreaseinMIPSis7.1years.

86

 Figure24:Processingpowerperdollarovertime. Amorerelevantmeasureistheamountofcomputingpoweradollarcanbuy.Usingthedata andadjustingforinflationto2007dollars(thishasbeendoneinallestimatesbelow)wegeta decadetimeofjust5.6years(Figure24).Thebootstrap95%confidenceintervalis5.35.9. Presentperformanceisabout1MIPSperdollar.Thetrendappearsreliable,butpossibly accelerating. Istherateofdevelopmentchanging?Fittingcurvesto20yearintervalsoftheMIPS/$ estimatesproducesarangeofestimatesofthedevelopmentexponent,clusteringtogether intotwoorthreegroups.Thedecadetimesareroughly4.4,8.7and3.5yearsrespectively.The slowestdevelopmentspeedoccurredduringthe70sand80s,perhapsduetothe proliferationofcheaphomecomputersbuiltwithlesspowerfulprocessorsforeconomical reasons.Sincethenithaspickedupspeedagain. Fittingtocomputersinthesamepriceclasses(pricesequalwithinanorderofmagnitude) producedecadetimesrangingfrom4.3to7.1yearswiththecheapestcomputersbecoming moreeconomicalfastest.Clearly,thecommoditymarketisdrivingdevelopmentfast(but parallelisationhasthepotentialtospeeddevelopmentevenfurther,asdescribedbelow). Giventheseresultsitlookslikelythatthetrendwillcontinue(forsomeunknownduration) withadecadetimeofbetween3.5and8.7years,with5.6asamiddleestimate.

FLOPSvs.MIPS
ThesemeasurementshaveusedMIPSasameasureofcomputationalpower,butitisnota reliableindicatorofactualperformanceinnumericheavyapplications(e.g.see(Nordhaus,

87

2001)foracriticism).Forthat,theMFLOPS(MillionFLoatingPointinstructionsperSecond) ismoresuitable.Unfortunately,itisnotaswidelytestedastheMIPS.

Figure25:Linpackperformanceperdollarovertime. ThefloatingpointheavymeasuremaximumLINPACKperformanceperdollar,producesa graphsimilar(ifnoisier)toMIPS(Figure25).Thedecadetimeis7.7years,similartotheMIPS estimatebutwithgreateruncertainty(thebootstrapconfidenceintervalis6.59.2years).

88

 Figure26:MIPSvs.MFLOPS. ThelogarithmsofestimatedMIPSandmaxFLOPScorrelateto0.79,suggestingafairlyrobust linkbetweenthem.FittingarelationshipsuggeststhatFLOPSscalesasMIPStothepowerof 0.89,i.e.slightlyslowerthanunity. Isthisexponentanartefactofearlycomputerswherenumericaloperationsweredonebyone ormoreinstructionsandnotrepresentativeofcurrentcomputerswherecoprocessorsand otheroptimizationsallowmultiplenumericaloperationsinthesameclockcycle?

89

 Figure27:FittedconversionexponentfromMIPStoFLOPSfordifferent5yearintervals. Fittingtosmallertimeintervals(length5years)from1970to2000producedarangeof exponents(Figure27).Totesttheirreliabilitytheirvalueswerecalculatedforeachinterval severaltimes,eachtimewithadifferentsampleremoved,producingadistribution(shownas greydots)ofexponentsthatallowedacalculationofmeanandstandarddeviation(shownin blue).Themorereliableexponentsareallcloseto1,withastatisticallysignificantincrease since1986until1995.However,anylongtermtrendtowardshigherexponentsseemshardto support. Istheeffectdifferentinsmallercomputersthansupercomputers?PlottingMIPSvsFLOPS, colouringthesamplesbytheirprice,andthenfittingexponentialsproducesthefollowing graph(Figure28).

90

 Figure28:RelationbetweenMIPSandFLOPSfordifferentpricerangecomputers. Thisgraphseemstosuggestthatthemoreexpensive(andpresumablymorepowerful computers)producefewerFLOPSperMIPSthanthecheaperones,whichhaveaslightly superlinearrelation.However,thismaybebiasedbythesmallsampleofcomputersinthe datasetforwhichinformationaboutbothFLOPS,MIPS,andpriceisavailable. Altogether,assumingthatFLOPSgrowasMIPStothepowerof0.8isprobablyasafe assumption,buttheexponentcouldbecome>1iftherewereafocusonhighperformance calculationinprocessors(orothercomputerpartssuchasgraphicscards).Thereisalsoa sizeablespreadofproportionalityconstants,atleasttwoordersofmagnitude. Puttingthepreviousconsiderationstogether,andassuminga1MIPS/$2007weshould expectFMFLOPSofperformanceachievablewithapriceofPdollarsin(T/0.8)log10(F/2.3P0.8) years,assumingadecadetimeofT.

Otherestimates
WilliamD.Nordhaushasdonehistoricalestimateofcomputerperformancesincethe1800s, includingpreelectroniccomputers(Nordhaus,2001)(partiallybasedondatafrom(Morevec, 1988)).Heusesmillionsofstandardizedoperationsperseconds(MSOPS)ashismeasure,1 MSOPSroughlycorrespondingto20million32bitadditions18.MSOPScorrelatesstrongly withadditionspersecondandcyclespersecond.

18ThecalculationusedisSOPS=0.05((6+log2(memory)+wordlength)/((7xaddtime+multiplication
time)/8))

91

 Figure29:Nordhausdataofcomputerperformanceperdollarovertime. Thedatashowasharpbreakpointaround1940whenelectroniccomputersbegantodevelop. Beforethistime,performanceperpricewasnearlystationary;afterwardsitgrew exponentially.Thedecadetimeis5.6years,agreeingwithpreviousestimates.Healso observesthatsupercomputerperformanceperpriceseemstolagafterperformanceof commoditycomputers. Thedatain(Kurzweil,1999)tellasimilarstory,withdecadetimeof6.7years.

92

 Figure30:PeakperformanceofTop500datasetofsupercomputers. TheTop500supercomputinglist19tabulatesthe500mostpowerfulsupercomputersinthe world.Over19932003thesemachineshadadoublingtimefortheLinpacktestof13.5 months(Feitelson,2005),givingadecadetimeofjust3.82years.Usingthefulldataset1993 2008givesaslightlyshorterdecadetime,3.7years(thisoccursbothwhenusingthegeneral Linpackperformanceorthepeakperformance).Alikelyreasonforthefastergrowthrate thansingleprocessorsystemsisthatthenumberofprocessorspercomputerisalso increasingexponentially.Whilethedatadoesnotincludepriceinformation,theatpresent (June2008)topsystemachieves1petaflopsperformancefor$100million,achieving10 megaflops/$.Thisappearstobreakprevioustrendpredictions,suggestingthatextremehigh performancesystemsmayactuallybeimprovingmuchfasterthanthesmallersystems. However,assmallersystemsbegintouseparallelcomputationthesamerapidexpansion mayoccurtheretoo.

19

www.top500.org

93

 Figure31:Projectedsupercomputerperformancefromwww.top500.org.

EstimatingavailablecomputingpowerforWBE
TherearethreewaysofestimatingthecomputingpoweravailablefordoingWBE.Thefirstis toextrapolatetheperformancetopriceratioofcommoditycomputers,thesecondistouse topsupercomputerperformancedirectly,thethirdistoestimatetheperformancetoprice ratioforsupercomputers.Allthreehavemeritsandproblems. Commoditycomputersrepresentalowerbound:theyarenotveryparallelsofarandthe economicforcesdrivingthemarenotallaimingformaximumperformanceoreven performanceperdollar.Wehavedatastretchingbacklongerforthemthanforhigh performancecomputers.Usingtheabovedataproducestheformula [MFLOPS/$]=2.33100.88(t2007)/D wheretistheyearandDisthedecadetimeofMIPS,5.6years(between3.5and8.7years with95%confidence). Absolutesupercomputerperformancefollowedaveryregularexponentialgrowthbetween 1993and2006,andhasenabledreliableshorttermpredictions(StrohmaierandMeuer,2004). Thegrowthisfasterthanforcommoditycomputersduetorapidlyincreasingparallelisation. However,thereisreasontobelievethisparallelisationgrowthmayconflictwiththe requirementsofenergy,coolingandspaceinthenearfuture(seebelow).SinceaWBEproject islikelytobeamajorresearchundertaking,havingatop500supercomputeravailableisvery likely(severalofthecurrenttopsupercomputershavebeenusedforcomputational neuroscience);thefullcostofthecomputermaybesharedbetweentheWBEprojectand otherprojects.

94

Assumingacomputeronparwiththe#500computer,theavailablepowergrowsas [FLOPS]=8.99610610(t2008)/D WhereDis3.92.Uptotwoordersofmagnitudemorecomputerpowerisavailablebyusing morepowerfulcomputers.Thecostofthecomputeritselfwouldbeontheorderof$10 million. Estimatingsupercomputerperformance/priceratiosishardersincetheirpricesarenot generallylisted.UsingtheRoadrunnercomputertocalibrate(~$100million)wouldgivea currentconversionfactorof1.026petaflops/$100million,increasingwithadecadetimeof 3.7years.Therangeofratiosateachtimealsolikelyspansatleastanorderofmagnitude:the personalsupercomputerMicrowulfsprice/performanceratio(2007)was$48/Gflop20 whiletheSunsSparcEnterpriceM9000mainframe(basepriceof$511,385)produced1.03 TFLOPSofmeasuredperformance,makingitsPPR>$496/Gflop.Similarlycustombuilt hardwarecanlikelyimprovetheratiobyatleast12ordersofmagnitude(Wehner,Olikeret al.,2008).Usingthiswegetanestimateof [MFLOPS/$]=10.2610(t2008)/D WhereD=3.7yearsandtheuncertaintyisatleastoneorderofmagnitudeatanypointintime.

Memory

Figure32:RAMmemoryperdollarovertime.
20

http://www.calvin.edu/~adams/research/microwulf/

95

TheamountofRAMmemoryperdollargrowsneatlyexponentially,withadecadetimeof4.8 years.

Figure33:RAMaccessspeedovertime. TheaccesstimesofRAMdeclineinthedata,althoughnumbersareonlyavailablefromthe late80sandonwards.Thecurvefitproducesadecadetimeof18.9years.

96

Disc drives

Figure34:Discstorageperdollarovertime. AsimilartrendtotheexponentialpriceperformancegrowthMooreslawincomputersis Kryderslawindiscdrives(Walter,2005).Theamountofstorageperdollarbegantogrow fasteraround1980(acurvewasfittedfor19802005),withadecadetimeofonly3.5years (Figure34).Relativelyfewdiskaccesstimesareavailable,butfittinganoverallexponential trendgivesadecadetimeof29.3years(Figure35). GiventhegrowingratiobetweenstoragecapacityandspeedofbothRAManddrives, updatingallofthestoreddatawilltakelongerandlonger:storagecapacityisoutrunningthe accessspeed.Thismayforcefinergranularityonbrainemulationssothattherearemore processorswiththeirownmemoryratherthanfewverypowerfulprocessorsbeinglimited bytheamountofdatatheycanrequest.

97

 Figure35:Discaccesstimeovertime.

Future
Theissueisnotwhetherthereexistphysicalsystemsabletoperformthecomputationsdone bybrains,sincesuchsystemsalreadyexist:thebrainsthemselves.Rather,theissueiswhether brainemulatinghardwarecanbeconstructedbyhumaningenuityintheforeseeablefutureat asufficientlylowcosttomakeWBEfeasible. WhileMooreslawandotherexponentialprogresslawsincomputinghaveheldrelatively steadyoverdecades,theywillpresumablybreakatsomepoint.Attheverylimit,the quantizednatureoftheuniverseislikelytolimitthem.Beforethatpoint,limitationson molecularbuildingmaterials,lightspeed,andenergydissipationwillproveincreasingly problematic. Semiconductorsstillhavealongwaytogo(Meindl,Chenetal.,2001).Theinternational semiconductorroadmap(ITRS07,2007)appearsrelativelyconfidentwithinthe2016time horizon,andliststheresearchchallengesneededtocontinuethetrendtowards2022. However,thisisstillrelativelyclosecomparedtothetimescaleslikelyforachievingWBE. Worse,improvedindividualchipperformancemaynotnecessarilycarryoverdirectlyinto improvedsupercomputerperformance. (DeBenedictis,2004),astudyofthelimitsofcurrenttechnologytrendsfoundthatthemain probleminthenottoodistantfuture(beyond1020years)isgoingtobeheatdissipation,at leastformassivelyparallelcomputers21.Theexpectedperformanceinthisendgame
Theexceptionmaybearchitecturesdominatedbymemoryratherthanprocessing,whichcanremaindenseand cooltoagreaterextent.WBEapplications,however,arelikelytoneedtoupdatemostofthestatevariableseach
21

98

beyond2016wouldbeintheexaflopstozettaflopsrange(10181021).Whilecoolingcanbe mademoreeffectivebydispersingthecomputationunitsthisintroducescommunications delays.Reversiblelogicmayreduceheatdissipationbutthebestcurrentimplementations stilldissipatetwoordersofmagnitudemoreenergythanthetheoreticalminimum.Thestudy arguesthatacosteffectivenessof58TFLOPS/$andyear(givenrunningcostsandprorated purchaseprice)isachievable.Moreefficientcooling,orlessdissipativecomponentsare recognizedasimportantforzettaflopscomputing(DeBenedictis,Koggeetal.,2006). Exactlywhatcouldsupersedesemiconductorsremainsconjecturalbutunderintenseresearch (Welser,Bourianoffetal.,2008;Compao,Molenkampetal.,1999).Afewofthepossibilities thatarebeingpursuedorconsideredare: Ybranchswitchesarenanoelectroniccomponentsthatuseelectronsmovinginthe ballisticregimethatarenotstoppedbybarriers(ForsbergandHieke,2002).Such devicescouldbeusedtobuildreversiblelogicgates,reducingenergydissipation (Forsberg,2004). RapidSingleFluxQuantum(RSFQ)Logicusessuperconductingquantumeffectsto switch,usingthefluxquantumasabit(LikharevandSemenov,1991).Itcanswitch extremelyfast(hundredsofgigaherz),hasalowpowerconsumptionandexisting chiptechnologycanbeadaptedtomakeRSFQcircuitry(Zinoviev,1997).Simple microprocessorshavebeendemonstrated(Yoshikawa,Matsuzakietal.,2002). However,RSFQrequireslowtemperaturesforsuperconductivity. Opticalcomputing.Usingphotonsratherthanelectronswouldincreasetransmission speedsifsuitablenonlinearcomponentscanbefound.Thefieldhasbeenpursuedon andoffsincethe1960swithavarietyoftechnologies(SawchukandStrand,1984; Higgins,1995).Allopticalgatesareapproachingtheperformanceoftheirelectronic counterpartsbutstillneedfurtherdevelopment(Zhang,Wangetal.,2005).Akey problemisthatopticalcommunicationtendstorequiremorepowerovershort distancesthanelectronics,duetotheneedtoavoidshotnoise. Quantumdotcellularautomata(QCA).Electronconfigurationsinpatternsof quantumdotsactasacellularautomaton(TougawandLent,1994;Li,Wuetal.,2003; Robledo,Elzermanetal.,2008;Amlani,Orlovetal.,1999).SinceQCAdonottransmit currentsandperformreversiblecomputation(TimlerandLent,2003)heatdissipation islow,andmayprovideanidealenvironmentforzettaflopscomputing (DeBenedictis,2005).Whilecurrentdotsaresemiconductorssmallerandmore temperatureresistantsystemscouldbeconstructedoutofmolecularquantumdots (Lieberman,Chellammaetal.,2002).Arelatedtechnologyisspintronics,where magneticpolarisationisusedforcomputing(Allwood,Xiongetal.,2005;Imre,Csaba etal.,2006).Whilealsodissipatinglittleheat,speedlimitationsmaymakeit unsuitableforrapidprocessing. Helicallogic,reversiblelogicusingelectronsconstrainedtohelicalpathwaysshifted byanexternalfield(MerkleandDrexler,1996).Whilereversibleitrequireslow temperaturestofunctionandhasspeedlimitations. Molecularelectronics,wheretraditionalelectronicssuchasdiodesortransistorsare reproducedonthemolecularlevel(TsengandEllenbogen,2001).Logiccircuits (Bachtold,Hadleyetal.,2001)andnonvolatilememory(Rueckes,Kimetal.,2000) basedoncarbonnanotubetransistorshavebeendemonstrated. Intramolecularnanoelectronics,whereswitchingoccursonthemoleculelevel.This wouldinprinciplebeabletoreachtheultimatebitdensitiesof1012bits/cm2,
timestep,whichmeanstheyareunlikelytobestoragedominated.

99

switchingcyclesof10psand1eVenergyperbitcycle(Compao,Molenkampetal., 1999).Itwouldeitherrequirehighprecisionselfassemblyorassemblythrough molecularnanotechnology. Rodlogic,computingusingnanoscalerigidrodsinteractingmechanically,was originallysuggestedmostlyasaproofofconceptthatcomputingcouldbedoneon thenanoscale(Drexler,1992).Variantsofrodlogiccanimplementreversible computation(Merkle,1993),makingitsuitableforveryhighdensityprocessing.It wouldalsorequiremolecularnanotechnologyforconstruction.

Pessimisticclaimsareoftenmadetotheeffectthatlimitationsofcurrenttechnologyare foreverunsurpassable,orthattheoreticallypossibletechnologicalsystemssuchastheabove willbetoocostlyandpracticallydifficulttoeverbecomefeasible.Still,giventhatcomputer technologyhasdevelopedinarelativelystablemannerdespiteseveralchangesofbasic principles(e.g.fromflipflopsviacorememorytoseveralsemiconductorgenerations,from vacuumtubestotransistorstointegratedcircuitsetc)thereisnostrongreasontoassumethey willbreakbecausecurrenttechnologywilleventuallybereplacedbyothertechnologies. Largevestedinterestsincontinuingthegrowtharewillingtospendconsiderableresources onclosingtechnologygaps.Amorelikelyendofgrowthscenarioisthatthefeedback producingexponentialgrowthisweakenedbychangesinthemarketplacesuchaslowered demand,loweredexpectations,risingproductionfacilitycosts,longdevelopmenttimesor perhapsmoreefficientsoftware22. Iffurtherminiaturizationproveshardtoachieveforonereasonoranother,increased performancecanstillbeachievedbyusingparallelism.Ifmanyunitsaresold,priceswillgo downeveniftheprocessorsarenotmorepowerful,andaslowerexponentialgrowthofMIPS perdollarwillcontinue.Itshouldbenotedthat,sincethecommoditymarkethasdriven developmentofcomputersverystrongly,andatpresentonlyabout15%ofhumanityownsa computer,therearestilllargeuntappedmarketsthatwillovertimebecomerichenoughto affordcomputersandhencedrivefurthertechnologicaldevelopment.Asdemonstratedby variousingenioususesofGPUchipprogrammingforhighperformancescientific applications,itisentirelypossibletoprofitfromacompletelyunrelatedconsumerdemandif thesoftwareisadaptedtohighperformancehardwareevenwhenthishardwarewas developedforacompletelydifferentpurpose(inthiscasecomputergames).Themain concernforWBEisthatitisentirelypossibleforconsumercomputingtogoinadirectionless suitableforemulationsoftware(e.g.tradingprocessingpowerforlowerpowerusage).

Asanexample,theexponentialgrowthofcomputingpowerhidessoftwarebloat,wherenewsoftwaretendsto requireincreasinglymoreresourcestodothesametasksasoldversions.Thisispartiallyduetotradingprogram efficiencyforprogrammingsimplicity.Theexpectationofexponentiallygrowinghardwarecapabilitiesmakesit worthwhiletogetnewhardwareoften,whichtogetherwiththebloatfeedsthecycle.Ifhardwaredevelopmentwere toslowincentivestoavoidbloatwouldemerge,andevenfornonbloatingsoftwareapplicationsthebenefitsof buyingnewhardwareoftenwouldlessen.Thiswouldreducethestrengthofthefeedbackloop(andpossiblyremove it),severelyreducingtheincreaseincomputingcapabilities.

22

100

AppendixC:Largescaleneuralnetwork simulations
Thesesimulationsrepresentasmallsamplingoftheliterature.Thereisabiastowards researchaimedathighperformancecomputationalneuroscienceanddevelopingnew methods. Table11:Largescaleneuralsimulations
Simulation Typeof simulation Neurons Synapses Hardware and Software Slowdown (time requiredfor 1biological second) 2 Timestep Notes

(Plesser, Eppler et al., 2007)

Integrate-andfire

12,500

1.56107

(Djurfeldt, Lundqvist et al., 2006)

6-compartment neurons

2.2107

1.11010

Sun X4100 cluster, 2.4Ghz AMD Opteron, 8GB ram, MPI, NEST IBM Warson Research Blue Gene, SPLIT

0.1 ms

5,942

Supralinear scaling until 80 virtual processes for 105 neuron/109 synapses. Computing requirements: 11.5 TFLOPS. Spatial delays corresponding 16 cm2 cortex surface.

(Izhikevich, 2005)

(Howell, Dyhrfjeld-Johnsen et al., 2000) (Howell, Dyhrfjeld-Johnsen et al., 2000)

Izhikevich neurons, Random synaptic connectivity generated on the fly. Compartment (Purkinje 4,500 comp./granule 1 comp.)

1011

1015

Beowulf cluster, 27 3GHz processors

4.2106

16 Purkinje, 244,000 granule cells 60,000 granule cells, 300 mossy fibers, 300 Golgi cells, 300 stellate cells, 1 Purkinje cell 900 8106

128processor Cray 3TE, PGENESIS Single workstation 1GB memory, GENESIS

4,500

Computing requirements: 76.8 GFLOPS 20 s

79,200

(Kozlov, Lansner et al., 2007) (Frye, Ananthanarayanan et al., 2007)

(Brette, Rudolph et al., 2007; Traub, Contreras et al., 2005) (Traub, Contreras et al., 2005)

5 compartment model Low complexity spiking like Izhikevich neurons, STDP synapses 14 cell types, conductance and compartment model 14 cell types, conductance and compartment model, 11 active conductances

10% per hemisegment 6300*8106= 5.041010

SPLIT 4096 processor BlueGene/L 256 Mb per CPU Cray XT3 2.4 Ghz, 800 CPUs 14 cpu Linux cluster, an IBM e1350, dualprocessor Intel P4 Xeon 10

100 s 1 ms

3,560

3,560

3,500 gap junctions, 1,122,520 synapses 3,500 gap junctions, 1,122,520 synapses

200?

Event driven 2 s

5954-8516 equations/CPU Gives # compartments

101

(Goodman, Courtenay Wilson et al., 2001)

(Frye, 2004)

3 layer cortical column, 25% GABAergic, conductance models, synapse reversal pot., conductance, abs. str., mean prob. release, time const. recover depress facilitation, Four cell types. Km, Ka, Kahp channels

5,150 per node, max 20*5,150=103, 000

In columns 192,585 per node, max 20 = 3.85106, intercolumn 1.4106

2.4-GHz, 512 Kbytes of L2 cache; or, e1350 Blade 1, at 2.8 GHz NCS, 30 dualPentium III 1GHz processor nodes, with 4 GB of RAM per node

165,000

1,000/cells column, 2,501 columns = 2.5106 4,083

250K per column, total 6.763108

BlueGene, 1024 CPUs

3,000?

2.3381010 spikes per processor. Memory requirement 110 Mb per processor

(Aberdeen, Baxter et al., 2000)

(Kondo, Koshiba et al., 1996) (Ccortex, 2003)

ANN feedforward network trained with iterative gradient descent Hopfield network Layered distribution of neural nets, detailed synaptic interconnections, spiking dynamics Point neurons, spikes, AHP, A & M potassium channels, synaptic adaptation, Spike shape and postsynaptic conductance specified by templates. Integrate and fire, event driven pulse connections with STDP plasticity. 1-8 dendritic segments. Palm binary network

1.73106

1,536 21010

2.4106 21013

(Harris, Baurick et al., 2002)

35,000 cells per node

6.1106 synapses per node

Cluster 196 pentium III processors, 550 MHz, 384Mb Special purpose chips 500 nodes, 1,000 processors, 1 Tb RAM. Theoretical peak of 4,800 Gflops. Cluster, 128 Xeon 2.2 GHz processors with 256GB, 1Tb disk

Claimed performance, not documented.

(Mehrtash, Jung et al., 2003)

256,000

106

(Shaojuan and Hammarstrom, 2002)

256,000

Order of 3.21010

Special purpose chip, connected UltraSparc processor 500 Mhz, 640 Mb 512 processors, 192 GB, 250 Mhz mips 12K processorr

1012.21 timessteps/ ms

(Johansson and Lansner, 2007)

BCPNN spiking, minicolumns hypercolumns

1.6106

2.01011

256 node, 512 processors, Dell Xeon cluster,

Execution time 4 min 7 sec, -13% MPI calls for retrieving 180K training vectors, ~O(10) iterations 9% real-time = 11.1

Weight updates 47 ms, update activities 59 ms

102

(Markram, 2006)

Morphologically complex compartment models

10,000

108

3.4GHz 8 Gb memory, peak performance 13.6 Gflop BlueGene/L, 4096 nodes, 8192 cpus, peak performances 22.4 TFLOPS IBM3090 computer Cray Y-MP 8/864 325 min/1.5 s bio time = 13,000 0.006 s (retrieval of 550,000 patterns; assuming human retrieval speed = 1 s) 0.05 ms -

120 kW usage. Computational demand 256*13.6 Gflops Not documented?

(Traub and Wong, 1982) (Traub, Miles et al., 1992) (Moll and Miikkulainen, 1997)

100 19 compartment cells, ionic currents Binary units with binary weights 1,200 79,500 70,000 7.82108

Plottingthesizeofthesimulationsovertimedoessuggestarapidlyincreasingscale, althoughthereisnotenoughdatatoestimateareliabletrend.Theredoesnotseemtoexista clearseparationbetweenbiologicallydetailedmodelsandhighlysimplifiedmodels;someof thelargestsimulationshavebeendetailedcompartmentmodels.

Figure36:Numberofneuronsindifferentlargecomputationalneurosciencemodels.

103

 Figure37:Numberofsynapsesindifferentcomputationalneurosciencemodels.

104

AppendixD:Historyandpreviouswork
TheearliestoriginsofthemindemulationideacanperhapsbetracedbacktoJ.D.BernalsThe World,TheFlesh,TheDevil(1929),wherehewrote: Menwillnotbecontenttomanufacturelife:theywillwanttoimproveonit.Forone materialoutofwhichnaturehasbeenforcedtomakelife,manwillhaveathousand; livingandorganizedmaterialwillbeasmuchatthecallofthemechanizedor compoundmanasmetalsaretoday,andgraduallythislivingmaterialwillcometo substitutemoreandmoreforsuchinferiorfunctionsofthebrainasmemory,reflex actions,etc.,inthecompoundmanhimself;forbodiesatthistimewouldbeleftfar behind.Thebrainitselfwouldbecomemoreandmoreseparatedintodifferent groupsofcellsorindividualcellswithcomplicatedconnections,andprobably occupyingconsiderablespace.Thiswouldmeanlossofmotilitywhichwouldnotbe adisadvantageowingtotheextensionofthesensefaculties.Everypartwouldnotbe accessibleforreplacingorrepairingandthiswouldinitselfensureapracticaleternity ofexistence,foreventhereplacementofapreviouslyorganicbraincellbyasynthetic apparatuswouldnotdestroythecontinuityofconsciousness. Finally,consciousnessitselfmayendorvanishinahumanitythathasbecome completelyetherealized,losingthecloseknitorganism,becomingmassesofatomsin spacecommunicatingbyradiation,andultimatelyperhapsresolvingitselfentirely intolight. Bernalsvisioncorrespondstoagradualreplacementofbiologywithartificialparts, graduallymakingitunnecessarytokeepthebraininonelocation. InthesciencefictionnovelCityandtheStars(1956)ArthurC.Clarkedescribedafarfuture citywherebodiesaremanufacturedbythecentralcomputer,mindsstoredinitsdatabanks downloadedintothem,andwhenaninhabitantdiestheirmindisstoredyetagaininthe computer,allowingcountlessreincarnations.Otherearlysciencefictiontreatmentswere RogerZelanzkysLordofLight(1968),BertilMrtenssonsDettarverkligheten(1968)and RudyRuckersSoftware(1979).Sincethen,mindemulation(uploading)hasbecomeastaple ofmuchsciencefiction23.Ofparticularnoteintermsoftechnologicalandphilosophical detailsarethenovelsandshortstoriesbyGregEgan(PermutationCity,Diaspora,Learningtobe Me,TransitionDreamsetc). Brain(andmind)emulationhasalsobeenwidelydiscussedinphilosophyofmind,although moreasGedankenexperimentethanpossibleactualpractice(e.g.(Parfit,1984;Chalmers,1995; Searle,1980)). Thefirstattemptatacarefulanalysisofbrainemulationwasatechnicalreport(Merkle, 1989b)predictingthatacompleteanalysisofthecellularconnectivityofastructureaslarge asthehumanbrainisonlyafewdecadesaway.Thereportreviewedautomatedanalysis andreconstructionmethods,goingintogreatdetailontherequirementsneededforparallel processingofbrainsamplesusingelectronmicroscopesandimageanalysissoftware.Italso clearlylistedassumptionsandrequirements,agoodexampleoffalsifiabledesign.
23

E.g.http://en.wikipedia.org/wiki/Mind_transfer_in_fiction

105

Thefirstpopularizationofatechnicaldescriptionofapossiblemindemulationscenariowas foundinHansMoravecsMindChildren(1990),wheretheauthordescribesthegradual neuronbyneuronreplacementofa(conscious)brainwithsoftware.Otherformsof emulationarealsodiscussed. (Hanson,1994)wasthefirstlookattheeconomicalimpactofcopyableminds,showingthat socialrolefitbrainemulationwouldlikelycausedramaticeconomicanddemographic changes. Onesketchofapersonemulationscenario(Leitl,1995)startsoutbythecryonicsuspensionof thebrain,whichisthendividedintocubicblocks<1mm.Theblockscanindividuallybe thawedforimmunostainingorothercontrastenhancement.Forscanning,variousmethods areproposed:Xrayfresnel/holographicdiffraction,Xrayorneutronbeamtomography(all riskingradiationdamage,mightrequirestrongstaining),transmissionEM(requiresverythin samples),UVabrasionofimmunostainedtissuewithmassspectrometry,orabrasiveatomic forcemicroscopescan.Whiledetailedintermsofthecryosuspensionmethods,thesketch becomeslessdetailedintermsofactualscanningmethodandimplementingtheemulation.

106

AppendixE:Nondestructiveandgradual replacement

Non-Destructive Scanning
Nondestructivescanningrequiresminimallyinvasivemethods.Thescanningneedsto acquiretherelevantinformationatthenecessary3Dresolution.Therearehoweverseveral seriouslimitations: Themovementofbiologicaltissue,requiringeitherimagingfasterthanitcanmove oraccuratetracking.Incats,thearterialpulseproduces110266mmovements lasting330400msandbreathinglarger(300950m)movements(BrittandRossi, 1982)24.Thestabilitytimeisasshortas520ms. Imaginghastooccuroveradistanceof>150mm(thewidthofanintactbrain)orbe invasive. Theimagingmustnotdepositenoughenergy(orusedyes,tracersorcontrast enhancers)tohurttheorganism. Themethodmustnotsignificantlyalterthementalorneuralstateofthesubjectbeing scannedinordertoavoidapossiblysignificantobservereffectanomaliesandfalse readingthatcouldproduceaflawedemulationmodel. OfthepossiblenoninvasivecandidatesonlyMRIappearsabletofulfilthelimitationsevenin principle.Opticimaging,evenusingfirstarrivinglightmethods,wouldnotworkacrosssuch alargedistance.Xraytomographyoftheintensityneededtoimagetissuewoulddeposit harmfulenergy(seediscussiononXraymicroscopy). TheresolutionofMRIdependsonthenumberofphasestepsused,gradientstrength, acquisitiontimeanddesiredsignaltonoiseratio.Torecordmicronscalefeaturesina movingbrain,veryshortacquisitiontimesareneeded,orawayofremovingthemovement artefacts.Eachdoublingofspatialresolutiondividesthesignaltonoiseratioby8,requiring longeracquisitiontimes,strongerfieldsormoresensitivedetectors.Finally,therearealso problemswithtissuewaterselfdiffusion,makingresolutionssmallerthan7.7mimpossible toachieve(GloverandMansfield,2002)25.Giventhatbrainemulationonthesynapticlevel requireshigherresolution,thisprobablyrulesoutMRIasanondestructivescanningmethod. However,ifthebrainisfrozen,waterdiffusionandmovementdonotoccurandverylong acquisitiontimescanbeused.OneproblemwithMRIoffrozenbraintissueisthatlimited protonmobilityreducesthesignal;thiscanbeamelioratedbykeepingthebrainat1oC (Longson,Hutchinsonetal.,1995;Doyle,Longsonetal.,1996).MRImightthereforebea possiblescanningmethodforfrozenorfixedbrains.Sinceitisnotdestructivetothetissueit mayalsoactasanadjuncttoother,destructive,scanningmethods.
24

Theauthorsofthepapersuggestsasasolutionusingacardiacbypasssystemtocreateanonpulsative flowofoxygenatedblood.

25Higherresolutionshavebeenreportedbasedonusingmicrocoilreceiversandverystrongmagneticfield gradients.(Ciobanu,Seeberetal.,2002)reportsaresolutionof3.7x3.3x3.3m,buttheimagingtimewas30hand theauthorswerepessimisticabouttheabilitytodetectdifferentchemicals.

107

Apossiblewayaroundtheproblemsofnoninvasivescanningwouldbetouseendoscopic methodstobringmeasurementdevicesintothebloodvesselsofthebrainandimageitfrom inside.EndoscopicMRIhasbeendemonstratedinthegastrointestinalsystemwhereaRFcoil wasaffixedtothetipofanendoscope(Inui,Nakazawaetal.,1998).Thisislimitedbythe rangeofthedevices,theirsizeandtherisksofpenetratingvessels.However,itappears unlikelythatallpartsofthebrainarecloseenoughtoalargebloodvesseltoallowaccurate scanning,evenleavingouttheresolutionandmovementissues. Correlationmappingusingnanoprobes(Strout,2006)hasalsobeensuggested.Alarge number(1012)ofnanomachinessetupresidenceinoronneurons,recordingtheiractivityand functionalcorrelations.Amoredetailedanalysisofhownanomachinescouldsenseneural activitycanbefoundin(FreitasJr,1999,section4.8.6)togetherwithasketchofaninvivo fibrenetworkwith1018bit/scapacity(FreitasJr,1999,section7.3.1).Thiswouldenable essentiallyrealtimemonitoringofallneuronsandtheirchemicalenvironment(Kurzweil, 2005,pp.548549).Whetherjusttheactivitywouldbeenoughtomaptheconnectivityand memorystatesofthebrainisunclear.However,giventheassumedtechnologyboth intracellularprobingandphysicalconnectivitymappingbyhistonatingnanorobotsappears possible.Whetherthiswouldinterferewithbrainfunctionishardtoestimate(butsee(Freitas Jr.,2003)forananalysis).Thefibrenetworkwouldhaveavolumeof30cm3,whichis41%of thenormalbloodvolumeinthebrain(basedonestimatesofbloodvolumein(Leenders, Peranietal.,1990)),possiblyimpedingbloodflowifextendedinsidevessels.Althoughthe capabilitiesofnanomachinescanbeconstrainedbyknownphysicsitisnotpossibletodayto inferenoughaboutmachine/cell/tissueinteractionstoestimatewhethernondestructive scanningisfeasible. Overall,theprospectsfornondestructivescanningdonotlookgoodatpresent.Atthevery leastitwouldneedtoinvolveveryinvasiveendoscopicscanningortheuseofadvanced nanomedicine.

Gradual replacement
Scanningmightalsooccurintheformofgradualreplacement,aspieceafterpieceofthebrain isreplacedbyanartificialneuralsysteminterfacingwiththebrainandmaintainingthesame functionalinteractionsasthelostpieces.Eventuallyonlytheartificialsystemremains,and theinformationstoredcanbemovedifdesired(Morevec,1988).Whilegradualreplacement mightassuagefearsoflossofconsciousnessandidentity26itappearstechnicallyvery complexasthescanningsystemnotonlyhastoscanaliving,changingorganismbutalso interfaceseamlesslywithit(atleastonthesubmicronscale)whileworking.Thetechnology neededtoachieveitcoulddefinitelybeusedforscanningbydisassembly.Gradual replacementisthereforenotlikelyasafirstformofbrainemulationscanning(thoughin practiceitmayeventuallybecomethepreferredmethodifnondestructivescanningisnot possible). Itissometimessuggestedthatextendingthebrainthroughinterfaceswithexternalsoftware mightachieveaformoftransferwheremoreandmoreoftheentirepersonisstoredoutside thebrain,possiblyreachingthepointwherethebrainisnolongeressentialforthecomposite person.However,thiswouldnotbebrainemulationpersebutratheratransitiontoa
26Butnotnecessarily.Searlehasarguedthatreplacementwouldgraduallyremoveconsciousexperience(Searle, 1980).Parfitsphysicalspectrumthoughtexperimentinvolvesinterpolatingbetweentwodifferentpeoplethat clearlyhavedifferentidentities,andthereplacementprocesscouldhaveasimilarproperty(Parfit,1984).

108

posthumanstate.Thetechnicalfeasibilityappearsilldefinedgivencurrentknowledge.It shouldbenotedthatevenrelativelypartialsuchinterfacesorliferecording(Gemmell,Bellet al.,2006)wouldproduceawealthofusefuldatafordevelopingbrainemulationsbyactingas ayardstickofnormalfunction27.

Again,itissometimessuggestedthatrecordingenoughofthesensoryexperiencesandactionswouldbeenoughto producebrainemulation.Thisisunlikelytoworksimplybecauseofthediscrepancyinthenumberofdegreesof freedombetweenthebrain(atleast1014synapticstrengthsdistributedina1022elementconnectivitymatrix)andthe numberofbitsrecordedacrossalifetime(lessthan21014bits(Gemmell,Belletal.,2006)).

27

109

AppendixF:Glossary
AFM ANN ATLUM Autoregulation Axon Blockface bouton CNS Confocalmicroscopy Connectome Dendrite Exaflop Extracellular FIBSEM FLOPS Fluorophore Atomicforcemicroscope(sometimescalledscanningforce microscope). ArtificialNeuralNetwork,amathematicalmodelbasedon biologicalneuralnetworks. AutomaticTapeCollectingLatheUltramicrotome Regulationofbloodflowtomaintainthe(cerebral)environment. Projectionfromanervcellthatconductssignalsawayfromthe neuronscellbody. Thesurfaceofanimagedsample,especiallywhencuttingtakes place. Thetypicalsynapticbumpthatgrowsbetweenaxonsand dendrites. CentralNervousSystem. Opticalimagingtechniquetoimage3Dsamplesbymakinguse ofaspatialpinhole. Thetotalsetofconnectionsbetweenregionsorneuronsina brain(Sporns,Tononietal.,2005). Branchedprojectionsofneuronsthatconductsignalsfromother neuronstothecellbody. 1018FLOPS. Theenvironmentoutsidethecells. FocusedIonBeamSEM. FloatingpointOperationsPerSecond.Ameasureofcomputing speed. Amoleculeorpartofmoleculethatcausesfluorescencewhen irradiatedbyUVlight.Usedforstainingmicroscope preparations. FieldProgrammableGateArray.Semiconductordevicethat containsprogrammablelogicandinterconnects,allowingthe systemdesignertosetupthechipfordifferentpurposes. RelatedtothetransmissionorreceptionofGABA,thechief inhibitoryneurotransmitter. Gigaflop,abillionFLOPS. Gliacellsarenonneuronalcellsthatprovidesupport,nutrition andmanyotherfunctionsinthenervoussystem. Agroupofcorticalminicolumnsorganisedintoamodulewitha fullsetofvaluesforagivensetofreceptivefieldparameters. IntheCNS,asmalllocallyprojectingneuron(unlikeneurons thatprojecttolongrangetargets). Anenzymethatphosphorylatesothermolecules. Amoleculethatbondstoanothermolecule,suchasareceptoror enzyme. Thecompletesetofsmallmoleculemetabolitesthatcanbefound inanorganism. MillionsofFloatingpointOperationsPerSecond.Ameasureof computingspeed. Acomponentofthecellskeleton,composedofsmallersubunits.

FPGA

GABAergic GFLOP Glia Hypercolumn Interneuron Kinase Ligand Metabolome MFLOPS Microtubule

110

Minicolumn

MIPS Motorneuron MRI Neocortex

Neurite Neurogenesis Neuromodulator Neuromorphic Neuron Neuropeptide Neurotransmitter Parallelisation PCR Petaflop Phosphorylation

PNS Potentiation SBFSEM SEM Sigmoidal

Skeletonization

Soma Spectromicrosopy SSET SSTEM Supervenience

Synapse Synapticspine

Averticalcolumnthroughthecerebralcortex;aphysiological minicolumnisacollectionofabout100interconnectedneurons,a functionalminicolumnconsistsofallneuronsthatsharethe samereceptivefield. MillionsofInstructionsPerSecond.Ameasureofcomputing speed. Aneuroninvolvedingeneratingmusclemovement. MagneticResonanceImaging. Thecerebralcortex,coveringthecerebralhemispheres. Neocortexdistinguishesitfromrelatedbutsomewhatmore primitivecortexthathavefewerthansixlayers. Aprojectionfromthecellbodyofaneuron,inparticularfroma developingneuronwhereitmaybecomeanaxonordendrite. Theprocessbywhichneuronsarecreatedfromprogenitorcells. Asubstancethataffectsthesignallingbehaviourofaneuron. Technologyaimingatmimickingneurobiologicalarchitectures. Anervecell. Aneurotransmitterthatconsistsofapeptide(aminoacidchain). Achemicalthatrelays,amplifiesormodulatessignalsfrom neuronstoatargetcell(suchasanotherneuron). Theuseofmultipleprocessorstoperformlargecomputations faster. PolymeraseChainReaction,atechniqueforamplifyingDNA fromasmallsample. 1015FLOPS. Additionofaphosphategrouptoaproteinmolecule(orother molecule),usuallydonebyakinase.Thiscanactivateor deactivatethemoleculeandplaysanimportantroleininternal cellsignalling. PeripheralNervousSystem. Theincreaseinsynapticresponsestrengthseenafterrepeated stimulation. SerialBlockFaceScanningElectronMicroscopy ScanningElectronMicroscopy. Sshaped,usuallydenotingamathematicalfunctionthatis monotonouslyincreasing,hastwohorizontalasymptotesand exactlyoneinflectionpoint. Imageprocessingmethodwhereashapeisreducedtothesetof pointsequidistantfromitsboundaries,representingits topologicalstructure. Thecellbodyofaneuron. Methodsofmakingspectrographicmeasuresofchemical compositioninmicroscopy. SerialSectionElectronTomography SerialSectionTransmissionElectronMicroscopy AsetofpropertiesAsupervenesonasetofpropertiesBifand onlyifanytwoobjectsxandythatsharealltheirBproperties mustalsosharealltheirAproperties.BeingBindiscernible impliesbeingAindiscernible. Ajunctionwhereone(ormore)neuronssignaltoeachother. Manysynapseshavetheirboutonsoffsetfromtheirparent

111

TEM TFLOPS Tortuosity V1 Voxel

dendritethroughathinnerfilament. TransmissionElectronMicroscopy. Teraflops,1012FLOPS. Ameasureofhowmanyturnsasurfaceorcurvemake. Theprimaryvisualcortex. Avolumeelement,representingavalueonaregulargridin3D space.

112

References
ABERDEEND,BAXTERJ,andEDWARDSR.92/mflops/s,ultralargescaleneuralnetwork trainingonapiiicluster.InSupercomputing,acm/ieee2000conference,2000,pp.4444. ACHARDPandDESCHUTTERE.Complexparameterlandscapeforacomplexneuronmodel. PlosComputationalBiology,2:794804,2006. ACKERMANNMandMATUSA.Activityinducedtargetingofprofilinandstabilizationof dendriticspinemorphology.NatureNeuroscience,6:11941200,2003. AJAYSMandBHALLAUS.Synapticplasticityinvitroandinsilico:Insightsintoan intracellularsignalingmaze.Physiology,21:289296,2006. ALKOFAHIKA,LASEKS,SZAROWSKIDH,etal.Rapidautomatedthreedimensionaltracingof neuronsfromconfocalimagestacks.IEEETransInfTechnolBiomed,6:17187,2002. ALLWOODDA,XIONGG,FAULKNERCC,etal.Magneticdomainwalllogic.Science,309:1688 1692,2005. AMLANII,ORLOVAO,TOTHG,etal.Digitallogicgateusingquantumdotcellularautomata. Science,284:289291,1999. ANDERSONFCandPANDYMG.Dynamicoptimizationofhumanwalking.JBiomechEng,123: 38190,2001. ARNOLDAS,SALINASS,ASAKAWADJ,andDELPSL.Accuracyofmusclemomentarms estimatedfrommribasedmusculoskeletalmodelsofthelowerextremity.Comput AidedSurg,5:10819,2000. ARRADIANCEINC.Massivelyparallelebeamsourcesforsemiconductorlithographyandmetrology. 2007http://arradiance.com/. ASCOLIGA.Progressandperspectivesincomputationalneuroanatomy.AnatomicalRecord, 257:195207,1999. ATWOODHLandWOJTOWICZJM.Silentsynapsesinneuralplasticity:Currentevidence. Learning&Memory,6:542571,1999. BACHTOLDA,HADLEYP,NAKANISHIT,andDEKKERC.Logiccircuitswithcarbonnanotube transistors.Science,294:13171320,2001. BAIRDRC,JOHARIH,andGY.J.Numericalsimulationofenvironmentmodulationofchemical signalstructureandodordispersalintheopenocean.ChemSenses.,21:121134,1996 BARZELR,HUGHESJF,andWOODDN.Plausiblemotionsimulationforcomputergraphics animation.InBoulicR.andHgronG.(Eds.),Computeranimationandsimulation96: Springer,1996,pp.183197. BEECHERCWW.Thehumanmetabolome.InHarriganG.G.(Ed.)Metabolicprofiling:Itsrolein biomarkerdiscoveryandgenefunctionanalysis:SpringerVerlag,2003. BEKENSTEINJD.Universalupperboundontheentropytoenergyratioforboundedsystems. PhysicalReviewD,23:287298,1981. BENHAMMP,WRIGHTDK,andBIBBR.Modellingsofttissueforkinematicanalysisofmulti segmenthumanbodymodels.BiomedSciInstrum,37:1116,2001. BERGRW,ALABURDAA,andHOUNSGAARDJ.Balancedinhibitionandexcitationdrivespike activityinspinalhalfcenters.Science,315:390393,2007. BHALLAUS.Signalinginsmallsubcellularvolumes.I.Stochasticanddiffusioneffectson individualpathways.BiophysicalJournal,87:733744,2004a. BHALLAUS.Signalinginsmallsubcellularvolumes.Ii.Stochasticanddiffusioneffectson synapticnetworkproperties.BiophysicalJournal,87:745753,2004b. BHALLAUSandIYENGARR.Emergentpropertiesofnetworksofbiologicalsignaling pathways.Science,283:381387,1999.

113

BHARDWAJRD,CURTISMA,SPALDINGKL,etal.Neocorticalneurogenesisinhumansis restrictedtodevelopment.ProceedingsoftheNationalAcademyofSciencesoftheUnited StatesofAmerica,103:1256412568,2006. BIGQandPOOMM.Synapticmodificationsinculturedhippocampalneurons:Dependence onspiketiming,synapticstrength,andpostsynapticcelltype.JournalofNeuroscience, 18:1046410472,1998. BIENENSTOCKEL,COOPERLN,andMUNROPW.Theoryforthedevelopmentofneuron selectivityorientationspecificityandbinocularinteractioninvisualcortex.Journalof Neuroscience,2:3248,1982. BINZEGGERT,DOUGLASRJ,andMARTINKAC.Aquantitativemapofthecircuitofcatprimary visualcortex.JournalofNeuroscience,24:84418453,2004. BIOMECHANICSRESEARCHGROUPINC.Brg.Lifemod2005:Biomechanicsmodelingpackage.2005 http://www.lifemodeler.com/. BLATOWM,CAPUTIA,andMONYERH.Moleculardiversityofneocorticalgabaergic interneurones.JournalofPhysiologyLondon,562:99105,2005. BLEMKERSS,ASAKAWADS,GOLDGE,andDELPSL.Imagebasedmusculoskeletalmodeling: Applications,advances,andfutureopportunities.JMagnResonImaging,25:44151, 2007. BLEMKERSSandDELPSL.Threedimensionalrepresentationofcomplexmusclearchitectures andgeometries.AnnBiomedEng,33:66173,2005. BOKILH,LAARISN,BLINDERK,ENNISM,andKELLERA.Ephapticinteractionsinthe mammalianolfactorysystem.JournalofNeuroscience,21:,2001. BOSTROMN.Howlongbeforesuperintelligence?Int.Jour.ofFutureStudies,2,1998. BOSTROMN.Areyoulivinginacomputersimulation?PhilosophicalQuarterly,53:243255, 2003. BOWERJMandBEEMAND.Thebookofgenesis:Exploringrealisticneuralmodelswiththegeneral neuralsimulationsystem.NewYork:SpringerVerlag,1998. BRAITENBERGVandSCHUZA.Cortex:Statisticsandgeometryofneuronalconnectivity.NewYork: SpringerVerlag.,1998. BRETTER,RUDOLPHM,CARNEVALET,etal.Simulationofnetworksofspikingneurons:A reviewoftoolsandstrategies.JournalofComputationalNeuroscience,23:349398,2007. BRIGGMANKLandDENKW.Towardsneuralcircuitreconstructionwithvolumeelectron microscopytechniques.CurrentOpinioninNeurobiology,16:562570,2006. BRITTRHandROSSIGT.Quantitativeanalysisofmethodsforreducingphysiologicalbrain pulsations.JournalofNeuroscienceMethods,6:219229,1982. BROWNIE,SCOTTSH,andLOEBGE.Mechanicsoffelinesoleus:Ii.Designandvalidationofa mathematicalmodel.JMuscleResCellMotil,17:22133,1996. BRUNELN,HAKIMV,ISOPEP,NADALJP,andBARBOURB.Optimalinformationstorageandthe distributionofsynapticweights:Perceptronversuspurkinjecell.Neuron,43:745757, 2004. BULLOCKTH.Revisitingtheconceptofidentifiableneurons.BrainBehaviorandEvolution,55: 236240,2000. CAILL,COURTINEG,FONGAJ,etal.Plasticityoffunctionalconnectivityintheadultspinal cord.PhilosophicalTransactionsoftheRoyalSocietyBBiologicalSciences,361:16351646, 2006. CAIANIELLOER.Outlineofatheoryofthoughtprocessandthinkingmachines.J.Theoret.Biol., 1:204235,1961. CARNEVALENTandHINESML.Theneuronbook.Cambridge,UK:CambridgeUniversityPress, 2006.

114

CASHAD,ALIEVG,SIEDLAKSL,etal.Microtubulereductioninalzheimersdiseaseandaging isindependentoftaufilamentformation.AmericanJournalofPathology,162:1623 1627,2003. CCORTEX.Artificialdevelopmenttobuildworldsbiggestspikingneuralnetwork.2003 http://www.corticaldb.com/ccortex.asp. CHALMERSDJ.Absentqualia,fadingqualia,dancingqualia.InMetzingerT.(Ed.)Conscious experience:ImprintAcademic,1995. CHANGYMandLUEBKEJI.Electrophysiologicaldiversityoflayer5pyramidalcellsinthe prefrontalcortexoftherhesusmonkey:Invitroslicestudies.Journalof Neurophysiology,98:26222632,2007. CHAOSH,DOUGHERTYWM,GARBINIJL,andSIDLESJA.Nanometerscalemagneticresonance imaging.ReviewofScientificInstruments,75:11751181,2004. CHENDTandZELTZERD.Pumpitup:Computeranimationofabiomechanicallybasedmodel ofmuscleusingthefiniteelementmethod.InThomasJ.J.(Ed.)Proceedingsofthe19th annualconferenceoncomputergraphicsandinteractivetechniques,1992,pp.8998. CHENLY,REXCS,CASALEMS,GALLCM,andLYNCHG.Changesinsynapticmorphology accompanyactinsignalingduringltp.JournalofNeuroscience,27:53635372,2007. CHERNIAKC.Theboundedbrain:Towardquantitativeneuroanatomy.JournalofCognitive Neuroscience,2:5868,1990. CHURCHLANDPandSEJNOWSKITJ.Thecomputationalbrain:MITPress,1992. CIOBANUL,SEEBERDA,andPENNINGTONCH.3dmrmicroscopywithresolution3.7mumby 3.3mumby3.3mum.JournalofMagneticResonance,158:178182,2002. CIVELLIO.Gpcrdeorphanizations:Thenovel,theknownandtheunexpectedtransmitters. TrendsinPharmacologicalSciences,26:1519,2005. COLOMBELLIJ,GRILLSW,andSTELZEREHK.Ultravioletdiffractionlimitednanosurgeryof livebiologicaltissues.ReviewofScientificInstruments,75:472478,2004. COLONNIERM.Theelectronmicroscopicanalysisoftheneuronalorganizationofthecerebral cortex.InSchmittF.,WordenF.,AdelmanG.,andDennisS.(Eds.),Theorganizationof thecerebralcortex.Cambridge:MITPress,1981,pp.125152. COMPAOR,MOLENKAMPL,andPAULDJ.Technologyroadmapfornanoelectronics. MicroelectronicsAdvancedResearchInitiative,1999 COTINS,DELINGETTEH,andAYACHEN.Realtimeelasticdeformationsofsofttissuesfor surgerysimulation.IeeeTransactionsonVisualizationandComputerGraphics,5:6273, 1999. CRAGGSJ,NICHOLSONC,KUMEKICKJ,TAOL,andRICEME.Dopaminemediatedvolume transmissioninmidbrainisregulatedbydistinctextracellulargeometryanduptake. JournalofNeurophysiology,85:17611771,2001. CROSSINKLandKRUSHELLA.Cellularsignalingbyneuralcelladhesionmoleculesofthe immunoglobulinsuperfamily.DevelopmentalDynamics,218:260279,2000. CROWNED,FERGUSONAR,JOYNESRL,andGRAUJW.Instrumentallearningwithinthespinal cordii.Evidenceforcentralmediation.Physiology&Behavior,77:259267,2002. DAISYPROJECT.Neocorticaldaisyarchitecturesandgraphicalmodelsforcontextdependent processing.2008http://daisy.ini.unizh.ch/. DEBENEDICTISE,KOGGEP,LENTC,NIEMIERM,andSTERLINGT.Thetechnologylaneontheroad toazettaflops,tr200615.UniversityofNotreDameComputerScienceand EngineeringFaculty,2006www.cse.nd.edu/Reports/2006/TR200615.pdf. DEBENEDICTISEP.Takingascisupercomputingtotheendgame,sand20040959.SandiaNational Laboratories,2004 DEBENEDICTISEP.Reversiblelogicforsupercomputing.InProceedingsofthe2ndconferenceon computingfrontiers,2005,pp.391402.

115

DENKWandHORSTMANNH.Serialblockfacescanningelectronmicroscopytoreconstruct threedimensionaltissuenanostructure.PlosBiology,2:19001909,2004. DIVENTURAB,LEMERLEC,MICHALODIMITRAKISK,andSERRANOL.Frominvivotoinsilico biologyandback.Nature,443:527533,2006. DILLONCandGODAY.Theactincytoskeleton:Integratingformandfunctionatthesynapse. AnnualReviewofNeuroscience,28:2555,2005. DIMAA,SCHOLZM,andOBERMAYERK.Automaticsegmentationandskeletonizationof neuronsfromconfocalmicroscopyimagesbasedonthe3dwavelettransform.Image Processing,IEEETransactionson,11:790801,2002. DIXA.Thebrainandtheweb:Aquickbackupincaseofaccidents.Interfaces,65:67,2005. DJURFELDTM,JOHANSSONC,EKEBERG,etal.Massivelyparallelsimulationofbrainscale neuronalnetworkmodels.Stockholm:CSC,KTH,2005http://www 03.ibm.com/servers/deepcomputing/pdf/Blue_Gene_Applications_Paper_KTH_0306. pdf. DJURFELDTM,LUNDQVISTM,JOHANSSONC,etal.Projectreportforbluegenewatsonconsortium days:Massivelyparallelsimulationofbrainscaleneuronalnetworkmodels.2006 http://wwwfp.mcs.anl.gov/bgconsortium/Past%20Results/kth%20bgwreport06.pdf. DOAKRB,GRISENTIRE,REHBEINS,etal.Towardsrealizationofanatomicdebroglie microscope:Heliumatomfocusingusingfresnelzoneplates.PhysicalReviewLetters, 83:42294232,1999. DONIZELLIM,DJITEMA,andLENOVEREN.Lgicdb:Amanuallycuratedsequencedatabase afterthegenomes.NucleicAcidsResearch,34:D267D269,2006. DOUGLASSJKandWILKENSLA.Directionalselectivitiesofnearfieldfiliformhair mechanoreceptorsonthecrayfishtailfan(crustacea:Decapoda).Journalof ComparativePhysiologyaNeuroethologySensoryNeuralandBehavioralPhysiology,183: 2334,1998. DOYLECA,LONGSOND,HUTCHINSONC,andSIMPSONMDC.Mrioffrozenbraintissue:A freshlookatmorphometryinschizophrenia.SchizophreniaResearch,18:Xg1Xg1, 1996. DREXLERKE.Enginesofcreation:Thecomingeraofnanotechnology:AnchorBooks,1986. DREXLERKE.Nanosystems:Molecularmachinery,manufacturingandcomputation:Wiley1992. DRUCKMANNS,BANITTY,GIDONAA,etal.Anovelmultipleobjectiveoptimization frameworkforconstrainingconductancebasedneuronmodelsbyexperimentaldata. Frontiersinneuroscience,1:718,2007. EKEBERG,BLMELM,andBSCHGESA.Dynamicsimulationofinsectwalking.Arthropod Structure&Development,33:287300,2004. EKEBERGOandGRILLNERS.Simulationsofneuromuscularcontrolinlampreyswimming. PhilosTransRSocLondBBiolSci,354:895902,1999. EKEBERGOandPEARSONK.Computersimulationofsteppinginthehindlegsofthecat:An examinationofmechanismsregulatingthestancetoswingtransition.JNeurophysiol, 94:425668,2005. ELIASMITHC.Attractiveandindiscreteacritiqueoftwoputativevirtuesofthedynamicist theoryofmind.MindsandMachines,11:417426,2001. ELIXSONE.Hypothermia.Coldwaterdrowning.CritCareNursClinNorthAm.,3:287292, 1991. EMBLEBI.Ligandgatedionchanneldatabase.2008http://www.ebi.ac.uk/compneur srv/LGICdb/LGICdb.php. FEITELSONDG.Thesupercomputerindustryinlightofthetop500data.ComputinginScience andEngineering,7:4247,2005. FELLINTandCARMIGNOTOG.Neuronetoastrocytesignallinginthebrainrepresentsa distinctmultifunctionalunit.JournalofPhysiologyLondon,559:315,2004.

116

FENSTERMACHERJandKAYET.Drugdiffusionwithinthebrain.AnnalsoftheNewYork AcademyofSciences,531:2939,1988. FERNANDEZJWandPANDYMG.Integratingmodellingandexperimentstoassessdynamic musculoskeletalfunctioninhumans.ExpPhysiol,91:37182,2006. FIALAJC.Threedimensionalstructureofsynapsesinthebrainandontheweb.InIjcnn02. Proceedingsofthe2002internationaljointconferenceonneuralnetworks:IEEEPress,2002, pp.14. FIALAJC.Synapses.Bu.Edu.2007,2007http://synapses.bu.edu/. FIALAJCandHARRISKM.Extendingunbiasedstereologyofbrainultrastructuretothree dimensionalvolumes.JournaloftheAmericanMedicalInformaticsAssociation,8:116, 2001. FIELDSRDandELLISMANMH.Synapticmorphologyanddifferencesinsensitivity.Science, 228:197199,1985. FORESIGHTNANOTECHINSTITUTEandBATELLEMEMORIALINSTITUTE.Productivenanosystems:A technologyroadmap.2007http://www.foresight.org/roadmaps/. FORSBERGE.Reversiblelogicbasedonelectronwaveguideybranchswitches.Nanotechnology, 15:S298S302,2004. FORSBERGEandHIEKEK.Electronwaveguideybranchswitchescontrolledbypt/gaas schottkygates.PhysicaScripta,T101:158160,2002. FORTUNEEandROSEG.Shorttermsynapticplasticityasatemporalfilter.TrendsNeuroscience, 24:381385,2001 FRANKW.Electrontomography:Threedimensionalimagingwiththetransmissionelectron microscope.NewYork:Plenum,1992. FREITASJRRA.Nanomedicine,volumei:Basiccapabilities.Georgetown,TX:LandesBioscience, 1999. FREITASJR.RA.Thefutureofcomputers.Analog,116:5773,1996. FREITASJR.RA.Nanomedicine,volumeiia:Biocompatibility.Georgetown,TX:LandesBioscience, 2003. FRYEJ.Bluegenetestreport.2004 http://brain.unr.edu/publications/NCS_BlugGene_report_07Nov04.pdf. FRYEJ,ANANTHANARAYANANR,andMODHADS.Towardsrealtime,mousescalecortical simulations.SaltLakeCity,Utah,Feb22252007,2007 http://www.modha.org/papers/rj10404.pdf. FUJISAKIH,TAKAHASHIS,OHZEKIH,etal.Softxraydamagetobiologicalsamples.Journalof MicroscopyOxford,182:7983,1996. FUNKHOUSERT,TSINGOSN,CARLBOMI,etal.Abeamtracingmethodforinteractive architecturalacoustics.JournaloftheAcousticalSocietyofAmerica,115:739756,2004. GAMMAITONIL,HANGGIP,JUNGP,andMARCHESONIF.Stochasticresonance.Reviewsof ModernPhysics,70:223287,1998. GARDNERMEDWINAR.Analysisofpotassiumdynamicsinmammalianbraintissue.Journalof PhysiologyLondon,335:393426,1983. GARRIGAA,SPAC,andLOPEZV.Computationofthecompleteacousticfieldwithfinitedifferences algorithms.Budapest,2005 http://www.tecn.upf.es/~agarriga/ARCHIVOS_ADAN/fa_2005_paper.pdf. GELPERINDandHETZELB.Thegrowthofsoftwaretesting.CommunicationsoftheAcm,31:687 695,1988. GEMMELLJ,BELLG,andLUEDERR.Mylifebits:Apersonaldatabaseforeverything. CommunicationsoftheAcm,49:8895,2006. GLOVERPandMANSFIELDP.Limitstomagneticresonancemicroscopy.ReportsonProgressin Physics,65:14891511,2002.

117

GOODMANPH,COURTENAYWILSONE,MACIOKASJB,etal.Largescaleparallelsimulationof physiologicallyrealisticmulticolumnsensorycortex.2001 http://brain.unr.edu/publications/gwm.largescalecortex_01.pdf. GRAYCM,KONIGP,ENGELAK,andSINGERW.Oscillatoryresponsesincatvisualcortex exhibitintercolumnarsynchronizationwhichreflectsglobalstimulusproperties. Nature,338:334337,1989. GRIMARandSCHNELLS.Asystematicinvestigationoftheratelawsvalidinintracellular environments.BiophysicalChemistry,124:110,2006. GRUTZENDLERJ,KASTHURIN,andGANWB.Longtermdendriticspinestabilityintheadult cortex.Nature,420:812816,2002. GUPTAA,WANGY,andMARKRAMH.Organizingprinciplesforadiversityofgabaergic interneuronsandsynapsesintheneocortex.Science,287:273278,2000. GUSTAFSSONMGL.Nonlinearstructuredilluminationmicroscopy:Widefieldfluorescence imagingwiththeoreticallyunlimitedresolution.ProceedingsoftheNationalAcademyof SciencesoftheUnitedStatesofAmerica,102:1308113086,2005. GUTMANGA,CHANDYKG,GRISSMERS,etal.Internationalunionofpharmacology.Liii. Nomenclatureandmolecularrelationshipsofvoltagegatedpotassiumchannels. PharmacologicalReviews,57:473508,2005. HAGANS,HAMEROFFSR,andTUSZYNSKIJA.Quantumcomputationinbrainmicrotubules: Decoherenceandbiologicalfeasibility.PhysicalReviewE,65:,2002. HAMEROFFSR.Ultimatecomputing:Biomolecularconsciousnessandnanotechnology:Elsevier SciencePublishers,1987. HAMMARLUNDPandEKEBERGO.Largeneuralnetworksimulationsonmultiplehardware platforms.JournalofComputationalNeuroscience,5:443459,1998. HANLONEB,MANOHARANR,KOOTW,etal.Prospectsforinvivoramanspectroscopy. PhysicsinMedicineandBiology,45:R1R59,2000. HANSONR.Ifuploadscomefirst:Thecrackofafuturedawn.Extropy,6,1994. HANSONR.Thenextreallybigenormousthing.FutureBrief,October6,2004. HANSONR.Economicsofbrainemulations.InHealeyP.(Ed.)Tomorrowspeopleproceedings ofthejamesmartininstitutesfirstworldforum:EarthScan,2008a. HANSONR.Economicsofthesingularity.IEEESpectrum:3742,2008b. HARDMANJG,BEDFORTHNM,AHMEDAB,MAHAJANRP,andAITKENHEADAR.Aphysiology simulator:Validationofitsrespiratorycomponentsanditsabilitytopredictthe patientsresponsetochangesinmechanicalventilation.BritishJournalofAnaesthesia, 81:327332,1998. HARRISFC,BAURICKJ,FRYEJ,etal.Anovelparallelhardwareandsoftwaresolutionforalargescale biologicallyrealisticcorticalsimulation.GoodmanBrainComputationLab,Universityof Nevada.,2002 HARRISKandSTEVENSJ.Dendriticspinesofca1pyramidalcellsintherathippocampus: Serialelectronmicroscopywithreferencetotheirbiophysicalcharacteristics. J.Neurosci,9:29822997,1989. HARRISONJ,RENSINKRA,andVANDEPANNEM.Obscuringlengthchangesduringanimated motion.AcmTransactionsonGraphics,23:569573,2004. HARVEYCDandSVOBODAK.Locallydynamicsynapticlearningrulesinpyramidalneuron dendrites.Nature,450:1195U3,2007. HAYMANS.Themccullochpittsmodel.NeuralNetworks,6:44384439,1999. HAYWORTHKJ.Singlebrainphysicalslicelibraryproposal:Creatingacompletehumanneural connectivitydatabaseviasparse,automaticallydirectedultramicroscopicimagingusingan automatedretrievalbrainslicestoragesystem.2002 http://www.extremeneuroanatomy.com/PhysicalSliceLibraryProposal.pdf.

118

HAYWORTHKJ.Atlumprojecthomepage.2007,2007 http://www.mcb.harvard.edu/lichtman/ATLUM/ATLUM_web.htm. HAYWORTHKJ,KASHTURIN,SCHALEKR,andLICHTMANJW.Automatingthecollectionof ultrathinserialsectionsforlargevolumetemreconstructions.Chicago,2006 HERZAVM,GOLLISCHT,MACHENSCK,andJAEGERD.Modelingsingleneurondynamicsand computations:Abalanceofdetailandabstraction.Science,314:8085,2006. HIGGINSTV.Computingwithphotonsatthespeedoflight.LaserFocusWorld,31:7277,1995. HILLERBRANDR.Scaleseparationasaconditionforquantitativemodelling.Whymathematics worksforsomeproblemsandfailsforothers.Synthese,2008. HINESM,MARKRAMH,andSCHUERMANNF.Fullyimplicitparallelsimulationofsingle neurons.JCompNeurosci,2008. HODGKINAandHUXLEYA.Aquantitativedescriptionofmembranecurrentandits applicationtoconductionandexcitationinnerve.JPhysiol.,117:500544,1952. HOKFELTT,BROBERGERC,XUZQD,etal.Neuropeptidesanoverview.Neuropharmacology, 39:13371356,2000. HOLSTBandALLISONW.Anatomfocusingmirror.Nature,390:244244,1997. HOLTMAATAJGD,TRACHTENBERGJT,WILBRECHTL,etal.Transientandpersistentdendritic spinesintheneocortexinvivo.Neuron,45:279291,2005. HOUWELINGARandBRECHTM.Behaviouralreportofsingleneuronstimulationin somatosensorycortex.Nature,451:65U8,2008. HOWELLFW,DYHRFJELDJOHNSENJ,MAEXR,GODDARDN,andDESCHUTTERE.Alargescale modelofthecerebellarcortexusingpgenesis.Neurocomputing,32:10411046,2000. HULLC,STUDHOLMEK,YAZULLAS,andVONGERSDORFFH.Diurnalchangesinexocytosisand thenumberofsynapticribbonsatactivezonesofanontypebipolarcellterminal. JournalofNeurophysiology,96:20252033,2006. HUMPHERYSMITHI.Ahumanproteomeprojectwithabeginningandanend.Proteomics,4: 25192521,2004. IMREA,CSABAG,JIL,etal.Majoritylogicgateformagneticquantumdotcellularautomata. Science,311:205208,2006. INUIK,NAKAZAWAS,YOSHINOJ,andUKAIH.Endoscopicmri.Pancreas,16:413417,1998. ITOAandSHINOHARAK.Imageblurringbythermaldiffusionintheobservationofhydrated biomoleculeswithsoftxraymicroscopy.CellStructureandFunction,17:209212, 1992. ITRS07.Internationaltechnologyroadmapforsemiconductors, http://www.Itrs.Net/links/2007itrs/home2007.Htm.2007 IVANCEVICVandBEAGLEYN.Determiningtheacceptablelimitsofheadmountedloads.Land OperationsDivision,SystemsSciencesLaboratory,AustralianDefenceScienceand TechnologyOrganisation,2004 IZHIKEVICHEM.Simplemodelofspikingneurons.IeeeTransactionsonNeuralNetworks,14: 15691572,2003. IZHIKEVICHEM.Whichmodeltouseforcorticalspikingneurons?IeeeTransactionsonNeural Networks,15:10631070,2004. IZHIKEVICHEM.Simulationoflargescalebrainmodels.2007,2005 http://vesicle.nsi.edu/users/izhikevich/human_brain_simulation/Blue_Brain.htm#Sim ulation%20of%20LargeScale%20Brain%20Models. IZHIKEVICHEM.Dynamicalsystemsinneuroscience:Thegeometryofexcitabilityandbursting. Cambridge,Massachusetts:MITPress,2007. JACOBSENC.Softxraymicroscopy.TrendsinCellBiology,9:4447,1999. JELINEKHFandFERNANDEZE.Neuronsandfractals:Howreliableandusefularecalculations offractaldimensions?JournalofNeuroscienceMethods,81:918,1998.

119

JOHANSSONCandLANSNERA.Towardscortexsizedartificialneuralsystems.Neural Networks,20:4861,2007. KALISMANN,SILBERBERGG,andMARKRAMH.Derivingphysicalconnectivityfromneuronal morphology.BiologicalCybernetics,88:210218,2003. KALISMANN,SILBERBERGG,andMARKRAMH.Theneocorticalmicrocircuitasatabularasa. ProceedingsoftheNationalAcademyofSciencesoftheUnitedStatesofAmerica,102:880 885,2005. KASPEREM,LARKMANAU,LUBKEJ,andBLAKEMOREC.Pyramidalneuronsinlayer5ofthe ratvisualcortex.1.Correlationamongcellmorphology,intrinsicelectrophysiological properties,andaxontargets.JournalofComparativeNeurology,339:459474,1994. KAUFMANA,DRORG,MEILIJSONI,andRUPPINE.Geneexpressionofcaenorhabditiselegans neuronscarriesinformationontheirsynapticconnectivity.PlosComputationalBiology, 2:15611567,2006. KERENN,PELEDN,andKORNGREENA.Constrainingcompartmentalmodelsusingmultiple voltagerecordingsandgeneticalgorithms.JournalofNeurophysiology,94:37303742, 2005. KEYHANIK,SCHERERPW,andMOZELLMM.Anumericalmodelofnasalodoranttransportfor theanalysisofhumanolfaction.JournalofTheoreticalBiology,186:279301,1997. KIKUCHIS,FUJIMOTOK,KITAGAWAN,etal.Kineticsimulationofsignaltransductionsystem inhippocampallongtermpotentiationwithdynamicmodelingofprotein phosphatase2a.NeuralNetworks,16:13891398,2003. KIRBASCandQUEKF.Areviewofvesselextractiontechniquesandalgorithms.Acm ComputingSurveys,36:81121,2004. KLATZKYRL,LEDERMANSJ,HAMILTONC,andGRINDLEYM.Feelingtexturesthroughaprobe: Effectsofprobeandsurfacegeometryandexploratoryfactors.Perception& Psychophysics,65:613631,2003. KOCHCandHEPPK.Quantummechanicsinthebrain.Nature,440:611612,2006. KOENERA.Netmorph.2008http://netmorph.org/. KOHWandMCCORMICKBH.Specificationsforvolumedataacquisitioninthreedimensionallight microscopy,technicalreporttr200375.CollegeStation,TX:DepartmentofComputer Science,TexasA&MUniversity,2003 KONDOY,KOSHIBAY,ARIMAY,etal.A1.2gflopsneuralnetworkchipforhighspeedneural networkservers.IeeeJournalofSolidStateCircuits,31:860864,1996. KOUZNETSOVD,OBERSTH,NEUMANNA,etal.Ridgedatomicmirrorsandatomicnanoscope. JournalofPhysicsBAtomicMolecularandOpticalPhysics,39:16051623,2006. KOZLOVAK,LANSNERA,GRILLNERS,andKOTALESKIJH.Ahemicordlocomotornetworkof excitatoryinterneurons:Asimulationstudy.BiologicalCybernetics,96:229243,2007. KOZLOVAS,ANGULOMC,AUDINATE,andCHARPAKS.Targetcellspecificmodulationof neuronalactivitybyastrocytes.ProceedingsoftheNationalAcademyofSciencesofthe UnitedStatesofAmerica,103:1005810063,2006. KRAFFTC.Bioanalyticalapplicationsoframanspectroscopy.AnalyticalandBioanalytical Chemistry,378:6062,2004. KRAFFTC,KNETSCHKET,SIEGNERA,FUNKRHW,andSALZERR.Mappingofsinglecellsby nearinfraredramanmicrospectroscopy.VibrationalSpectroscopy,32:7583,2003. KRIMERLS,ZAITSEVAV,CZANNERG,etal.Clusteranalysisbasedphysiologicalclassification andmorphologicalpropertiesofinhibitoryneuronsinlayers23ofmonkey dorsolateralprefrontalcortex.JournalofNeurophysiology,94:30093022,2005. KRNJEVICK.Ephapticinteractions:Asignificantmodeofcommunicationsinthebrain.News inPhysiologicalSciences,1:2829,1986.

120

KRUITP.Highthroughputelectronlithographywiththemultipleaperturepixelbypixel enhancementofresolutionconcept.JournalofVacuumScience&TechnologyB,16:3177 3180,1998. KULLMANNDM.Silentsynapses:Whataretheytellingusaboutlongtermpotentiation? PhilosophicalTransactionsoftheRoyalSocietyofLondonSeriesBBiologicalSciences,358: 727733,2003. KURZWEILR.Theageofspiritualmachines.NewYork:Viking,1999. KURZWEILR.Thesingularityisnear:Whenhumanstranscendbiology:VikingPress,2005. KUZIRIANAMandLEIGHTONSB.Oxygenplasmaetchingofentireblockfacesimprovesthe resolutionandusefulnessofserialscanningelectronmicroscopicimages.Scanning ElectronMicroscopy:18771885,1983. KWONJ,MAYERICHD,CHOEY,andMCCORMICKBH.Automatedlateralsectioningforknife edgescanningmicroscopy.InIeeeinternationalsymposiumonbiomedicalimaging:From nanotomacro,2008. LANDAUERTK.Howmuchdopeopleremembersomeestimatesofthequantityoflearned informationinlongtermmemory.CognitiveScience,10:477493,1986. LEDOA,FRADEJ,BARBOSARM,andLARANJINHAJ.Nitricoxideinbrain:Diffusion,targets andconcentrationdynamicsinhippocampalsubregions.MolecularAspectsof Medicine,25:7589,2004. LEEWCA,HUANGH,FENGGP,etal.Dynamicremodelingofdendriticarborsingabaergic interneuronsofadultvisualcortex.PlosBiology,4:271280,2006. LEENDERSKL,PERANID,LAMMERTSMAAA,etal.Cerebralbloodflow,bloodvolumeand oxygenutilizationnormalvaluesandeffectofage.Brain,113:2747,1990. LEIGHTONSB.Semimagesofblockfaces,cutbyaminiaturemicrotomewithinthesema technicalnote.ScanningElectronMicroscopy:7376,1981. LEITLE.Neurosuspensionanduploading.2007,1995 http://www.aleph.se/Trans/Individual/Cryonics/neurosusp_upload.txt. LESTIENNER.Determinationoftheprecisionofspiketiminginthevisualcortexof anaesthetisedcats.BiologicalCybernetics,74:5561,1996. LEVENSONJMandSWEATTJD.Epigeneticmechanismsinmemoryformation.NatureReviews Neuroscience,6:108118,2005. LIC,YUJ,andLIAOX.Chaosinathreeneuronhysteresishopfieldtypeneuralnetworks. PhysicsLettersA,285:368372,2001. LIKW.Proteomicsofsynapse.AnalyticalandBioanalyticalChemistry,387:2528,2007. LIXQ,WUYW,STEELD,etal.Anallopticalquantumgateinasemiconductorquantumdot. Science,301:809811,2003. LIEBERMANM,CHELLAMMAS,VARUGHESEB,etal.Quantumdotcellularautomataata molecularscale.MolecularElectronicsIi,960:225239,2002. LIKHAREVKKandSEMENOVVK.Rsfqlogic/memoryfamily:Anewjosephsonjunction technologyforsubterahertzclockfrequencydigitalsystems.Applied Superconductivity,IEEETransactionson,1:328,1991. LINMCandOTADUYMA.Sensationpreservinghapticrendering.IeeeComputerGraphicsand Applications,25:811,2005. LITTA,ELIASMITHC,KROONFW,WEINSTEINS,andTHAGARDP.Isthebrainaquantum computer?CognitiveScience,30:593603,2006. LLOYDDGandBESIERTF.Anemgdrivenmusculoskeletalmodeltoestimatemuscleforces andkneejointmomentsinvivo.JBiomech,36:76576,2003. LONDONMandHAUSSERM.Dendriticcomputation.AnnualReviewofNeuroscience,28:503 532,2005. LONGSOND,HUTCHINSONCE,DOYLECA,etal.Useofmriformeasuringstructuresinfrozen postmortembrain.BrainResearchBulletin,38:457460,1995.

121

LOVESandCOAKHAMHB.Trigeminalneuralgiapathologyandpathogenesis.Brain,124: 23472360,2001. LUIV,FRSTERF,andBAUMEISTERW.Structuralstudiesbyelectrontomography:From cellstomolecules.AnnualReviewofBiochemistry,74:83365,2005. MAYY,HUH,BERREBIAS,MATHERSPH,andAGMONA.Distinctsubtypesofsomatostatin containingneocorticalinterneuronsrevealedintransgenicmice.Journalof Neuroscience,26:50695082,2006. MABUCHIM,SHIMADAJ,OKAMOTOK,etal.Timeresolvedfluorescencespectroscopyof dopamineinsinglecells.InLakowiczJ.R.andThompsonR.B.(Eds.),Proceedingsof spievolume4252:Advancesinfluorescencesensingtechnologyv:SPIE,2001,pp.140148. MACLEODKM,HORIUCHITK,andCARRCE.Aroleforshorttermsynapticfacilitationand depressionintheprocessingofintensityinformationintheauditorybrainstem.J Neurophysiol,97:286374,2007. MAGNENATTHALMANNNandCORDIERF.Constructionofahumantopologicalmodelfrom medicaldata.IeeeTransactionsonInformationTechnologyinBiomedicine,4:137143, 2000. MAHVASHMandHAYWARDV.Highfidelityhapticsynthesisofcontactwithdeformable bodies.IeeeComputerGraphicsandApplications,24:4855,2004. MALICKASA.Gradualuploadingasacognitionofmind.2007,1996 http://www.aleph.se/Trans/Global/Uploading/gupload.html. MALINSKIT,TAHAZ,GRUNFELDS,etal.Diffusionofnitricoxideintheaortawallmonitored insitubyporphyrinicmicrosensors.BiochemicalandBiophysicalResearch Communications,193:10761082,1993. MARKRAMH.Neocorticalmicrocircuitdatabase.EPFL,2005http://microcircuit.epfl.ch/. MARKRAMH.Thebluebrainproject.NatureReviewsNeuroscience,7:153160,2006. MARKRAMH,LUBKEJ,FROTSCHERM,andSAKMANNB.Regulationofsynapticefficacyby coincidenceofpostsynapticapsandepsps.Science,275:213215,1997. MARRONEDF.Ultrastructuralplasticityassociatedwithhippocampaldependentlearning:A metaanalysis.NeurobiologyofLearningandMemory,87:361371,2007. MARRONEDFandPETITTL.Theroleofsynapticmorphologyinneuralplasticity:Structural interactionsunderlyingsynapticpower.BrainResearchReviews,38:291308,2002. MASONAandLARKMANA.Correlationsbetweenmorphologyandelectrophysiologyof pyramidalneuronsinslicesofratvisualcortex.2.Electrophysiology.Journalof Neuroscience,10:14151428,1990. MATTIAMandGIUDICEPD.Efficienteventdrivensimulationoflargenetworksofspiking neuronsanddynamicalsynapses.NeuralComputation,12:23052329,2000. MAYERICHD,ABBOTTL,andMCCORMICKB.Knifeedgescanningmicroscopyforimagingand reconstructionofthreedimensionalanatomicalstructuresofthemousebrain.2008 MAYERICHD,MCCORMICKBH,andKEYSERJ.Noiseandartifactremovalinknifeedge scanningmicroscopy.InPiscatawayN.(Ed.)Proceedingsof2007ieeeinternational symposiumonbiomedicalimaging:Fromnanotomacro:IEEEPress,2007. MAYERICHDMandKEYSERJ.Filamenttrackingandencodingforcomplexbiological networks.InProceedingsofacmsymposiumonsolidandphysicalmodeling,2008,pp.353 358. MCCALLUMJC.Cpupriceperformance19442003.2007,2003http://www.jcmit.com/cpu performance.htm. MCCALLUMJC.Diskdriveprices(19552007).2007,2007ahttp://www.jcmit.com/diskprice.htm. MCCALLUMJC.Memoryprices(19572007).2007,2007b http://www.jcmit.com/memoryprice.htm. MCCLUNGCAandNESTLEREJ.Neuroplasticitymediatedbyalteredgeneexpression. Neuropsychopharmacology,33:317,2008.

122

MCCORMICKBH.Braintissuescannerenablesbrainmicrostructuresurveys.Neurocomputing, 44:11131118,2002a. MCCORMICKBH.Developmentofthebraintissuescanner.DepartmentofComputerScience, TexasA&MUniversity,2002b MCCORMICKBH,KOHW,CHOEY,etal.Constructionofanatomicallycorrectmodelsof mousebrainnetworks.Neurocomputing,5860:379386,2004. MCCULLOCHWSandPITTSWH.Alogicalcalculusoftheideasimmanentinnervousactivity. .BulletinofMathematicalBiophysics,5:115133,1943. MCDOUGALLMPandWRIGHTSM.Initialresultsinwidefield3dmrmicroscopyusing parallelimaging.InBiomedicalimaging:Fromnanotomacro,2007.Isbi2007.4thieee internationalsymposiumon:IEEE,2007,pp.10721075. MCGUIGANM.Graphicsturingtest,arxiv:Cs.Gr/0603132.2007,2006 http://arxiv.org/abs/cs.GR/0603132. MEHRTASHN,JUNGD,HELLMICHHH,etal.Synapticplasticityinspikingneuralnetworks ((spinn)i2):Asystemapproach.IeeeTransactionsonNeuralNetworks,14:980992, 2003. MEINDLJD,CHENQ,andDAVISJA.Limitsonsiliconnanoelectronicsforterascaleintegration. Science,293:20442049,2001. MERKLERC.Energylimitstothecomputationalpowerofthehumanbrain.ForesightUpdate, 6,1989a. MERKLERC.Largescaleanalysisofneuralstructures.PaloAlto,California:XeroxPaloAlto ResearchCenter,1989bhttp://www.merkle.com/merkleDir/brainAnalysis.html. MERKLERC.Twotypesofmechanicalreversiblelogic.Nanotechnology,4:114131,1993. MERKLERC.Themolecularrepairofthebrain.Cryonicsmagazine,15,1994. MERKLERCandDREXLERKE.Helicallogic.Nanotechnology,7:325339,1996. METHEO,SPRINGH,GUTTMANNP,etal.Transmissionxraymicroscopyofintacthydrated ptk2cellsduringthecellcycle.JournalofMicroscopyOxford,188:125135,1997. MICHELSONSandCOLEM.Thefutureofpredictivebiosimulationindrugdiscovery.Expert OpiniononDrugDiscovery,2:515523,2007. MICHEVAKDandSMITHSJ.Arraytomography:Anewtoolforimagingthemolecular architectureandultrastructureofneuralcircuits.Neuron,55:2536,2007. MIGLIOREM,CANNIAC,LYTTONWW,MARKRAMH,andHINESML.Parallelnetwork simulationswithneuron.JournalofComputationalNeuroscience,21:119129,2006. MILLERCAandSWEATTJD.CovalentmodificationofDNAregulatesmemoryformation. Neuron,53:857869,2007. MIZUTANIR,TAKEUCHIA,HARAT,UESUGIK,andSUZUKIY.Computedtomographyimaging oftheneuronalstructureofdrosophilabrain.JournalofSynchrotronRadiation,14:282 287,2007. MOLLMandMIIKKULAINENR.Convergencezoneepisodicmemory:Analysisand simulations.NeuralNetworks,10:10171036,1997. MOOREJWandHINESML.Simulationswithneuron,1994. MORAVECH.Whenwillcomputerhardwarematchthehumanbrain?JournalofEvolutionand Technology,1,1998. MORAVECH.Robot:Meremachinetotranscendentmind.NewYork:OxfordUniversityPress, 1999. MOREVECH.Mindchildren:Thefutureofrobotandhumanintelligence:HarvardUniversityPress, 1988. MORRISONA,MEHRINGC,GEISELT,AERTSENA,andDIESMANNMA.Advancingthe boundariesofhighconnectivitynetworksimulationwithdistributedcomputing. NeuralComputation,17:17761801,2005.

123

MULDERSJL,KNOTTG,andLICHBH.Dualbeamslice&view:Practicalaspectsforcollecting3d corteximagedata.Chicago,2006 MULLERTIDOWC,SCHWABLEJ,STEFFENB,etal.Highthroughputanalysisofgenomewide receptortyrosinekinaseexpressioninhumancancersidentifiespotentialnoveldrug targets.ClinicalCancerResearch,10:12411249,2004. MUOTRIARandGAGEFH.Generationofneuronalvariabilityandcomplexity.Nature,441: 10871093,2006. NADALJPandTOULOUSEG.Informationstorageinsparselycodedmemorynets.Network: ComputationinNeuralSystems,1:6174,1990. NADALJP.Associativememoryonthe(puzzling)sparsecodinglimit.JournalofPhysicsa MathematicalandGeneral,24:10931101,1991. NADIMFandMANORY.Theroleofshorttermsynapticdynamicsinmotorcontrol.CurrOpin Neurobiol,10:68390,2000. NANOROADMAPPROJECT.Roadmapsat2015onnanotechnologyapplicationinthesectorsof: Materials,health&medicalsystems,energy.EuropeanCommission,2006 http://www.nanoroadmap.it/. NELLISTPD,CHISHOLMMF,DELLBYN,etal.Directsubangstromimagingofacrystallattice. Science,305:17411741,2004. NEUMANNJV.Thecomputerandthebrain.NewHaven:YaleUniversityPress,1958. NEWMANEAandZAHSKR.Modulationofneuronalactivitybyglialcellsintheretina.Journal ofNeuroscience,18:40224028,1998. NICHOLSONC.Diffusionandrelatedtransportmechanismsinbraintissue.ReportsonProgress inPhysics,64:815884,2001. NORDHAUSWD.Theprogressofcomputing.2001http://ssrn.com/abstract=285168. NUSBAUMMPandBEENHAKKERMP.Asmallsystemsapproachtomotorpatterngeneration. Nature,417:343350,2002. OBERSTH,KOUZNETSOVD,SHIMIZUK,FUJITAJ,andSHIMIZUF.Fresneldiffractionmirrorfor anatomicwave.PhysicalReviewLetters,94:,2005. ORDT.Themanyformsofhypercomputation.AppliedMathematicsandComputation,178:143 153,2006. ORTOLEVAP,BERRYE,BRUNY,etal.Thekaryotephysicochemicalgenomic,proteomic, metaboliccellmodelingsystem.OMICS,7:269283,2003. PAJCINIV,MUNROCH,BORMETTRW,WITKOWSKIRE,andASHERSA.Uvraman microspectroscopy:Spectralandspatialselectivitywithsensitivityandsimplicity. AppliedSpectroscopy,51:8186,1997. PALAYSL,MCGEERUSSELLSM,GORDONJR.S,andGRILLOMA.Fixationofneuraltissuesfor electronmicroscopybyperfusionwithsolutionsofosmiumtetroxide.TheJournalof CellBiology,12,1962. PARFITD.Reasonsandpersons:OxfordUniversityPress,1984. PARPURAV,TONGW,YEUNGES,andHAYDONPG.Laserinducednativefluorescence(linf) imagingofserotonindepletionindepolarizedneurons.JournalofNeuroscience Methods,82:151158,1998. PEARSONK,EKEBERG,andBUSCHGESA.Assessingsensoryfunctioninlocomotorsystems usingneuromechanicalsimulations.TRENDSinNeurosciences,29:625+631,2006. PENCZEKP,MARKOM,BUTTLEK,andFRANKJ.Doubletiltelectrontomography. Ultramicroscopy,60:393410,1995. PENROSER.TheemperorsnewmindNewYork:OxfordUniversityPress,1989. PEREAGandARAQUEA.Propertiesofsynapticallyevokedastrocytecalciumsignalreveal synapticinformationprocessingbyastrocytes.JournalofNeuroscience,25:21922203, 2005. PETERSA.Thalamicinputtothecerebralcortex.TrendsNeurosci.,2:11831185,1979.

124

PETERSAandPALAYSL.Themorphologyofsynapses.JournalofNeurocytology,25:687700, 1996. PFEIFFERJ,JOHNSOND,andNEHRKEK.Oscillatorytransepithelialh+fluxregulatesarhythmic behaviorinc.Elegans.CurrentBiology,18:297302,2008. PICKARDDS,GROVESTR,MEISBURGERWD,CRANET,andPEASERF.Distributedaxiselectron beamtechnologyformasklesslithographyanddefectinspection.JournalofVacuum Science&TechnologyB,21:28342838,2003. PLESSERHE,EPPLERJM,MORRISONA,DIESMANNM,andGEWALTIGMO.Efficientparallel simulationoflargescaleneuronalnetworksonclustersofmultiprocessorcomputers. KermarrecA.M.,BougL.,andPriolT.:672681,2007 PUPPELSGJ,OLMINKHOFJHF,SEGERSNOLTENGMJ,etal.Laserirradiationandraman spectroscopyofsinglelivingcellsandchromosomessampledegradationoccurs with514.5nmbutnotwith660nmlaserlight.ExperimentalCellResearch,195:361367, 1991. RAGHUVANSHINandLINMC.Interactivesoundsynthesisforlargescaleenvironments.Redwood City,California:ACMPress:1011082006 RAGHUVANSHINandLINMC.Physicallybasedsoundsynthesisforlargescalevirtual environments.IeeeComputerGraphicsandApplications,27:1418,2007. RALLW.Theoryphysiologicalpropertiesofdendrites.AnnalsoftheNewYorkAcademyof Sciences,96:1071&,1962. RALSTONTS,MARKSDL,CARNEYPS,andBOPPARTSA.Interferometricsyntheticaperture microscopy.NaturePhysics,3:129134,2007. REHNMandLANSNERA.Sequencememorywithdynamicalsynapses.Neurocomputing,5860: 271278,2004. REUTSKIYS,ROSSONIE,andTIROZZIB.Conductioninbundlesofdemyelinatednervefibers: Computersimulation.BiologicalCybernetics,89:439448,2003. RICEME.Distinctregionaldifferencesindopaminemediatedvolumetransmission.Volume TransmissionRevisited,125:277290,2000. RIEKEF,WARLANDD,DERUYTERVANSTEVENINCKR,andBIALEKW.Spikesexploringthe neuralcode.Cambridge,MA.:MITPress,1996. ROBLEDOL,ELZERMANJ,JUNDTG,etal.Conditionaldynamicsofinteractingquantumdots. Science,320:772775,2008. ROBLESDELATORREG.Theimportanceofthesenseoftouchinvirtualandreal environments.IeeeMultimedia,13:2430,2006. ROSALPandFABERJ.Quantummodelsofthemind:Aretheycompatiblewithenvironment decoherence?PhysicalReviewE,70:,2004. RUECKEST,KIMK,JOSELEVICHE,etal.Carbonnanotubebasednonvolatilerandomaccess memoryformolecularcomputing.Science,289:9497,2000. RUGARD,BUDAKIANR,MAMINHJ,andCHUIBW.Singlespindetectionbymagnetic resonanceforcemicroscopy.Nature,430:32932,2004. RUMELHARTDE,MCCLELLANDJL,andTHEPDPRESEARCHGROUP.Paralleldistributed processing:Explorationsinthemicrostructureofcognition.Cambridge:MITPress,1986. SAKAKURAM,KAJIYAMAS,TSUTSUMIM,etal.Femtosecondpulsedlaserasamicroscalpelfor microdissectionandisolationofspecificsectionsfrombiologicalsamples.Japanese JournalofAppliedPhysicsPart1RegularPapersBriefCommunications&ReviewPapers, 46:58595864,2007. SALIOC,LOSSIL,FERRINIF,andMERIGHIA.Neuropeptidesassynaptictransmitters.Celland TissueResearch,326:583598,2006. SANDIANATIONALLABORATORIES.Onemilliontrillionflopspersecondtargetedbynewinstitute foradvancedarchitectures.2008 http://www.sandia.gov/news/resources/releases/2008/exaflop.html.

125

SANTAMARAPANGA,BILDEATS,COLBERTC,SAGGAUP,andKAKADIARISIA.Towards segmentationofirregulartubularstructuresin3dconfocalmicroscopeimages.In Proc.Ofthemiccaiworkshopinmicroscopicimageanalysisandapplicationsinbiology (miaab),denmark,copenhangen,oct16,2006,2006. SAWCHUKAAandSTRANDTC.Digitalopticalcomputing.ProceedingsoftheIeee,72:758779, 1984. SAXEMD,MALLERETG,VRONSKAYAS,etal.Paradoxicalinfluenceofhippocampal neurogenesisonworkingmemory.PNAS,104464246462007. SCHAFFJ,SLEPCHENKOB,andMORGANF.Virtualcell.2001http://www.nrcam.uchc.edu/. SCHEEPERSF,PARENTRE,CARLSONWE,andMAYSF.Anatomybasedmodelingofthehuman musculature.InOwenG.S.,WhittedT.,andMonesHatta;B.(Eds.),Proceedingsofthe 24thannualconferenceoncomputergraphicsandinteractivetechniques,1997,pp.163172. SCHMITTO,MODERSITZKIJ,HELDMANNS,WIRTZS,andFISCHERB.Imageregistrationof sectionedbrains.InternationalJournalofComputerVision,73:539,2007. SCHNELLSandTURNERTE.Reactionkineticsinintracellularenvironmentswith macromolecularcrowding:Simulationsandratelaws.ProgressinBiophysics& MolecularBiology,85:235260,2004. SCHULZDJ,GOAILLARDJM,andMARDERE.Variablechannelexpressioninidentifiedsingle andelectricallycoupledneuronsindifferentanimals.NatureNeuroscience,9:356362, 2006. SCHULZDJ,GOAILLARDJM,andMARDEREE.Quantitativeexpressionprofilingofidentified neuronsrevealscellspecificconstraintsonhighlyvariablelevelsofgeneexpression. ProceedingsoftheNationalAcademyofSciencesoftheUnitedStatesofAmerica,104:13187 13191,2007. SCHUMANEMandMADISONDV.Locallydistributedsynapticpotentiationinthe hippocampus.Science,263:532536,1994. SEARLEJR.Minds,brains,andprograms.BehavioralandBrainSciences,3:417425,1980. SEITZBTJ.Thegreatgrayravelledknot.2007 http://www.geocities.com/rnseitz/The_Great_Gray_Ravelled_Knot.htm. SENNW,MARKRAMH,andTSODYKSM.Analgorithmformodifyingneurotransmitterrelease probabilitybasedonpreandpostsynapticspiketiming.NeuralComputation,13:35 67,2001. SERRANOL.Syntheticbiology:Promisesandchallenges.MolecularSystemsBiology,3:,2007. SHAOJUANZandHAMMARSTROMD.Simulationofassociativeneuralnetworks.InProceedings ofthe9thinternationalconferenceonneuralinformationprocessing,2002.Iconip02.,2002, pp.16391643. SHEPHERDGM.Thehumansenseofsmell:Arewebetterthanwethink?PlosBiology,2:572 575,2004. SHEPHERDGMG,STEPANYANTSA,BUREAUI,CHKLOVSKIID,andSVOBODAK.Geometricand functionalorganizationofcorticalcircuits.NatureNeuroscience,8:782790,2005. SHIMIZUFandFUJITAJ.Reflectiontypehologramforatoms.PhysicalReviewLetters,88:,2002. SIDIROPOULOUK,PISSADAKIEK,andPOIRAZIP.Insidethebrainofaneuron.EMBORep,7: 88692,2006. SIEGELMANNHTandSONTAGED.Onthecomputationalpowerofneuralnets.Journalof ComputerandSystemSciences,50:132150,1995. SILBERBERGG,GUPTAA,andMARKRAMH.Stereotypyinneocorticalmicrocircuits.TRENDSin Neurosciences,25:227230,2002. SINGLESandBORSTA.Dendriticintegrationanditsroleincomputingimagevelocity.Science, 281:18481850,1998. SPORNSO,TONONIG,andKTTERR.Thehumanconnectome:Astructuraldescriptionofthe humanbrain.PlosComputationalBiology,1:245251,2005.

126

STEFANOVA,GISINN,GUINNARDO,GUINNARDL,andZBINDENH.Opticalquantumrandom numbergenerator.JournalofModernOptics,47:595598,2000. STEPANYANTSA,HOFPR,andCHKLOVSKIIDB.Geometryandstructuralplasticityofsynaptic connectivity.Neuron,34:275288,2002. STROHMAIEREandMEUERHW.Supercomputing:Whathavewelearnedfromthetop500 project?ComputingandVisualizationinScience,6:227230,2004. STROUTJ.Minduploadinghomepage.2007,2006 http://www.ibiblio.org/jstrout/uploading/MUHomePage.html. STUHRMANNB,JAHNKEHG,SCHMIDTM,etal.Versatileopticalmanipulationsystemfor inspection,laserprocessing,andisolationofindividuallivingcells.ReviewofScientific Instruments,77:,2006. STUMPFMPH,THORNET,DESILVAE,etal.Estimatingthesizeofthehumaninteractome. ProceedingsoftheNationalAcademyofSciencesoftheUnitedStatesofAmerica,105:6959 6964,2008. SUZUKIM,GOTOT,TSUJIT,andOHTAKEH.Adynamicbodymodelofthenematodec. Eleganswithneuraloscillators.JournalofRoboticsandMechatronics,17:318326,2005. SWANSONJandKHEIFETSL.Biophysicalmechanisms:Acomponentintheweightofevidence forhealtheffectsofpowerfrequencyelectricandmagneticfields.RadiationResearch, 165:470478,2006. SZABON.Falsifiabledesign:Amethodologyforevaluatingtheoreticaltechnologies.2007 http://unenumerated.blogspot.com/2007/02/falsifiabledesignmethodologyfor.html. TAKAHASHIK,YUGIK,HASHIMOTOK,etal.Computationalchallengesincellsimulation:A softwareengineeringapproach.IeeeIntelligentSystems,17:6471,2002. TANWH,PARPURAV,HAYDONPG,andYEUNGES.Neurotransmitterimaginginlivingcells basedonnativefluorescencedetection.AnalyticalChemistry,67:25752579,1995. TEGMARKM.Importanceofquantumdecoherenceinbrainprocesses.PhysicalReviewE,61: 41944206,2000. TERANJ,SIFAKISE,BLEMKERSS,etal.Creatingandsimulatingskeletalmusclefromthevisible humandataset.IeeeTransactionsonVisualizationandComputerGraphics,11:317328, 2005. THAGARDP.Howmoleculesmattertomentalcomputation.PhilosophyofScience,69:429446, 2002. THOMASS.Endogenousneuroactiveextracellularsignaltransducers.Neurotransmitter.net,2006 http://www.neurotransmitter.net/neurosignaling.html. THOMSONA.Facilitation,augmentationandpotentiationatcentralsynapses.Trends Neuroscience,23:305312,2000. TIMLERJandLENTCS.Maxwellsdemonandquantumdotcellularautomata.Journalof AppliedPhysics,94:10501060,2003. TOLEDORODRIGUEZM,BLUMENFELDB,WUCZ,etal.Correlationmapsallowneuronal electricalpropertiestobepredictedfromsinglecellgeneexpressionprofilesinrat neocortex.CerebralCortex,14:13101327,2004. TOMITAM.Ecell.2001http://www.ecell.org/ecell/. TOUGAWPDandLENTCS.Logicaldevicesimplementedusingquantumcellularautomata. JournalofAppliedPhysics,75:18181825,1994. TRAUBRD,CONTRERASD,CUNNINGHAMMO,etal.Singlecolumnthalamocorticalnetwork modelexhibitinggammaoscillations,sleepspindles,andepileptogenicbursts.Journal ofNeurophysiology,93:21942232,2005. TRAUBRD,MILESR,andBUZSAKIG.Computersimulationofcarbacholdrivenrhythmic populationoscillationsintheca3regionoftheinvitrorathippocampus.Journalof PhysiologyLondon,451:653672,1992.

127

TRAUBRDandWONGRKS.Cellularmechanismofneuronalsynchronizationinepilepsy. Science,216:745747,1982. TSAIPS,FRIEDMANB,IFARRAGUERRIAI,etal.Allopticalhistologyusingultrashortlaser pulses.Neuron,39:2741,2003. TSANGWJ.Synapticultrastructuralreconstructionusingserialelectronmicroscopy.148149,2005 TSENGGYandELLENBOGENJC.Nanotechnologytowardnanocomputers.Science,294:1293 1294,2001. TSODYKSM,PAWELZIKK,andMARKRAMH.Neuralnetworkswithdynamicsynapses.Neural Computation,10:821835,1998. TUSZYNSKIJ.Thedynamicsofcterminiofmicrotubulesindendrites:Apossibleclueforthe roleofneuralcytoskeletoninthefunctioningofthebrain.JournalofGeoethical Nanotechnology,1,2006. TYSONJ.Jigcell.2001http://jigcell.biol.vt.edu/. UEHARAC,COLBERTC,SAGGAUP,andKAKADIARISI.Towardsautomaticreconstructionof dendritemorphologyfromliveneurons.ConfProcIEEEEngMedBiolSoc,3:1798801, 2004. URBANS,OMALLEYSM,WALSHB,etal.Automaticreconstructionofdendritemorphology fromopticalsectionstacks.InProc.2ndinternationalworkshoponcomputervision approachestomedicalimageanalysis,graz,austria,may2006.,2006,pp.190201. VALBERGPA,KAVETR,andRAFFERTYCN.Canlowlevel50/60hzelectricandmagneticfields causebiologicaleffects?RadiationResearch,148:221,1997. VANBRUGGENMJ,VANSOMERENB,andKRUITP.Developmentofamultielectronbeam sourceforsub10nmelectronbeaminduceddeposition.JournalofVacuumScience& TechnologyB,23:28332839,2005. VANGEITW,ACHARDP,andDESCHUTTERE.Neurofitter:Aparametertuningpackagefora widerangeofelectrophysiologicalneuronmodels.Frontiersinneuroinformatics,1:1 18,2007. VANSOMERENB,VANBRUGGENMJ,ZHANGY,HAGEMCW,andKRUITP.Multibeamelectron sourceusingmemselectronopticalcomponents.JournalofPhysicsaMathematicaland General,Conferenceseries34:10921097,2006. VANIERMCandBOWERJM.Acomparativesurveyofautomatedparametersearchmethods forcompartmentalneuralmodels.JournalofComputationalNeuroscience,7:149171, 1999. VENTERJCADAMSMDMYERSEW,etal.Thesequenceofthehumangenome.Science,291: 1304+,2001. VONBOHLENUNDHALBACHOandDERMIETZELR.Neurotransmittersandneuromodulators. Weinheim:WileyVCH,2006. WALTERC.Kryderslaw.ScientificAmerican:3233,2005. WANGY,LIUD,andWANGY.Discoveringthecapacityofhumanmemory.BrainandMind,4: 189198,2003. WANGZHandGRABOWSKIPJ.Cellandstagespecificsplicingeventsresolvedinspecialized neuronsoftheratcerebellum.RnaaPublicationoftheRnaSociety,2:12411253,1996. WEDELBJandGARBERSDL.Guanylylcyclases:Approachingyearthirty.Trendsin EndocrinologyandMetabolism,9:213219,1998. WEHNERM,OLIKERL,andSHALFJ.Towardsultrahighresolutionmodelsofclimateand weather.InternationalJournalofHighPerformanceComputingApplications,22:149165, 2008. WEINSTEINRKandLEERH.Designofhighperformancephysiologicallycomplexmotoneuronmodels infpgas.Arlington,Virginia,2005 WEINSTEINRKandLEERH.Architecturesforhighperformancefpgaimplementationsof neuralmodels.JournalofNeuralEngineering,3:2134,2006.

128

WELSERJJ,BOURIANOFFGI,ZHIRNOVVV,andCAVINRK.Thequestforthenextinformation processingtechnology.JournalofNanoparticleResearch,10:110,2008. WHITEJG,SOUTHGATEE,THOMSONJN,andBRENNERS.Thestructureofthenervoussystem ofthenematodecaenorhabditiselegans.PhilosophicalTransactionsoftheRoyalSociety ofLondonSeriesBBiologicalSciences,314:1340,1986. WILKINSMR,SANCHEZJC,WILLIAMSKL,andHOCHSTRASSERDF.Currentchallengesand futureapplicationsforproteinmapsandposttranslationalvectormapsinproteome projects.Electrophoresis,17:830838,1996. WILSONMA,BHALLAUS,UHLEYJD,andBOWERJM.Genesis:Asystemforsimulatingneural networks.InTouretzkyD.(Ed.)Advancesinneuralinformationprocessingsystems.San Mateo:MorganKaufmann,1989,pp.485492. WISEA,JUPESC,andREESS.Theidentificationofligandsatorphangproteincoupled receptors.AnnualReviewofPharmacologyandToxicology,44:4366,2004. WISHARTDS,TZURD,KNOXC,etal.Hmdb:Thehumanmetabolomedatabase.NucleicAcids Research,35:D521D526,2007. WOODJandGARTHWAITEJ.Modelsofthediffusionalspreadofnitricoxideimplicationsfor neuralnitricoxidesignalinganditspharmacologicalproperties.Neuropharmacology, 33:12351244,1994. XUFandDINGHS.Anewkineticmodelforheterogeneous(orspatiallyconfined)enzymatic catalysis:Contributionsfromthefractalandjamming(overcrowding)effects.Applied CatalysisaGeneral,317:7081,2007. YAMAMOTOYandSHINOHARAK.Applicationofxraymicroscopyinanalysisofliving hydratedcells.TheAnatomicalRecord,269:217223,2002. YOSHIKAWAN,MATSUZAKIF,NAKAJIMAN,andYODAK.Designandcomponenttestofa1bit rsfqmicroprocessor.PhysicaCSuperconductivityandItsApplications,378:14541460, 2002. YUFH,YAROVYAROVOYV,GUTMANGA,andCATTERALLWA.Overviewofmolecular relationshipsinthevoltagegatedionchannelsuperfamily.PharmacologicalReviews, 57:387395,2005. YUCESOYCA,KOOPMANBHFJM,HUIJINGPA,andGROOTENBOERHJ.Threedimensionalfinite elementmodelingofskeletalmuscleusingatwodomainapproach:Linkedfiber matrixmeshmodel.JBiomech,35:12531262,2002. ZENILHandHERNANDEZQUIROZF.Onthepossiblecomputationalpowerofthehuman mind.InGershensonC.,AersD.,andEdmondsB.(Eds.),Worldviews,scienceandus: Philosophyandcomplexity:WorldScientific,2007,pp.315334. ZHANGM,WANGL,andYEP.Allopticalxorlogicgates:Technologiesandexperiment demonstrations.CommunicationsMagazine,IEEE,43:S19S24,2005. ZHANGWX,ZHANGY,ZHENGH,etal.Syndb:Asynapseproteindatabasebasedonsynapse ontology.NucleicAcidsResearch,35:D737D741,2007. ZHAOK,DALTONP,YANGGC,andSCHERERPW.Numericalmodelingofturbulentand laminarairflowandodoranttransportduringsniffinginthehumanandratnose. ChemicalSenses,31:107118,2006. ZHENGY,KREUWELHTC,YOUNGDL,etal.Thevirtualnodmouseapplyingpredictive biosimulationtoresearchintype1diabetes.HowDoWeBestEmployAnimalModelsfor Type1DiabetesandMultipleSclerosis?,1103:4562,2007. ZINOVIEVDY.Designissuesinultrafastultralowpowersuperconductorbatcherbanyan switchingfabricbasedonrsfqlogic/memoryfamily.AppliedSuperconductivity,5:235 239,1997. ZORDANVB,CELLYB,CHIUB,andDILORENZOYPC.Breatheeasy:Modelandcontrolof simulatedrespirationforanimation.InBoulicR.andPaiD.K.(Eds.),Eurographics/acm siggraphsymposiumoncomputeranimation(2004),2004,pp.2937.

129

ZUBEKJPandMACNEILLM.Effectsofimmobilizationbehaviouralandeegchanges.Canadian JournalofPsychology,20:316316,1966. ZUOY,LINA,CHANGP,andGANWB.Developmentoflongtermdendriticspinestabilityin diverseregionsofcerebralcortex.Neuron,46:181189,2005.

130