A Tour of the Native API Purpose of this document

ThisdocumentisaimedatprovidingageneralviewofthenativeAPIwhichcomeswith Xenomai.Newcomersshouldfinddesigninformationdescribingthelogicbehindthis interface.ThisdocumentshouldbeausefulcomplementtotheAPIreferencemanual.

What is this API for?

SinceXenomaiisintrinsicallyAPIagnostic,itcanrunvariouskindsofinterface flavours,forinstanceVxWorks,pSOS+,uITRONorVRTXlikeemulators,mimicking thesetraditionalRTOSAPIs,usuallyforportinglegacyapplications.Butyoumaynot necessarilywanttobuildyourbrandnewapplicationoverthoseemulatorsbecause compatibilitywithsuchinterfacesisnotanissue;incontrast,youmayjustwanttousea programminginterfacewhichleveragesallthecapabilitiesoftheunderlyingrealtime core,andmakesfulluseofitshighintegrationlevelwiththeGNU/Linuxenvironment. Tothisend,Xenomaibringstwopossibilities:arealtimeextensionofthestandard POSIXAPI,anditsownAPIofchoiceanyapplicationcanuse,whichissemantically closertotheinterfacesprovidedbytraditional(nonPOSIX)RTOS.Thelatterinterfaceis simplycalledthenativeAPI,todifferentiatefromtheothers,whichhavebeendefined outsideoftheXenomairealm.

API characteristics
SinceXenomaiisaboutprovidingastableanddeveloperfriendlyrealtimesystemina GNU/Linuxenvironment,thenativeAPIfollowstheseguidelines:

AcompactAPI
ThenativeAPIisreasonablycompact,hopefullystillprovidingacomfortable programmingenvironment,inlessthanahundredofdistinctservices.

Orthogonality
Carehasbeentakentoavoidmultiplevariationsofsemanticallyandfunctionallyclose services,thatwouldendupbeingonlydifferentiatedbyvaryingnamesordetailsintheir respectiveprototype. Therationalebehindsuchdesignchoiceisplainsimple:wefeelthattheleveloffreedom broughttotheapplicationdeveloperbyanAPIdoesnotdependonthenumberof availablesystemcalls,butonthecapacitysheisgiventoidentify,pickandcombinethe existingservicesinanonambiguousandreliablefashion.Forthisreason,thoseservices mustbeorthogonal,alwayshaveastraightforwardpurpose,andrelyonrocksolidbasic NativeAPITourRevC03/20/06

mechanismsindependentfromthegeneralflavourorwindowdressingexposedbythe API.

Contextindependence
EvenifthepreferredexecutionenvironmentforXenomaiapplicationsisuserspace context,theremightstillbeafewcaseswhererunningsomeoftherealtimecode embodiedintokernelmodulesisrequired,especiallywithlegacysystemsorverylow endplatformswithunderperformingMMUhardware.Forthisreason,Xenomai'snative APIprovidesthesamesetofrealtimeservicesinaseamlessmannertoapplication regardlessoftheirexecutionspace.Additionally,someapplicationsmayneedrealtime activitiesinbothspacestocooperate,thereforeaspecialcarehasbeentakentoallowthe lattertoworkontheexactsamesetofAPIobjects.

Seamlesstopology
Whatanapparentlycomplexdefinitionforarathersimpleidea!Itjustmeansthatifwe wanttobeabletousethesamesystemcallforperforminganactionwithoutbothering aboutthelocation(e.g.theexecutionspace)ofthetargetobject(e.g.atask,asemaphore etc.),weneedtohidethenittygrittydetailsofhowtoreachsuchobjectintosome opaque,normalizedandshareableobjectidentifier. Tothisend,eachcategoryofservicesinXenomai'snativeAPIdefinesauniform descriptortypetorepresenttheobjectsitmanages,whichcanbeusedeitherinkernelor userspacecontextsindifferently.Forinstance,ataskwillalwaysberepresentedbya RT_TASKdescriptor,amessagequeuebyaRT_QUEUEdescriptor,andasemaphoreby aRT_SEMdescriptor,regardlessoftheexecutionspacefromwhichtheyareused.Such descriptorscanbetransparentlysharedbetweenexecutionspaces,sothatservicescanbe calledforobjectswhichhavebeencreatedfromtheoppositespace;e.g.asemaphore createdbyauserspaceapplicationcanbepostedbyarealtimeinterrupthandlerin kernelspaceandconversely. Then,weneedtoprovideameantoapplicationsforfindingthosedescriptorsona systemwidebasis,preferablyusingafreeformsymbolicnameasthesearchkey,since manipulatingabstractinformationiseasierthanworkingwithcryptictokens.Xenomai's answertothisisaunifiedregistry,whichallowsallcategoriesofrealtimeservicesto indexeachobjecttheycreateonauniquesymbolicsearchkeydefinedbytheapplication, andconversely,providestheapplicationameantoretrievetheunifieddescriptor associatedwithanyregisteredobjectuponrequest. Bythetimethoselinesarewritten,thenativeAPIisnotnetworkdistributable.Thissaid, youwilllikelyseehowthecombinationoftheunifieddescriptorscheme,anextended registryinterfacedwithanetworknameservice,andrealtimenetworklayerssuchas RTNetwillbeabletoprovidewellintegratedandrobustmultinodecapabilitiesto Xenomai'snativeAPI. NativeAPITourRevC03/20/06

NativeAPITourRevC03/20/06

The design of the native API

Architecture
AkeycharacteristicofthenativeAPIstandsinthefactthatitisanoptionalpartofthe Xenomaisystem,justlikeanyotherrealtimewindowdressinginterfacealsoavailable (e.g.VxWorks,pSOS+,VRTXoruITRON).Likeeachoftheseinterfaces,itisbasedon thesamerealtimenanokernel(akathenucleus)underlyingXenomai,whichprovidesa setofgenericRTOSservicesonecanspecializetobuilditsAPIflavourofchoice.The figurebelowillustratessuchlayeredapproach:

syscallinterface

Native POSIX VxWorks pSOS uITRON VRTX

Realtimenucleus HAL Adeos

Userspaceapplications Kernelbasedapplications
Asthefigureaboveshows,thenativeXenomaiinterfacehasnospecialprivileges comparedtoothers,andisentirelybasedonthegenericcore'spublicinterfacefora simplereason:thisistheonlywaytomakesurethatsignificantoptimizationsandbug fixesoccurringinthecoreareimmediatelyinheritedbytheouterAPIs,withoutany additionalrisksofregression.Moreover,theservicesprovidedbytherealtimenucleus endupbeingironedbymultipleclientAPIsexercisingitinvariousways,which increasestheoverallrobustnessofthewholesystem.

CallingthenativeAPI
ThenativeAPIservicesareimplementedbyakernelmodulenamedxeno_nativeinthe standardXenomaidistribution.Theseservicescanbecalledeitherdirectlyfromkernel NativeAPITourRevC03/20/06

spaceusingintermodulefunctioncalls,orfromuserspacecontext,throughaninterface library(libnative.so)issuingthepropersystemcalls,whichendsuprunningtheactual serviceprovidedbythekernelmodule.

Categoriesofservices
Xenomai'snativeAPIdefinessixmajorcategoriesofservices.Eachcategorydefinesa setofsystemcalls,mostofwhichbeingavailableseamlesslyinbothkernelanduser spaceexecutioncontexts.

Taskmanagement.Thiscategorydefinesthesetofservicesrelatedtotask schedulingandgeneralmanagement.Anapplicationneedsthesesystemcallsto createtasksandcontroltheirbehaviour.Allofthesebasicservicesarealways builtinthenativeAPImodule,regardlessofthecurrentconfiguration.Thenative APIusesofanincreasingpriorityscalefortasks,whichiscompatiblewiththe POSIXscalefortheSCHED_FIFOschedulingclasssupportedbyLinux,i.e.1is thelowesteffectiveprioritylevelfornativeXenomaitasks,and99isthehighest. Timingservices.Thiscategorygroupsallsystemcallswhicharerelatedtothe systemtimermanagementandqueries.Thisgroupalsoprovidesforgeneral watchdogtimerscalledAlarms. Synchronizationsupport.Sincetasksneedtosynchronizetheiroperationstowork properly,thenativeAPIprovidesseveralsynchronizationobjectswhichcanbe usedforthispurpose:

Countingsemaphores, Mutexes, Conditionvariables, Eventflaggroups.

Messagingandcommunication.Thiscategoryimplementsseveralwaysof exchangingbulksofdatabetweenrealtimetasks,orbetweenrealtimeactivities inkernelspaceandregularLinuxprocessesinuserspace:

Intertasksynchronousmessagepassingsupport, Messagequeues,whichcanspanthekernel/userspaceboundary, Memoryheaps,whichcanalsospanthekernel/userspaceboundary, Messagepipes,forexchangingdatabetweenrealtimetasksandregular Linuxprocesses.

DeviceI/Ohandling.SincehandlingexternalI/Oisoneoftheleastwelldefined areasinRTOSdesign,thenativeAPIonlyfocusesonprovidingsimple mechanismsfordealingwithinterrupts,andaccessingdeviceI/Omemoryfrom NativeAPITourRevC03/20/06

userspacecontext.Usersseekingacompleteframeworkforbuildingrealtime devicedriversshouldrefertotheRTDMinterfaceavailablewithXenomai instead.

Registrysupport.Thiscategoryofservicesisoneofthecornerstonesofthe nativeAPI,since,asdescribedearlier,itbringsafundamentalfeaturewithrespect toprovidingaseamlessaccesstosystemcallsfromdifferentexecutionspaces.

Categoriesaremadeofoneormoresubsetsofcloselyrelatedservices,andmostofthese subsetsareoptional;inotherwords,theycanbeselectedforinclusionornotinthe nativeAPImodule,dependingonyourconfiguration(e.g.yourapplicationmayneed semaphoresbutnotconditionvariables). Moreover,itispossibletodisabletheentiresupportfortheuserspaceexecutioncontext, intheseldomcaseswhereallrealtimeactivitiesareembodiedintokernelmodules. Themainideahereistoprovideenoughconfigurationflexibilitytoembeddedsetups.

LinkingagainstthenativeAPI
Xenomaiprovidesthexenoconfigscript,whichsendsbackvariousinformationupon request,likethevalidcompilationandlinkflagstopasstotheGCCtoolchainwhen buildingapplicationsforvariousexecutioncontexts(i.e.kernel,userspace,simulation), andthisscriptshouldbequeriedthesamewayforgettingtheproperargumentsinorder tocompileandlinkanapplicationagainstthenativeAPI. Forinstance,atrivialMakefileforcompilingasimpleuserspaceprogramnamed myapp.cwhichusesthenativeAPIwouldlooklikethis:
prefix := $(shell xeno-config --prefix) ifeq ($(prefix),) $(error Please add <xeno-install-path>/bin to your PATH variable) endif CFLAGS := $(shell xeno-config --xeno-cflags) LDFLAGS := -lnative $(shell xeno-config --xeno-ldflags) myapp: myapp.c $(CC) -o $@ $< $(CFLAGS) $(LDFLAGS)

Mixable execution modes

Xenomaiallowsrealtimetasksembodiedintouserspaceprogramstoeitherexecutein theXenomaidomain,orintotheLinuxdomainwithoutincurringthepotential perturbationsinducedbytheasynchronousLinuxkernelactivities,mainlybymeanofthe appropriatemanagementofaninterruptshield.Intheformercase,therealtimetasksare saidtoruninprimaryexecutionmode,whilstinthelattercase,thosetasksaresaidtorun NativeAPITourRevC03/20/06

insecondaryexecutionmode.Byconvention,realtimetasksembodiedintokernel modulesalwaysruninprimarymode,sincethereisnowayforthemtocallregularLinux kernelservicesdirectly. WhenrunningintotheXenomaidomain(i.e.primarymode),arealtimetaskinuser spacestillhasthebenefitofmemoryprotection,butisscheduledbyXenomaidirectly, andnolongerbytheLinuxkernel.Theworstcaseschedulinglatencyofsuchkindoftask isalwaysneartothehardwarelimitsandpredictable,sinceXenomaiisnotboundto synchronizingwiththeLinuxkernelactivityinsuchcontext,andcanpreemptanyregular Linuxactivitywithnodelay.Inthisexecutionmode,thefollowingconsiderationsapply:

WhenarealtimetaskinprimarymodeissuesaregularLinuxsystemcall,itis immediatelyandtransparentlymigratedbyXenomaitotheLinuxdomain,since theoperationcouldnotbefulfilledintotheXenomaidomaindirectly.Atthat point,therealtimetaskreenterstheLinuxkernelattheclosestrescheduling point,andresumesthestandardLinuxsystemcallprocedure.Timewise,thecost ofsuchmigrationobviouslydependsonthegranularityoftheLinuxkernel. Specifically,thelongestpathbetweentwoexecutionsoftherescheduling procedureinsidetheLinuxkerneldefinestheworstcase.Thankstothe continuoustrendofimprovementsofLinux2.6regardingpreemptability,the worstcaselatencyfiguresareconstantlydecreasing.Thisalsoexplainswhy XenomaiisbestusedoverpreemptibleLinuxkernelswhenthereisaneedfor reasonablepredictabilityevenfortaskstemporarilyswitchingtosecondarymode.

WhenrunningintotheLinuxdomain(i.e.secondarymode),arealtimetaskhas accesstoallregularLinuxsystemcalls,however,thefollowingconsiderationsapply:

ThebehaviourofaregularLinuxsystemcallisnotalteredbythefactofbeing invokedfromarealtimecontext.ThisalsomeansthatLinuxsystemcalls designedforfairnessandnotdeterminismwilllikelyhavepredictabilityissuesin thiscontexttoo. WhentheinterruptshieldfeatureisenabledintotheXenomainucleus (CONFIG_XENO_OPT_ISHIELD),realtimetaskscallingregularLinuxservices areprotectedfromunwantedpreemptionbyLinuxinterrupthandlers,untilthey areblockedbyakernelservicewaitingforsomeI/Oeventtooccur(e.g.regular Linuxdevicedrivers).Inthelattercase,anyinterruptdrivenwakeupcouldbe delayeduntilnootherrealtimetasksisrunningintheLinuxspace,sincethe interruptshieldwouldpreventtheinterruptpropagation.Inordertoavoidsuch potentialpriorityinversionsbetweenrealtimetasksaccordingtothisscenario,it isstillpossibletoproviderealtimeI/Odevicedriverswhichwoulddirectlyrun intotheXenomaidomain.Inthelattercase,theinterruptshieldwouldnotapply totheinterruptsdirectedatthisdomainwhichhashigherpriorityintheAdeos pipeline,thusallowingfortheimmediatewakeupofthependingrealtimetask, attheexpenseofnotreusingtheregularLinuxinfrastructureforwritingsuchI/O NativeAPITourRevC03/20/06

devicedrivers.Tothisend,therecommendedwayofdesigningrealtimedevice driversistobasethemontheRTDM(RealTimeDriverModel)infrastructure whichcomeswithXenomai.Anotheroptionwouldsimplyconsistofsuppressing theinterruptshieldingforrealtimetasksinuserspace(seert_task_set_mode() usingtheT_SHIELDmodebit),butpredictabilityofexecutiontimesmightthen beaffectedbyregularLinuxinterrupthandlers.Itshouldbenotedthatonecan globallydisabletheinterruptshieldsupportatbuildtimeforXenomai (CONFIG_XENO_OPT_ISHIELD=n).

WhenarealtimetaskexecutesintotheLinuxdomainonagivenprocessor,the Linuxkernelasawholeinheritstherealtimepriorityofsuchtask,andthus competesfortheCPUresourcebyprioritywithotherrealtimetasksregardlessof thedomain(i.e.LinuxorXenomai)theyhappentobelongto.Inotherwords,the realtimepriorityschemeenforcedbythenativeAPIisconsistentacrossdomains. WhenarealtimetaskexecutinginsecondarymodecallsanativeXenomai service,itmaybeimmediatelyandtransparentlyswitchedbacktoprimarymode, iftheinvokedsystemcallrequiresit.Byconvention,allXenomaiserviceswhich mightblockthecalleralwayscausesuchtransition;inotherwords,realtimetasks blockedonXenomairesourcesarealwaysasleptinprimarymode.

Thefollowingfigureillustratesthemigrationofarealtimetaskacrossdomains dependingontherequestedtypeofservice: Regular Linux service

Linux domain Xenomai domain Syscall migration Xenomai Task execution Interrupt shield

Native API call

ItshouldbenotedthatdomainmigrationsaremadelazilybyXenomaionlywhena pendingsystemcallrequestsit.Inotherwords,arealtimetaskremainsinprimarymode untilitcallsaregularLinuxservice,andconversely,thesecondarymodeisonlyleft wheneveranativeAPIcallrequiresit. NativeAPITourRevC03/20/06

Tosumup,Xenomai'sprimarymodedeliverstruerealtimeperformancesinthelowest microsecondlatencyrange.ButXenomaialsobetsonfutureevolutionsofLinux,like PREEMPT_RT,toimprovethekernel'soverallgranularity,sothatthesecondarymode willstillberealtimeinthedeterministicsense,withpredictablealbeithigherworstcase latencies.

Working with the native API

ConfiguringthenativeAPI
Startingwithversion2.1,Xenomaifollowsasplitsourcemodel,decouplingthekernel spacesupportfromtheuserspacelibrariesusedinaccessingtheformer. ThekernelspacesupportisseenasabuiltinextensionoftheLinuxkernel,andnomore asacollectionofseparateoutoftreemoduleslikewithearlierXenomaiversions.A directbenefitofsuchapproachistheabilitytobuildtheXenomairealtimesubsystem staticallyintothetargetkernel,orasloadablemodulesaswithearlierversions. Therefore,theusualLinuxkernelconfigurationprocesswillbenormallyusedtodefine thevarioussettingsfortheXenomaikernelcomponents. Ontheotherhand,varioususerspacelibrariesandcommandsprovidedbytheXenomai frameworkneedtobebuiltseparatelyfromthekernelsupport. TherearevariouswaystoruntheconfigurationsystemwithXenomai,andthereismore tosettingupacompleteXenomaisystemthanconfiguringthenativeAPI,butwewill onlyhavealookatthestepstodothelatterhere,usingtheGTKbasedgraphical interface.PleaseseetheREADME.INSTALLfilefromtheXenomaidistributiontolearn howtoconfigureXenomaiusingalternateways.Ifyoufeellucky,youcanalsoreferto theREADME.QUICKINSTALLfileforsettingupaconfigurablesystemevenmore quickly. Theprocedureexplainedbelowisthesameforboththe2.4and2.6kernelsupports.

Configuringthekernelspacesupport
XenomaiprovidesarealtimesubsystemseamlesslyintegratedtoLinux,thereforethe firststepistobuilditaspartofthetargetkernel.Tothisend,scripts/preparekernel.shis ashellscriptavailablefromthedistributionwhichsetsupthetargetkernelproperly.This scriptisalsoabletoapplytheAdeospatchtothetargetkernel,whichisneededto supportXenomai'srealtimecapabilities.Thesyntaxisasfollows: $scripts/preparekernel.shlinux=<linuxsrctree>[adeos=<adeospatch>][ arch=<targetarch>] linuxspecifiesthepathofthetargetkernelsourcetree.Suchkerneltreebeing configuredornotmakesnodifferenceandisvalideitherway. NativeAPITourRevC03/20/06

adeosspecifiesthepathoftheAdeospatchtoapplyagainstthekerneltree.This parametercanbeomittedifAdeoshasalreadybeenpatchedin.Inanycase,thescript willnottrytoapplyitagainwheneveraformerpatchisdetected. archtellsthescriptaboutthetargetarchitecture.Ifunspecified,thebuildsystem architectureisdetectedandsuggestedasareasonabledefault. Forinstance,thefollowingcommandwouldpreparetheLinuxx86treefoundat/ usr/src/linux2.6.15ipipeinordertoincludetheXenomaisupport: $cd<xenomaisrctree> $scripts/preparekernel.shlinux=/usr/src/linux2.6.15ipipearch=x86 adeos=ksrc/arch/i386/patches/adeosipipe2.6.15i3861.201.patch ThenextstepissimplytoconfigureXenomai'skernelcomponents,usingtheregular kernelconfigurationinterface. Forinstance,onceyouhaveenteredtheGTKbasedconfigurationtoolissuingthemake gconfigcommandintheLinuxkerneltree,atypicalconfigurationwindowforthereal timesusbsystemshouldlookliketheonebelow:

NativeAPITourRevC03/20/06

Configuringtheuserspacesupport
SinceXenomaiiscomposedofbothkernelcomponentsononehand,andlibrariesand toolsontheotherhand,weneedtobuildthelatter,inordertobeabletoinvokerealtime servicesfromuserspaceapplications. Thecompleteanduptodateproceduretobuildthissupportisavailablefromthe README.INSTALLfileshippedwithXenomai'sdistribution,includingthecross compilationissues,andexamplesforothertargetarchitectures.Thefollowingisan exampleofbuildingsuchsupportnativelyforax86target: Aregularautoconfscriptisprovidedinordertoprepareforbuildingtheuserspace support.Let'ssaythatwewanttobuildXenomaiforaPentiumbasedx86platform,using thenativehosttoolchain;thetypicalstepswouldbeasfollows: $mkdir<xenomaibuildtree>&&cd<xenomaibuildtree> $<xenomaisrctree>/configureenablex86sep $makeinstall Oncethecompilationhascompleted,/usr/xenomaishouldcontaintheuserspace librairiesandheaderfilesonewouldusetobuildapplicationsthatcallXenomai'sreal timesupportinkernelspace.

Managingtasks
ThenativeAPIprovidesasetofmultitaskingmechanismsasitsfirstcategoryofservices. Thebasicprocessobjectperformingactionsisatask,alogicallycompletepathof applicationcode.EachnativeXenomaitaskisanindependentportionoftheoverall applicationcodeembodiedintoaCprocedure,whichexecutesonitsownstackcontext. ThenativeXenomaischedulerensuresthatconcurrenttasksarerunaccordingtooneof thesupportedschedulingpolicies.Currently,thenativeXenomaischedulersupports fixedprioritybasedFIFOandroundrobinpolicies. Tasksarefirstcreatedandleftinasuspendedstateusingrt_task_create()thenstartedby acalltort_task_start().Ashorthandcombiningbothoperationsiscalledrt_task_spawn ().Thereisnofundamentaldifferencebetweencreatingataskforexecutinginkernel spaceoruserspacecontext;actually,theonlyvariationconcernstheFPUoperations supportwhichisalwaysenabledforuserspacetasksregardlessofthecreationmodebits passedtort_task_create(),whilstkernelbasedonescanchoosetoactivateitornot. Asideofcommonoperationslikesuspend/resumeorsleeprequests,ataskcanbemade periodicbyprogrammingitsfirstreleasepointanditsperiodintheprocessortimeline (seert_task_set_periodic()).Combinedwiththehighprecisionoftheoneshottiming mode(seebelow),thisfeaturemakesitverysimpletosustainfastsamplingloops accurately. NativeAPITourRevC03/20/06

Timingissues
Thereistwokeypointstoknowandkeepinmindaboutthetimingsupportexhibitedby thenativeAPI:

Onhardwarearchitecturesthatprovideaoneshotprogrammabletimesource,the systemtimercanoperateeitherinoneshotorperiodicmode.Inoneshotmode, theunderlyinghardwarewillbereprogrammedaftereachclockticksothatthe nextoneoccursaftera(possiblynonconstant)specifiedintervalwithahigher accuracy,attheexpenseofalargeroverheadduetohardwareprogramming duties.Periodicmodeprovidestimingservicesatalowerprogrammingcostwhen theunderlyinghardwareisatrueProgrammableInterruptTimer(andnotasimple decrementer),butattheexpenseofalowerprecisionsincealldelaysarerounded uptotheconstantintervalvalueusedtoprogramthetimerinitially.Theselection oftheproperoperationmodeismadeatbuildtimeusingtheconfigurationswitch namedCONFIG_XENO_OPT_TIMING_PERIODfromthenucleussection.A zerovaluecausesthenativeskintooperateinoneshotmode;otherwise,periodic modeisusedaccordingtothegivendurationoftheclocktick,expressedin nanoseconds.Ifsupportforperiodictiminghasbeendisabledatconfiguration level(i.e.CONFIG_XENO_OPT_TIMING_PERIODICisunset),thetimermode alwaysdefaultstooneshot. NOTE:Xenomaiversionspriorto2.1rc3usedtorequirethetimermodetobeset dynamicallybytheapplication,usingthenowdeprecatedrt_timer_start()system call.Thisstepisnolongerrequired,sincetheXenomainucleusautomatically startsthesystemtimerwhenthenativeskinisloaded,usingtheconfiguration valuedefinedatbuildtime.Conversely,thert_timer_stop()systemcallhasalso beendeprecated.Itisstillpossibletochangethecurrenttimingmodedynamically though,usingthert_timer_set_mode()call.

Dependingonthecurrenttimeroperationmode,alltimedserviceswhichare passedtimeouts,delaysordateswillinterpretsuchvalueseitherasacountof nanosecondsinoneshotmode,orasacountofjiffies(i.e.peiodiocticks)in periodicmode.

Thereisabsolutelynoexceptiontothedescriptionabove,whichevernativeAPIcallis beingconsidered.Thefollowingsnippetillustratestheinitializationofa10Khzsampling loopinauserspaceapplication,overaXenomaitask:

#include <native/task.h> #include <native/timer.h> #define TASK_PRIO 99 /* Highest RT priority */ #define TASK_MODE 0 /* No flags */ #define TASK_STKSZ 0 /* Stack size (use default one) */

NativeAPITourRevC03/20/06

#define TASK_PERIOD 100000 /* 100 usc period */ RT_TASK task_desc; void sampling_task (void *cookie) { unsigned long overruns; int err; /* The task will undergo a 100 usc periodic timeline. */ err = rt_task_set_periodic(NULL,TM_NOW,TASK_PERIOD); ... for (;;) { err = rt_task_wait_period(&overruns); if (err) break; /* Work for the current period */ } } int main (int argc, char *argv[]) { int err; /* Disable paging for this program's memory. */ mlockall(MCL_CURRENT|MCL_FUTURE); /* Create a real-time task */ err = rt_task_create(&task_desc, "MyTaskName", TASK_STKSZ, TASK_PRIO, TASK_MODE); if (!err) /* If successfully created, start the task. */ rt_task_start(&task_desc,&sampling_task,NULL); ... }

Synchronizingtaskoperations
ThenativeAPIprovidesfourdistinctclassesofcommonsynchronizationobjects,namely mutexes,conditionvariables,semaphores,andeventflaggroups:

Semaphorescaneithercountanddispatchresources,orworkinpulsemode, actingasasynchronizationbarrier. Aneventflaggroupisasynchronizationobjectrepresentedbyalongword NativeAPITourRevC03/20/06

structure.Everyavailablebitinsuchwordcanbeusedtomapauserdefined eventflag,taskscaneitherwaitfortoberaisedinaconjunctivemanner(all awaitedeventsmusthavebeenraisedforthependingtasktowakeup),orina disjunctiveway(atleastoneoftheawaitedeventsmusthavebeenraisedforthe pendingtasktowakeup).

Mutexesshouldbeusedtoserializeaccesstosomecriticalportionofcode.They supportthepriorityinheritanceprotocol,sothatpriorityinversionsamong competingtasksaresolveddynamically. ConditionvariablesaresemanticallycompatiblewiththeirPOSIXcounterparts. Whenproperlyassociatedwithmutexes,conditionvariablesallowtaskstowait foruserdefinedeventstobesignaledatomically.Conditionvariablescanbeused tobuildadhocsynchronizationtoolswhentheprecannedonesalreadyprovided bythenativeAPIdonotfitthejob.

Atanytime,ataskpendingonanyofsuchobjectscanbeforciblyreleasedfromitswait byacalltort_task_unblock().Insuchacase,aspecificerrorcode(i.e.EINTR)will alwaysbereturnedtothecaller.Itshouldbenotedthatauserspacetaskpendingon XenomaimanagedresourcesoreventsremainsinterruptiblebyregularLinuxsignals,in whichcasethewaitisaborted,thetaskisswitchedtosecondaryexecutionmodethen EINTRiseventuallyreturnedtothecaller.

Sharingmemoryareas
Sharedmemoryisthefastestintertaskcommunicationmechanism,thereforeitis supportedbythenativeAPIasabuiltinfeaturetofacilitatetheexchangeofinformation betweentaskspossiblybelongingtodifferentexecutionspaces. ThebasicobjectformemorymanagementwithinthenativeAPIisthememoryheap,as createdbythert_heap_create()systemcall.Itsprimarypurposeistoprovidearegionof memoryusablefordynamicallocationinatimeboundedfashion(seert_heap_alloc() andrt_heap_free()).Additionally,thenativeAPIallowstheheapmemorytobevisible frommultipleaddressspacesuponrequestbypassingtheH_SHAREDmodebitat creationtimetort_heap_create().Insuchacase,taskswhichmusthaveaccesstothe samememoryareafromdistinctaddressspacesthanthecreator'sjustneedtobindtothe initialheapthroughacalltort_heap_bind().Thesnippetbelowdescribeshowtosharea 16Kbmemoryheapbetweenakernelmoduleandauserspaceprogram: Fromakernelmoduleinitializationroutine:
#include <native/heap.h> RT_HEAP heap_desc; void *shmem; int init_module (void)

NativeAPITourRevC03/20/06

{ int err; /* Create the heap in kernel space */ err = rt_heap_create(&heap_desc,"MyHeap",16384,H_SHARED); ... /* Get the shared memory address */ err = rt_heap_alloc(&heap_desc,0,TM_NONBLOCK,&shmem); ... }

Fromauserspacecontext:
#include <native/heap.h> RT_HEAP heap_desc; void *shmem; int main (int argc, char *argv[]) { int err; /* Disable paging for this program's memory. */ mlockall(MCL_CURRENT|MCL_FUTURE); /* Bind to the heap created in kernel space */ err = rt_heap_bind(&heap_desc,"MyHeap",TM_NONBLOCK); ... /* Get the shared memory address */ err = rt_heap_alloc(&heap_desc,0,TM_NONBLOCK,&shmem); ... }

Intheexampleabove,noneofthecallingcontextscouldblockonthebindingcall, waitingforthesearchedobjecttobeeventuallycreated,sincethosecontextsarenot underlaidbyrealtimetasks(init_module()andmain()routinesarenotpartofanyreal timecontextsalthoughitispossibletoconvertthelatterusingthert_task_shadow() call),sothisexamplebasicallyillustrateshowtoshareamemoryareafirstdefinedbya kernelmoduleaspartofitsinitializationchores,withanemerginguserspaceprogram whichhasbeenstartedafterwards.However,theuserspaceportioncouldbemovedover arealtimetaskcontextifnecessary,sothatthebindingoperationcouldblockthecaller untilthesharedmemoryheapiscreated.

Exchangingmessages
Asideofsharedmemoryheaps,thereareseveralotherwaystoexchangebulksofdata NativeAPITourRevC03/20/06

betweentasks,withinagivenexecutionspaceorbetweenkernelanduserspacecontexts. ThenativeAPIprovidesthefollowingmessageorientedfeaturestothisend:

Messagequeuescanbecreatedbythert_queue_create()service,andusedby taskstoreceivevariablesizedmessageswhichhavebeenpostedbyothertasksor interruptserviceroutines.Thecurrentimplementationexhibitsazerocopy mechanismfromwithinthekernel,todeliverthequeuedmessages,andalso allowsforbroadcastingasinglemessagetoseveralreaderswithoutdata duplication. Asynchronousmessagepassingextensionisalsoavailableaspartofthetask interface.Itallowsatasktoreceive,throughthert_task_receive()service,a variablesizedmessagesentfromanothertaskusingthert_task_send()service.In ordertoprovideforfullysynchronousoperations,thesendingtaskisblocked untilthereceiverinvokesrt_task_reply()tofinishthetransaction. Messagepipesprovideameanforexchangingvariablesizedmessagesbetween XenomaitasksandstandardLinuxprocessesusingtheregularfilei/ointerface. Unlikethetwopreviousfeatures,messagepipesdonotdeliverrealtime communications,butratherimplementapracticalandefficientdatapathallowing standardLinuxprocessestoexchangedatawithrealtimetasks.Practically,the realtimetaskconnectstoonesideofthepipeusingthert_pipe_create()service, thenexchangemessageswiththeothersideusingthert_pipe_write()and rt_pipe_read()systemcalls.Theothersideofthepipeistypicallyopenedbya standardLinuxprocessthroughtheregularopen(2)callwhichispassedthename oftheassociatedspecialdevicefileforthecommunication(i.e./dev/rtp0to/ dev/rtp31),thenthepipeisreadandwrittenbytheprocessusingtheusualread (2)andwrite(2)services.Althoughtheprimaryuseofthemessagepipesis messageorientedcommunication,byteorientedorstreamingmodeisalso possiblewhensendingdatafromXenomaitaskstostandardLinuxprocesses usingrt_pipe_stream().Thesnippetbelowdescribeshowtocreateapipebetween akernelmoduleusablebyrealtimetasksandastandardLinuxprocess:

Fromakernelmoduleinitializationroutine:
#include <native/pipe.h> RT_PIPE pipe_desc; int init_module (void) { int err; /* Connect the kernel-side of the message pipe to the special device file /dev/rtp7. */ err = rt_pipe_create(&pipe_desc,"MyPipe",7,NULL); ...

NativeAPITourRevC03/20/06

FromaregularLinuxprocess:
#include <native/pipe.h> int pipe_fd; int main (int argc, char *argv[]) { /* Open the Linux side of the pipe. */ pipe_fd = open(/dev/rtp7,O_RDWR); ... /* Write a message to the pipe. */ write(pipe_fd,hello world,11); ... }

Sincethepipeobjecthasbeenregisteredwhencreatedfromkernelspace,wecould alternativelyopentheassociatedspecialdevicefileusingthepipe'ssymbolicnameas exportedbytheregistry,whichwouldendupopening/dev/rtp7,i.e.:

pipe_fd = open(/proc/xenomai/registry/native/pipes/MyPipe,O_RDWR);

Asynchronousnotifications
Xenomaitasksrunninginkernelspacecanbenotifiedofvariouseventsasynchronously uponrequest.Whenapplicable,asynchronousmethodforcollectingthesameeventsis alsoavailabletouserspacetasks.Wecandistinguishfourtypesofasynchronously notifiableeventsinthecontextofthenativeAPI:

Alarmsaregeneralpurposewatchdogs.AnyXenomaitaskmaycreateany numberofalarmsbycallingthert_alarm_create()service,andusethemtoruna userdefinedhandler,afteraspecifieddelayhaselapsed.Sincekernelbased asynchronoushandlerscannotrunuserspacecode,Xenomaitasksinuserspace canstillexplicitelywaitforthenextalarmshotinstead,bycallingthe rt_alarm_wait()service.Inotherwords,asynchronoushandlerexecutionis replacedbyasynchronousalarmserverinuserspace. Interrupts(actualdevicegeneratedeventsorvirtualAdeosinterrupts)canbe associatedauserdefinedhandlerinkernelspaceusingthert_intr_create() service.Sincekernelbasedasynchronoushandlerscannotrunuserspacecode, Xenomaitasksinuserspacecanstillexplicitelywaitforthenextinterrupt occurrenceinstead,bycallingthert_intr_wait()service.Inotherwords, asynchronoushandlerexecutionisreplacedbyasynchronousinterruptserverin NativeAPITourRevC03/20/06

userspace.

Xenomaitaskshaveanadhocmechanismforsending(rt_task_notify())and receiving(rt_task_catch())signalnotifications,inparalleltotheregularone implementedbytheLinuxkernelforitsprocesses.Pendingsignalscausethe installedsignalhandlerinkernelspacetobefiredeachtimetherecipienttask resumesexecution. Undersomecircumstances,itmaybeusefultobenotifiedwheneverataskswitch occurs,orwheneverXenomaitasksarecreatedordeleted.ThenativeAPIexports hooksfromitsschedulingcoreinordertoprovidesuchinformationtokernel spacemodules,throughthert_task_add_hook()interface.

Mixingexecutionspaces
Asyoumayhavealreadynoticed,thenativeAPIprovidesacontextindependentsupport torealtimetasks,regardlessoftheirexecutionspace,eitherinkerneloruserspace context.Sometimes,splittingtheapplicationwithusuallysmallportionsinkernel spaceandtheremainingpartinuserspaceisuseful.Forinstance,arealtimeI/Odriver mightbemoreeasilyimplementedembodiedintoakernelmodule,whilstthedatait collectswouldbebetterprocessedbyauserspaceprogram,stillwithrealtime guarantees. Insuchacase,itisimportanttobeabletosharetherealtimeobjectsbetweenkerneland userspacecontextsseamlessly.Tothisend,allobjectscreatedbythenativeAPIcanbe indexedonafreeformsymbolickeymaintainedbyasystemwideregistry.Conversely, specificobjectscanbesearchedforandsubsequentlymadeavailabletothecallerwhen found,throughaparticularoperationcalledbinding.Thereisabindingsystemcallfor eachclassofregistrableobjectsexportedbythenativeAPI;forinstancert_task_bind() canbeinvokedtobindtaskstothecaller'scontext,rt_queue_bind()forqueues, rt_sem_bind()forsemaphoresandsoon. Usingtheregistryforthepurposeofsharingrealtimeobjectsissimple:creatorsofthose objectsmustpassavalidanduniquenametotheobjectcreationroutine,whichwillin turnbeusedbytheregistrytoindexthenewobjectinaglobaltable.Meanwhile,binding requestsbearingthesamenamemaybeissuedfromotheruserspacecontexts,inorderto getbacktheuniformobjectdescriptor,whichcouldinturnbeusedseamlesslyinall relevantnativeAPIservices.Bindingrequestscanblockwaitingfortheobjecttobe createdifthecallerisarealtimetask,orreturnimmediatelywiththepropererrorcodeif thecallercannotwait,orcompletesuccessfullyiftheobjectisknownfromtheregistry onentry. Forinstance,thefollowingsnippetillustrateshowtosendXenomaisignalstoakernel basedtaskfromauserspaceprogram,andforthat,wefirstneedtogetbacktheuniform descriptorassociatedtothistaskinthecontextoftheprogram:

NativeAPITourRevC03/20/06

Fromakernelmodule,createthetask:
#include <native/task.h> #define TASK_PRIO 99 #define TASK_MODE T_FPU|T_CPU(0) #define TASK_STKSZ 4096 RT_TASK task_desc; void task_body (void *cookie) { for (;;) { /* Task processing loop in kernel space. */ } } int init_module (void) { int err; err = rt_task_create(&task_desc, "MyTaskName", TASK_STKSZ, TASK_PRIO, TASK_MODE); if (!err) rt_task_start(&task_desc,&task_body,NULL); ... } /* Highest RT priority */ /* Uses FPU, bound to CPU #0 */ /* Stack size (in bytes) */

Fromauserspaceprogram,bindtothetaskinkernelspace:
#include <native/task.h> #define SIGNALS (0x1|0x4) RT_TASK task_desc; int main (int argc, char *argv[]) { int err; /* Disable paging for this program's memory. */ mlockall(MCL_CURRENT|MCL_FUTURE);

NativeAPITourRevC03/20/06

/* Bind to the real-time task in kernel space. */ err = rt_task_bind(&task_desc,"MyTaskName",TM_NONBLOCK); if (!err) /* Send this task a couple of signals. */ rt_task_notify(&task_desc,SIGNALS); ... }

Writinguserspacedevicedrivers
Seamlessintegrationofstringentrealtimecapabilitiesinuserspacecontextallowsto implementi/odevicedriversascommonLinuxprograms.Tothisend,thenativeAPI exportsthefollowingfeaturestorealtimeapplicationsembodiedintoregularLinux programs:

Interruptbindingandsynchronizationservices.Therelatedsystemcallsprovidea meantouserspaceprogramstocontrolinterruptchannelscreatedinkernelspace. Forthispurpose,interruptchannelsarenamedobjectsknownfromtheglobal registrywhichsimplyneedtobeboundtothecaller'scontext(rt_intr_bind()), thenusualoperationscanthenbeappliedontheassociatedchannelusingthe returneduniforminterruptdescriptor,suchasenabling(rt_intr_enable())or disabling(rt_intr_disable())it.Itisalsopossibletocreatetheinterruptobject directlyfromauserspacecontext,incasesharingitwithsomekernelspacecode isnotrequired.Additionally,auserspacetaskcanwaitforinterruptoccurrences usingthert_intr_wait()service. I/Oregionmapping.AccessingaregionofI/Omemorymaybeaprerequisiteto communicatewithsomedevices.ThenativeAPIprovidesasystemcalltorequest accesstosuchregions(rt_misc_get_io_region())andconverselyanotheroneto releasethem(rt_misc_put_io_region()).Thesesystemcallsaresimplewrappers totheregularkernelserviceswhichhandlesuchrequests,namelyrequest_region ()andrelease_region().

Thefollowingsnippetillustrateshowtocreateaninterruptservertaskinuserspaceto processinterrupttriggeredbysomeI/Odevice:
#include <native/task.h> #include <native/intr.h> #define #define #define #define IRQ_NUMBER TASK_PRIO TASK_MODE TASK_STKSZ 7 99 0 0 /* /* /* /* Intercept interrupt #7 */ Highest RT priority */ No flags */ Stack size (use default one) */

NativeAPITourRevC03/20/06

RT_INTR intr_desc; RT_TASK server_desc; void irq_server (void *cookie) { for (;;) { /* Wait for the next interrupt on channel #7. */ err = rt_intr_wait(&intr_desc,TM_INFINITE); if (!err) { /* Process the interrupt. */ } } } int main (int argc, char *argv[]) { int err; /* Disable paging for this program's memory. */ mlockall(MCL_CURRENT|MCL_FUTURE); /* ... */ /* Create the interrupt object, asking for automatic re-enabling of the channel after each IRQ. */ err = rt_intr_create(&intr_desc,IRQ_NUMBER,I_AUTOENA); /* ... */ err = rt_task_create(&server_desc, "MyIrqServer", TASK_STKSZ, TASK_PRIO, TASK_MODE); if (!err) rt_task_start(&server_desc,&irq_server,NULL); /* ... */ }

Debuggingapplications
NativeAPITourRevC03/20/06

Fromuserspacecontext,applicationsusingthenativeAPIcanbedebuggedusingGDB. XenomaihasbeendesignedtoworkassymbioticallyaspossiblewiththeLinuxkernel, andassuch,standardptracingandLinuxsignalsupportsarekeptavailableforusewith realtimetasksinuserspacecreatedbythenativeAPI.Internally,thetaskswillbe switchedautomaticallytosecondarymode,i.e.underthecontroloftheLinuxkernel, whenevertheirrespectivecodeissteppedbyGDB,afaultisraised,orabreakpointis otherwisehit.Xenomaiguaranteesthattheywillbeswitchedbackautomaticallyto primarymodeassoonasitisneeded.Fromauser'spointofview,debuggingarealtime applicationwithGDBisnodifferentfromdebugginganyothernonXenomaienabled program. Fromkernelcontext,modulesembeddingrealtimeapplicationsusingthenativeAPI shouldbeinitiallywrittenanddebuggedusingthesimulator,fortheirhardware independentportions,thatis.Then,tediousdebuggingusingthecommonkernelbased techniquescouldbeappliedtofinishupthework.

Tips and tricks

PortingfromRTAI
ProvidingcompleteportingguidelinesfromRTAItoXenomai'snativeAPIisoutsidethe scopeofthisdocument;however,thefollowingtablepresentsthemajorcorrespondences betweenbothprogramminginterfaces: RTAIAPIpackage Tasks Tasklets Semaphores Conditionvariables Mutexes Bits Fifos Messagequeues(POSIXish) Messages Sharedmemory Localmemory(malloc) Userspaceinterrupts Extendedmailboxes XenomainativeAPIpackage Tasks Alarms Semaphores Conditionvariables Mutexes Eventflaggroups Messagepipes Messagequeues Messagepassingsupport (Shared)Heaps (Local)Heaps InterruptAPI Messagequeues NativeAPITourRevC03/20/06

Detectingunwantedswitchestothesecondaryexecutionmode
Sometimes,switchingtothesecondaryexecutionmodeforarealtimetasksothatitcan reentertheLinuxkernelcontextmaybeanunwantedbehaviour,likelyduetoinvokinga Linuxsystemcallinvoluntarily.Inordertotrapthissituation,thenativeAPIallowsreal timetasksinuserspacetorequiretheSIGXCPUsignal(i.e.CPUtimelimitexceeded)to besenttothemeachtimetheyswitchfromtheprimarytothesecondaryexecutionmode. Tousethisfeature,theapplicationneedstoinstallahandlerfortheSIGXCPUsignal (e.g.usingsigaction(2)),thensettheT_WARNSWmodebitforthetasktomonitor,using thert_task_set_mode()systemcall.IfnoSIGXCPUhandlerhasbeenset,thewhole applicationisterminateduponreceiptofthissignal. AsimplewaytoidentifythecodeissuingtheLinuxsystemcallwithinthemonitored applicationconsistsofrunningthelatteroverGDB,thenissuingthebacktrace commandwheneverGDBreportstheSIGXCPUsignalreceipt.Anotherway,whichdoes notrequiretoruntheapplicationoverGDB,consistsofdumpingthecurrentstackframe usingthestandardbacktrace(3)routine(see/usr/include/execinfo.h)fromtheSIGXCPU signalhandler.Thestatementcausingthetransitionshouldappearastheoutmostcall frameinbothcases. Additionally,onecancheckthenumberofmodeswitchesundergonebyanyactivetask bylookingattheMSWfieldavailablefroma/proc/xenomai/statsdump:thenumberof switchesfromsecondarytoprimarymodeisgivenfirst,followedbythenumberof oppositeswitches(e.g.thosemighthavetriggeredtheSIGXCPUsignalwhenever T_WARNSWhasbeensetforthetaskbeingconsidered).

Detectingrunawaytasks
XenomaithreadsunderlyingthetaskscreatedfromthenativeAPIcanbemonitoredfor runawaydetectionbytherealtimenucleus.Inordertoenablethisfeaturefordebugging yourapplication,youwillneedtoenterthekernelconfigurationtool,thenenablethe WatchdogsupportoptionavailablefromXenomai'sNucleusmenu. Whenenabled,thebuiltinwatchdogtimerpreventshardlockupsduetorunawayreal timethreadsconsumingalltheCPUpower,byforciblysuspendingthecurrentrealtime threadwhenevertheLinuxkernelwastotallystarvedfromCPUformorethanfour seconds.Akernelmessageisalsologgedtoreportsuchaction.

Conclusion
IntegratingstringentrealtimefeaturesintoaGPOSsystemlikeGNU/Linuxis fundamentallydifferentfromprovidingastandaloneRTOS:thosefeaturesmustcoexist onthesamehardwarewiththegeneralpurposeLinuxkernelandapplications,whichmay NativeAPITourRevC03/20/06

wellhaveoppositerequirements.Thisfactraisesanumberofstructuralissueswhichgo wellbeyondsolelyimplementingthemereRTOSmachinery. Inthisrespect,Xenomai'sapproachresidesinatightintegrationwiththeGNU/Linux environment,whichkeepsXenomai'sinternalsawareofandcompatiblewithimportant Linuxsemantics,asfaraspredictabilityisnotweakenedbysuchdesigndecision.For instance,nativeXenomaitasksaremappedoverLinuxtaskcontexts,escapingLinux schedulingcontrolwhenrunninginthemoststringentrealtimemode,butarestillableto processregularLinuxevents,likesignals,senttothem. Forthisintegrationtobeefficient,theXenomaiAPIalsoattemptstoaddressthe compactness,orthogonalityandusabilityissues,sothatiteventuallyappearsasanative supportprovidedbytheLinuxkerneltotherealtimeapplicationdesigners.

Useful Links

TheXenomaidocumentationtree: http://download.gna.org/xenomai/documentation/ ThereferencemanualforthenativeAPI: http://download.gna.org/xenomai/documentation/branches/v2.1.x/html/api/ AdetailedexplanationofXenomai'sfundamentalsoverAdeos: http://download.gna.org/xenomai/documentation/branches/v2.1.x/pdf/Lifewith Adeos.pdf

NativeAPITourRevC03/20/06

rpm@xenomai.org

NativeAPITourRevC03/20/06