Escolar Documentos
Profissional Documentos
Cultura Documentos
www.snaplogic.com
Table of Contents
SnapLogic Tutorials Table of Contents SnapLogic Overview Glossary The Tutorial Environment About the Tutorial Exercise 1: Working with Components Reviewing the Sample Leads to Prospects Pipeline Hands-on Exercise: Creating Components Creating a Writer Component Building the Read-Write Pipeline Exercise 2: Filtering Data Reviewing the California Prospects Pipeline Hands-on Exercise: Creating a Filter Component Duplicating Components Building the Read-Write-Filter Pipeline Exercise 3: Data Services Reviewing the Census Feed and Qualified California Pipelines Hands-On Exercise: Data Services Creating a Join Component Duplicating a Writer Component Building a Data Service Pipeline Exercise 4: Data Sorting Reviewing the Filtered Qualifying California Prospects Pipeline Hands-on Exercise: Sorting Data Exercise 5: Utilizing Pass-Through Reviewing the California Prospects Pipeline, Exercise 5 Hands-on Exercise: Implementing Pass-Through 1 2 4 4 4 5 6 6 8 11 12 16 16 17 19 20 22 22 25 27 28 29 32 32 34 38 38 39
-2-
SnapLogic Tutorials
-3-
1
SnapLogic Overview
SnapLogicistheonlyintegrationplatformbuiltonmodernwebstandardsand"containerized" Snaps,allowingyoutoeasilyconnectanycombinationofCloud,SaaSorOn-premiseapplicationsanddatasources.Likethewebitself,SnapLogicstandardizestheaccessprotocols, accessmethodsanddatastructureforeveryinteraction.TheseRESTful1interactionsallow datatobeexchangedbetweenserversinanorganizedwaythatmakesiteasyforthedatato beprocessedandinterpreted.Thiscreatesaresource-oriented,data-serviceslayerthatprovidestransformationandintegrationservicesforallkindsofintegrations.
Glossary
l Components:Componentsprovidebasicintegrationcapabilitiessuchasdatabasequeries,filereadandwrite,aggregate,sort,filter,join,andothers.Componentsareconfiguredwithspecificproperties. l Pipelines:Pipelinescanbeassembledintohierarchiesthatcanimplementcomplex logicandtransformationsondataforsophisticatedintegrations. l Repository:ResourcedefinitionsarestoredlocallyinaSnapLogicServersRepository.
-4-
SnapLogic Tutorials
-5-
2
Exercise 1: Working with Components
Exercise1readsintheleads.csvfileandwritesoutafilewhichreorderstheFirstNameand LastNamefieldsandleavesoutthehomeandmobilephonenumbers. Toaccomplishthis,Exercise1constructsaPipelinewithtwoComponents:AfileReaderanda fileWriter.Theoutputfilewillbenamed:ex_1_prospects.csv. Note: TheTutorialPipelinesgeneratethesampleoutputfilesex_n_prospects.csv,(forexample,ex_1_prospects.csv,ex_2_prospects.csv).When yourunyourexercises,youwilloverwritethesefiles.Toavoidthis,youcan renameyourexerciseoutputfilestobeunique.Oryoucansimplyre-runthesamplepipelinestorecreatetheoutputfilesifyouneedtoexaminethem. YourversionofSnapLogicincludesanexampleforExercise1thatillustratesallthestepsyou needtotaketocreatethenecessaryPipeline. FirstletslookattheExercise1Components, thenwellcreateitourselves.
-6-
SnapLogic Tutorials
l TheOutputtabdefinestheinput.csvfilefields,datatypes,anddescriptions. l TheParameterstabassignsdefaultvaluestothedelimiterandinputfilenamevariables. l ThePreviewtaballowsyoutoseethedataproducedbytheComponent.OnlyComponentswhichare"Producers",thosewhichhavenoinputviewsandonlyoutputviews, willhavethistabpresent. Clickon theComponentnamedProspectstoviewitsconfiguration. l ThePropertiestabdefinestheoutput .csvfiletobecreated(writtenout)byspecifyingifithasaheaderrow,whatthedelimiteris,andwhatthefilenameandpathis. ThistabalsoprovidesoptionaldescriptiveinformationforthisWriterComponent. l TheInputtabspecifiesthefields(name,datatype,anddescription)thataretoberead fromtheleads.csv file. l TheParameterstabassignsvaluestothedelimiterandinputfilenamevariables. ClickonthePipeline SettingsbuttonintheupperlefthandcornerofthePipeline.Clickthe followingtabs: l TheGeneral InfotabprovidesoptionaldescriptiveinformationforthisPipelineaswell asbuttonstoaccesstheRelatedPipelinesinformationandtheScheduler. l TheParameterstabshowswhatparametersareusedinthePipeline,andwhatthe defaultvaluesshouldbe.Anyparameterwhichdoesnothaveadefaultvalueisconsideredarequiredparameter,whichmustbeprovidedwhenexecutingthePipeline. ThistabpresentsonlytheinformationthataPipelineuserwouldneed. l TheRuntabcontainstheRunbuttonwhichstartsthePipelineprocessing.Thereisalso aparameterstablethatletsyouspecifyparametersatruntimethatmightbedifferent fromthedefaultvaluesindicatedontheParameterstab. Note: ThepurpleandgreenblocksontheLeadsandProspectsComponentsare output/inputviews.Inthiscase,thepurpleblockontheLeadsComponentisan outputviewforthedatathattheComponenthasreadin.Thegreenblockonthe ProspectsComponentisaninputviewforthisWriterComponenttoobtainthe datafromtheReader. ThelinkbetweenComponentsiscreatedbyclickingonapurpleoutputboxatthebottomof oneComponentanddraggingittoagreeninputboxatthetopofanother. ClickanywhereonthelinktodisplaytheFieldlinkingpage. TheFieldLinkerletsyoumapfieldsfromthesourcetothedestination.Eachinputfieldmust belinkedbeforethePipelinecanbeexecuted.Noteveryoutputfieldmustbeusedandyou canuseanoutputfieldmorethanonce.Here,theinputfortheWriterProspectsisbeing linkedtotheoutputfromtheReaderLeads.Therearetwowaystomaketheconnections.For eachinputfieldyoucanselectanoutputfieldfromthelistofvalues.YoucanalsoclickSuggesttohavefieldswiththesamenamesanddatatypesautomaticallylinked.
-7-
-8-
SnapLogic Tutorials
ToverifywhattypeofComponentyouareworkingwith,theComponenttypeisalways displayedintheGeneral InfotabandinthePropertiestabasshowninthepreceding figure. 5. SelecttheGeneral Infotab.ThissectioncontainsfieldsforsettingtheAuthorand DescriptionoftheComponent.Thesepropertiesareinformationalonlyandareoptional. YouwillnotneedtosetanyofthesepropertiesfortheTutorialexercises. 6. SelecttheParameterstab.YoushouldseethetwoparamatersDELIMITERandINPUTFILEadded.Ifnot, clickAdd Rowtwiceandnamethem. IfyouneedhelpconfiguringtheComponent,opentheversionoftheLeadsComponent inthe"Ex1_Leads_to_Prospects"Pipeline. 7. ForINPUTFILE, ,specifytheDefaultValuecolumnastheinputpathandfilename file://tutorial/data/leads.csv.Thispointstoafileinthe/snaplogic/data/tutorial/datadirectoryontheSnapLogicServer. 8. FortheDELIMITERparameter,enter,(comma)asthedefaultvalue. 9. ClickSave.Yourscreenshouldlooklikethefollowingfigure.
-9-
recommendthecorrectdelimiter,quotecharacter,andevenfieldnamesfortheoutput view. 10. ClickSuggest. Adialogopenswiththesuggestedoutputviewandasuggestedquotecharacter. 11. AcceptthesuggestionsbyclickingontheApply All Changesbutton. ClicktheOutputtab.YoushouldseetheviewOutput1withallthefieldnamescorrectly derived.IfyouchoosenottouseSuggest,youcancreatetheoutputviewmanuallyby clickingtheAdd OutputbuttonandclickFinishtoacceptthedefaultviewname(Output1). 12. TheoutputviewoftheLeadsReaderComponentshouldmatchthefieldsinthe leads.csvfile.ClickeachnameField001,Field002,andsoon,entermeaningfulnames, andspecifytheirdatatypeasstring.AllofthefieldsoftheLeadsComponentshouldbe definedasstrings. Keepinmindthatthefieldsinleads.csvcontainthefollowingtypesofinformation: 1. First Name 2. Last Name 3. Home Phone 4. Cell Phone 5. Work Phone 6. Street Address 7. City 8. State 9. Zip Code 13. Asampleoutputdefinitionsscreenisshowninthefollowingfigure.Notethatfield namescannotcontainspaces.Useunderscorecharactersifyouwanttoconcatenate wordssuchasNet ProfitbecomesNet_Profit.
- 10 -
SnapLogic Tutorials
- 11 -
l Wehavespecifiedlastnamefirst.Theorderoftheinputfieldsdefinestheorder inwhichtheywillbewrittenout.Thisallowsustoreorderthefieldsfrom upstreamComponents. l SomeofthefieldnamesdonotmatchexactlywiththoseusedbytheLeadsComponent(forexample,Zipvs.Zip_Code).ThisisintentionalandwillbeusedtoillustratethatexactmatchesarenotnecessarywhenlinkingComponentstogether. Note: Everyfieldyoudefineinaninputviewhastobemappedfrom upstreamviews,sodonotdefinefieldsyoudonotwant.Also,forthese exercises,allfieldshavetobespecifiedusingstringasthedatatype. 14. ClickSave. 15. ClicktheParameterstabandclickAdd Row. 16. IntheNamecolumn,changePARAM_001tobetheoutputfileparameternameweintroducedinthePropertiestab,OUTPUTFILE. 17. IntheDefaultValuecolumn,specifytheoutputfilepathandname,file://tutorial/data/ex_1_prospects.csv. 18. ClickAdd RowandrepeatfortheDELIMITERparameter. 19. ClickSave. TheWriterComponentisnowdefined.
- 12 -
SnapLogic Tutorials
9. ClicktheLeadsoutputviewbox(purplebox)anddragtolinktotheProspectsinputview box(greenbox). Bydefault,theFieldLinkerdialogdisplaysautomaticallywhenever youconnecttwoComponents. 10. ClickApply. 11. Mapthefields.Foreachinputviewfield,selectoneoftheavailableoutputviewfields. YoucanalsoclicktheBbutton,whichwilllinkanyfieldswhosenamesanddatatypes match. Bydefault,SmartLinkwillbeinvokedautomaticallywhenyouconnecttwoComponents together.Again,thisbehaviorcanbecontrolledinthePipelineOptions.Assomefield namesdonotmatch,linktheremainingunlinkedfields. 12. ClickSave. 13. ClickPipeline Propertiesonthecanvastoolbar. 14. ClicktheParameterstab.WearenowgoingtodefinethePipelineparametersandhow theymaptothePipeline'sComponents. WhenyoudragaComponentontothePipelinecanvas,theComponent'sparametersare automaticallymappedasPipelineparameters.Double-clickonaparametertoeditit's nameorvalue.IfyoudonotwishtoexposeaComponent'sparameter,youcandeleteit byselectingtheparameterandclickingtheRemovebutton.Toaddaparameter,you canclicktheAddbuttontoenteranewparameter.Toaddaparameter,specifyaPipelineparameternameandselecttheComponentparametertowhichitmaps,anda defaultvalueifany. 15. Editeachparameterandchangethedefaultparameternames.BydefiningtheDELIMITERparametersasINPUT_DELIMITERandOUTPUT_DELIMITER,ratherthansimply DELIMITERaswasdefinedforbothComponents,wecannowusedifferentdelimitersif weneedto.Inthiscase,wewillleavebothdelimitersasacomma.Youcanalsousinga singlepipelineparameterandmapittomultipleComponentparameters.Forexample, ifyouknowthatallinputfilesusecommaasthedelimiter,youcouldmapasingle parameterFILE_DELIMITERtobothComponentDELIMITERparameters. 16. IntheDefault Valuefield,enterthevaluetobeusedwhenrunningthePipeline. 17. ClickSave. 18. ClicktheRuntab. TheRuntabcontainsthreesub-tabs,Run,PreviewDataandRuntimeInformation.The ParameterstableintheRunsub-tabispopulatedwiththeparameterstobeusedinthis Pipeline.Noticethatthedefaultvaluesappearinthetable. 19. Ifwewantedto,wecouldchangethevaluesrighthereontheRuntabandrunit. 20. ClicktheRunbutton. ThePipelinewillreadinthedataspecifiedintheReaderComponentandwriteittothe filespecifiedintheWriterComponent(ex_1_prospects.csv).ThestatusofthePipe-
- 13 -
line'sexecutionisdisplayedintheRuntimeInformationsub-tab.Youcanaccess detailedinformationbyclickingontherowtoaccesstheShowLogmenu. 21. VerifythatthePipelineperformedthistaskbyopeningtheoutputfileandverifyingthat theFirst,Last,...Cityfieldsdisplayproperly. NoticethatwehaveachievedthegoalofcreatingaprospectsoutputfilefromtheoriginalleadsinputfilewheretheFirstNameandLastNamecolumnshavebeenreordered andwhichcontainsjusttheworkphonenumbersforalltheleads.Alsonoticethatthe inputfilecontains1,000records.Theoutputfilealsocontains1,000records. Note: Iftheex_1_prospects.csvfiledoesnotappearasexpected,openthe Show Logmenufortheruntoseeiftherewasanerror.
- 14 -
SnapLogic Tutorials
- 15 -
3
Exercise 2: Filtering Data
Exercise2duplicatestheComponentsfromExercise1andcreatesanotherPipelinethatfilters theleadstoincludeonlythoseinCalifornia.Toaccomplishthis,theFilterLeadsComponent examinestheStatecolumnintheleads.csvfileforthevalueequaltoCA.Thefilteredoutput iswrittentothefilenamedex_2_prospects.csv. Note: TheTutorialPipelinesgeneratethesampleoutputfilesex_n_prospects.csv,(forexample,ex_1_prospects.csv,ex_2_prospects.csv).When yourunyourexercises,youwilloverwritethesefiles.Toavoidthis,youcan renameyourexerciseoutputfilestobeunique.Oryoucansimplyre-runthesamplepipelinestorecreatetheoutputfilesifyouneedtoexaminethem. YourversionofSnapLogicincludesanexampleforExercise2thatillustratesthestepsto createthenecessaryComponentsandlinkthemwiththePipeline. ThisexercisewillshowyouhowtocreatethesetwoComponents,howtoduplicateandmodify theLeadsandProspectsComponentsfromExercise1,andcombinetheFilterLeadsComponentwiththeLeadsandProspectsComponentsinanewPipelinethatwillfilteroutallthe non-Californialeads. FirstletslookattheExercise2components,thenwellgocreateitourselves.
- 16 -
SnapLogic Tutorials
l TheParameterstabshowswhatparametersareusedinthePipeline,andwhatthe defaultvaluesareifany. l TheRuntabcontainstheRunbuttonwhichstartsthePipelineprocessing.Thereisalso aparameterstablethatletsyouspecifyparametersatruntimethatmightbedifferent fromthedefaultvaluesindicatedontheParameterstab. ThepurpleblockontheLeadsComponentisanoutputviewandthegreenblockontheFilterLeadsComponentisaninputview.ThelinkbetweenthemiscreatedbyclickingonaComponentthatcangenerateoutputanddraggingtoanotherComponent.Inthiscase,youclick LeadstodrawalinktoFilterLeads.YoucannotlinkProspectstoFilterLeadsbecausethe WriterComponentdoesnotprovideoutput. Nowdouble-clicktheFilterLeadsComponent: l ThePropertiestabdefinesthefieldtobeusedtofilterdata,itsvalue,andtheconditiontobemet.Inthisexample,tofindalltheCalifornialeads,weneedtospecifythat theColumnbeState,thevaluebeCA,andtheconditionbe==. l TheInputtabdefinesthestructureofthedatathatisprovidedbytheupstreamComponentsintotheFilter. l TheOutputtabdefinesthestructureofthedatafromtheFilterLeadsComponent.In thisexercise,Filteroutputviewshouldmatchitsinputview. ClickthelinkbetweentheLeadsComponentandtheFilterLeadsComponenttodisplaythe FieldLinker. ThefieldscominginfromtheReaderLeadsaremappedheretofieldsintheFilterLeadsinput viewbyclickingonthefieldsintheIncomingfields(availabletomap)panelanddragging themtotheFrompanel.ThenthesearematchedtotheFilterLeadsfieldsbydragging UnmappedoutgoingfieldstothecorrespondingTopanelsuchthatFromfieldsareonthe samelineasthefieldstherearegoingTo. ClickanywhereonthelinkbetweentheFilterLeadsComponenttotheProspectsComponentto displaytheLinkmenuandselectField LinkingtodisplaytheFieldLinkingpage. ThefieldscominginfromtheFilterLeadsaremappedheretofieldsintheWriterProspectsby clickingonthefieldsintheIncomingfields(availabletomap)panelanddraggingthemtothe Frompanel.ThenthesearematchedtotheWriterfieldsbydraggingUnmappedoutgoing fieldstothecorrespondingTopanelsuchthatFromfieldsareonthesamelineasthefields therearegoingTo. WhenyouclickRun,thePipelineprocessestheReaderoutput,passesitthroughtheFilterand intotheWriterinput,thencreatestheex_2_prospects.csvfile.Youshouldseethattheoutputresultcontains675records. NowletscreatetheComponentsinExercise2.
- 17 -
3. ClickOK. 4. SelecttheInputtab,clicktheAdd Inputbutton,andclickNextontheView Name pop-uptostarttheviewcreationwizard.Hereyouwillbeabletochoosebetweencreatinganewview,justlikewedidinExercise1,orcopyinganexistingviewfrom anotherComponent. 5. SelecttheCopy an Existing viewoptionandthenclickNext. 6. NavigatetotheExercise1LeadsComponentyoucreatedinthefirsttutorial.Whenyou selectaComponent,thewizardwilldisplayalistofanyinputoroutputviewsithas. Whenyouplaceyourmouseoveraview,youwillgetabriefsummaryofthefieldsin thatview.SelecttheoutputviewOutput1andclickFinish.Theoutputviewdefinition willbecopiedfromyourLeadsComponent. 7. WedonotrequirethePhone_handPhone_mfieldsinthisComponentsoclickthe"X" besidethoserowstodeletethefieldsfromthisview. 8. ClickSave. 9. ClicktheOutputtab. 10. RepeatthestepsyouperformedtodefinetheinputviewInput1tocreatetheoutput viewOutput1,butinsteadofcopyingtheviewfromtheLeadsComponent,justcopythe inputviewofyourFilterComponent. 11. ClickSave. 12. ClickthePropertiestab. 13. Specifythepropertiestobe: l Field:State l Operator:== l Value:CA l View:Output1 AnyComponentwhichdefinesapropertyasbeinganinputviewfieldnamewillpopulate thelistofvaluesforthatpropertywithvaluesderivedfromanyinputviewitfindsatthe time.Thelistofvalueswillbeemptyifnoinputviewhasbeendefinedyet.Youmust definetheinputviewfirstbeforeattemptingtoenteravaluefortheproperty.This restrictionappliestopropertieswhichreferenceinputoroutputviewnamesorfield names. 14. ClickClose. ThiscreatesaFilterLeadsComponentthatwillsearchtheStatecolumnoftheincoming datastreamandselectallrecordsthathaveCAasthestatename. 15. ClicktheGeneral Infotabandenteranauthorname,anddescriptionthenclickSave. Thesevaluesareoptional.
- 18 -
SnapLogic Tutorials
Duplicating Components
Duplicating a Reader Component
TheLeadsComponentcreatedinyourExercise1folderisthesameaswhatweneedfora Readerinthisexercise.SoletsduplicatethatversionoftheReaderanduseithere. ThisstepdivergesfromthesampleTutorialversionofExercise2inthatthereisnoLeads Componentduplicatedinthesampleversion.InthesampleversionofExercise2,yousimply dragtheLeadsiconfromtheExercise1folderanddropitontheExercise2Pipelinecanvas. Bothmethodsarevalid.However,theinstructionsinthisHands-onsectionassumethatyou duplicatetheLeadsComponent.DuplicatingaComponentallowsyoutomodifyaComponent withoutaffectingthepipelinesthatreferencetheoriginalComponent. YoucanalsoCTRL+Dragitemswithinthetreetocreateduplicates,orusethe"Copy"commandfromthe"Edit"menu. 1. Double-clicktheLeadsComponenticonfromTutorialExercise1toopenit. 2. SelectSave AsfromtheSavemenubutton. 3. FromtheSaveComponentAs...pop-up,specifytheURISnapLogic/User/Exercise_ 2/Leads. 4. ClickOK. TheduplicatedLeadsComponentappearsintheLibrary. SincetheLeadsReaderwillfunctionexactlyasitdidinExercise1(thatis,readinginthe fieldsfromtheleads.csvfile)wecanleaveitsProperties,BasicProperties,Output,and Parametersunchanged.
- 19 -
- 20 -
SnapLogic Tutorials
- 21 -
4
Exercise 3: Data Services
Exercise3startswithalloftheCalifornia-basedleadsandaddsinformationaboutincomelevelsbyzipcode.ThisexerciseintroducestheconceptofaDataServicePipeline.ADataServicePipelineisonethatprovidesafeedofdatathatcanbeusedbyotherPipelinesorother applications.DatafromthesepipelinescanbepresentedindifferentRepresentationswhen usedbyotherapplications.WewillcreateasimpleDataServicePipelinethatprovidesafeed ofincomedatatakenfromthefileCAIncomeByZip.csv.Next,wewillintroduceaJoinComponentthatreadstheincomedatafromtheDataServicefeedandjoinsit(looksup)withthe filtereddatafromtheleadsLeadsandFilterLeadsComponents.Itthenwritestheoutputtoa filenamedex_3_prospects.csv.Thisfilestillincludesalloftherecordsfromex_2_prospects.csv,butthistime,incomeinformationisadded. Note: TheTutorialPipelinesgeneratethesampleoutputfilesex_n_prospects.csv,(forexample,ex_1_prospects.csv,ex_2_prospects.csv).When yourunyourexercises,youwilloverwritethesefiles.Toavoidthis,youcan renameyourexerciseoutputfilestobeunique.Oryoucansimplyre-runthesamplepipelinestorecreatetheoutputfilesifyouneedtoexaminethem. YourversionofSnapLogicincludesanexampleforExercise3thatillustratesallthestepsyou needtotaketocreatethenecessaryComponentsandlinkthemwiththePipeline. FirstletslookattheExercise3components,thenwellgocreatethemourselves.
- 22 -
SnapLogic Tutorials
mark,andbracesindicatetotheComponentthatthevaluesforthesepropertiesare parametersthatwillbesetatruntime(thatis,whenthePipelineprocessesassociated Components). l TheInputtabspecifiesthefields(name,datatype,anddescription)thataretobe writtentotheoutputfile.ThismatchesthefieldsdefinedintheReaderOutputtab. l TheParameterstabdefinesthevaluesofthefiledelimiterandoutputfilename. Double-clickJoinandthenclick: l ThePropertiestabdefinesthejoinexpressionsandthefieldaliasingproperties.It alsoprovidesoptionaldescriptiveinformationforthisWriterComponent. TheJoinExpressionsdialogcontainsthelistofexpressionsusedtojointheinputviews. Ifthereismorethanonesuchexpression,thereisanimplicitANDbetweenthem.A joinexpressiontakesthefollowinglogicalform: (Input1.name == Input2.name). TheAliasesdialogisusedtodisambiguatetheoutputviewfieldnames.Sincetheinput viewsthatarebeingjoinedoftenhavefieldsofthesamename,inordertodistinguish thesefields,anyambiguousoutputfieldsmustbealiased.Afully-qualifiedinputfieldis givenanalias,whichishowthisfieldwillberepresentedintheoutput.Forexample, consideraJoinComponentwith l twoinputviews:Input1:name,zipandInput2:name,company.Thesewillbe joinedbyname,(Input1.name=Input2.name) Andanoutput:name,company,zip. Joinneedstoknowwhichofthesenamesshouldbeusedfortheoutput.ThatiswhatAliasesspecify. l TheInputtabsspecifythefields(name,datatype,anddescription)thataretoberead fromthe leads.csvandCAIncomeByZip.csvfile. l TheOutputtabspecifiesthefields(name,datatype,anddescription)thataretobeoutputfromtheJoinComponent. l TheParameterstabdefinesanyparametersusedbytheComponent.Inthisexample theJoinComponentdoesnotuseanyparameters. Double-clicktheCensusFeedPipelinetoseewhatthePipelineincludes.Clickthefollowing tabs: l TheGeneral InfotabprovidesdescriptiveinformationforthePipeline.Thisdatais optional. l TheInputtabdefinesthestructureofthedatathatupstreamComponentsprovideto thePipeline.Inthisexercise,thePipelinegetsitsdatafromaCSVReaderComponent thatreadsitfromalocalfile.ThatisdifferentfromdatathatentersaPipelinefromone ofitsinputviews.SincethereisnoupstreamComponentfeedingthisPipeline,thereare noinputviews.
- 23 -
l TheOutputtabdefinesdataexitingthePipeline.TheCensusFeedPipelineisdirecting theoutputfromtheCensusComponent'soutputviewOutput001tothePipeline'soutput viewOutput1. l TheParameterstaballowsyoutodefinethePipelineparametersandmaptheparameterstothePipeline'sComponents. l ThePipelinetabshowsthe singleComponent. l TheRuntabcontainstheRunbuttonwhichstartsthePipelineprocessing.Thereisalso aparameterstablethatletsyouspecifyparametersatruntimethatmightbedifferent fromthedefaultvaluesindicatedontheParameterstab. Double-clicktheQual_CA_ProspectsPipelinetoseewhatthePipelineincludes.Clickthefollowingtabs: l TheGeneral InfotabprovidesdescriptiveinformationforthePipeline.Thisdatais optional. l TheInputtab,liketheCensusFeedPipelineabove,definesnoinputviews. l TheOutputtabdefinesdataexitingthePipeline.Inthisexercise,thisPipelinehasa CSVWriterComponentthatwritesthedatatoalocalfile.Sincenodataisexitingthe Pipelinedirectly,ithasnooutputviews. l TheParameterstaballowsyoutodefinethePipelineparametersandmaptheparameterstothePipeline'sComponents. l ThePipelinetabshowshowtheComponentsarelinked. l TheRuntabcontainstheRunbuttonwhichstartsthePipelineprocessing.Thereisalso aparameterstablethatletsyouspecifyparametersatruntimethatmightbedifferent fromthedefaultvaluesindicatedontheParameterstab. ClickthelinkfromLeadstoFilterLeadstodisplaytheLinkmenuandselectFieldLinkingtodisplaytheFieldLinkingpage. ThefieldscominginfromtheLeadsaremappedheretofieldsintheFilterLeadsbyclickingon thenamesintheIncomingfields(availabletomap)panelanddraggingthemtotheFrom panel.ThenthesearematchedtotheFilterLeadsfieldsbydraggingUnmappedoutgoingfields tothecorrespondingTopanelsuchthatFromfieldsareonthesamelineasthecorresponding Tofieldnames. ClickthelinkfromCensusFeedtoJoinandselectFieldLinkingtodisplaytheFieldLinking page. ThefieldscominginfromCensusFeed(theDataServicePipeline)aremappedheretofieldsin Join. WhenyouclickthelinkfromFilterLeads(theFilterComponent)toJoin(theJoinComponent) andselectFieldLinkingtodisplaytheFieldLinkingpage,youseehowthefilteredfieldsfrom FilterLeadsaremappedtotheJoinComponentwiththefieldsinCensus. TounderstandhowJoincombinestheFilterLeadsandCensusfields,leftclickthelinkfrom JointoProspects.SelectField LinkingtodisplaytheFieldLinkingpage.
- 24 -
SnapLogic Tutorials
- 25 -
13. ClickAdd Rowfivetimesandassignfieldnamesanddatatypesasfollows. Field Name Zip LessThan10K MoreThan10K MoreThan25K MoreThan50K Type string number number number number ZipCode Incomelessthan$10Kperyear Incomebetween$10Kand$25K Incomebetween$25Kand$50K Incomegreaterthan$50K Description
Forthepurposesofthisexample,thedatatypeforZipmustbestringforthisexercise toworkcorrectly.Theremainingfieldsmustbenumbertypes. 14. ClickSave. TheCensusReaderisnowdefined.NextwewillcreatetheData ServicePipelineincorporatingtheCensusComponent. 15. ClicktheNew Pipelinebutton. 16. IntheURIfield,enterSnapLogic/User/Exercise_3/CensusFeed. 17. ClickOK. TheCensusFeedPipelinetabshouldopen. 18. DragtheCensusComponentontothePipelinecanvas,namingitCensus. 19. ClickSave. 20. ClickPipeline Properties. 21. IntheOutputtab,clicktheAddbutton. 22. NamethePipelineoutputviewOutput001andmapittoCensus.Output1,thenclickFinish. 23. ClickSave. 24. ClicktheParameterstabandedittheparameterdefinitionstomatchthefollowing: l Pipeline Parameter:INPUT_DELIMITER l Mapped To:Census.DELIMITER l Default Value:, l Pipeline Parameter:CENSUS l Mapped To:Census.INPUTFILE l Default Value:file://tutorial/data/CAIncomeByZip.csv 25. ClickSave. 26. ClicktheRuntab.IntheRunsub-tab,youcanseethatthePipelinehasanoutputview availabletoreadfrom.
- 26 -
SnapLogic Tutorials
27. ClickthePreviewbutton.ThiswillrunthepipelineandsendthePipeline'soutputtothe Previewsub-tab. 28. ClickthePreview Datasub-tabtoseethedataoutputbythedataservice. AnotherwaytodemonstratetheutilityoftheDataServicePipelineistoviewitdirectlyfrom anotherbrowserwindow.OpenanewbrowserwindowandentertheURIasfollows: http://servername:8088/feed/SnapLogic/User/Exercise_3/CensusFeed/Output001?sn.content_type=text/html where servername:8088isreplacedwiththenameandportofyourSnapLogicDataServer. The/feed prefixisnecessarytoindicatetotheserverthataGETtothisURIwilltriggerPipelineexecution.Theviewnamesuffixishowyouselectwhichoutputisbeingrequested.The browser'sHTTPrequestwillstarttheCensusFeedPipelineandreturntheoutputviewdata fromOutput001inasimpleHTMLrepresentation.Formoreinformationonavailablerepresentations,seethesectionentitled"OutputDataRepresentationFormats"intheSnapLogic UserGuide. YoucanalsoswitchtotheGeneral tabandclickontheComponenthyperlinktoaccessalist ofavailablefeeds.Foryourconvenience,theDesignerwilllaunchtheseinanewwindow whenclicked.
- 27 -
11. Enterthefollowingvalues: l IntheFullyQualifiedFieldcolumn,selectInput1.Zip. l IntheOutputFieldcolumn,selectZip. 12. ClickClose. 13. ClickView/EdittosettheJoinExpressionssettings. 14. ClickAdd row. 15. Enterthefollowingvalues: l IntheJoinField1column,selectInput1.Zip. l IntheJoinField2column,selectInput2.Zip. 16. ClickClose. 17. ClickSave. 18. SelecttheGeneral Infotab.Fillintheauthoranddescriptionfieldsasdesiredand clickSave.Thesefieldsareoptional. TherearenoparametersdefinedforthisComponent,soyoucanleaveitunchanged. 19. ClickSave. TheJoinComponentisnowdefined.
- 28 -
SnapLogic Tutorials
7. ClickSave. TheProspectsComponentisnowdefined.
- 29 -
13. Edittheparameterdefinitionsasfollows. Pipeline Parameter LEADS CENSUS PROSPECTS Mapped to Leads.INPUTFILE CensusFeed.CENSUS Default Value file://tutorial/data/leads.csv file://tutorial/data/CAIncomeByZip.csv , , ,
Prospects.OUTPUTFILE file://tutorial/data/ex_3_prospects.csv
14. NotethatwehavedistinguishedtheinputdelimitersasINPUT1_DELIMITER,INPUT2_ DELIMITER,andOUTPUT_DELIMITERtogiveustheoptionofdefiningthemuniquely shouldtheneedarise.Forthisexercise,allthedelimitersarecommas. 15. ClickSave. 16. SelecttheRuntab. 17. SelecttheRunsub-tab. ThePipelineParameterstableisnowpopulatedwiththeparameterstobeusedinthis Pipeline.Noticethatthedefaultvaluesappearinthetable.Ifnecessary,wecould changethevalueshere. 18. ClickRunbutton. ConfirmationthatthePipelineisstartingisprovidedonthestatuslineintheRuntime Informationsub-tab. VerifythatthePipelineperformedthistaskbyopeningtheoutputfileex_3_prospects.csv andverifyingthattheMoreThan50Kfieldhasbeenadded. WehaveaccomplishedthegoalofcreatingaDataServicePipelineandusingitaspartof anotherPipeline,whichtakesthedataintheoriginalleadsinputfile,refiningittoyieldjust theworkphonenumbersforalltheleadsandanewcolumnhasbeenpopulatedwithcensus dataforeachZipcode.(ThisnewcolumnshowsthepercentageofpeopleinaZipCodewho makemorethan$50,000ayear.)
- 30 -
SnapLogic Tutorials
- 31 -
5
Exercise 4: Data Sorting
Exercise4filtersandsortsjoineddatabasedonzipcodeswhere25%ormoreofhouseholds haveanincomegreaterthan$50,000.Theresultsarewrittentoafilenamedex_4_prospects.csv.
- 32 -
SnapLogic Tutorials
l TheParameterstabshowsparametersusedbythisComponentaswellastheiroptional defaultvalues. ClickthePipeline Propertiesbutton. l TheGeneral InfotabprovidesdescriptiveinformationforthisPipeline.Thesefields areoptional. l TheInputtabdefinesthefieldsconsumedbythisPipeline.NoInputViewsareusedby thisPipeline. l TheOutputtabspecifiesthefieldsoutputbythisPipeline.NoOutputViewsareusedby thisPipeline. l TheParameterstabshowsparametersusedbythisComponentaswellastheir optionaldefaultvalues. TheCensus,Leads,FilterLeads,andJoinComponentsandusedinthisPipelineandtheirfield mappingsareidenticaltohowtheyaredefinedinExercise3.Thefollowinginformation focusesonwhathasbeenintroducedinthisexercise. ClickthelinkbetweentheJoinComponentandFilterProspectsComponent.Thisdisplaysthe followingfieldmappings:
NowclickthelinkbetweentheFilterProspectsComponentandSortProspectsComponent.You willseethefollowingfieldmappings:
- 33 -
- 34 -
SnapLogic Tutorials
15. SpecifytheFieldFilterspropertiestobe: l Field:MoreThan50K l Value:0.25 l Comparison:>= l View:Output1 16. ClickClose. 17. ClickSave.
- 35 -
5. Definefieldmappingsasdescribedin"ReviewingtheFilteredQualifyingCaliforniaProspectsPipeline". 6. ClicktheParameterstabandmodifythePipelineParameterandDefaultfieldsfollows: Pipeline Parameter LEADS CENSUS PROSPECTS Mapped to Leads.INPUTFILE Census.INPUTFILE Prospects.OUTPUTFILE Default Value file://tutorial/data/leads.csv file://tutorial/data/CAIncomeByZip.csv file://tutorial/data/ex_4_prospects.csv , ,
OUTPUT_DELIMITER Prospects.DELIMITER , Wehavedistinguishedtheinputdelimitersas`INPUT1_DELIMITER`,`INPUT2_DELIMITER`,and`OUTPUT_DELIMITER`togiveustheoptionofdefiningthemuniquelyshould theneedarise.Forthisexercise,allthedelimitersarecommas. 7. ClickSave. 8. ClicktheRuntab. ThePipelineParameterstableispopulatedwiththeparameterstobeusedbythisPipeline.Noticethatdefaultvaluesappearinthetable.Ifnecessary,thesevaluescouldbe changedrightherebeforerunningthePipeline. 9. ClickRun. ThePipelinewillreadthedataspecifiedintheReaderCensusandLeadsComponents andwriteittothefilespecifiedintheProspectsComponent(ex_4_prospects.csv). Youcanverifytheresultsofthisexercisebyexaminingthat112recordscontainedin theex_4_prospects.csvfileallhaveanpercentagegreaterthan25%inthelastcolumn.
- 36 -
SnapLogic Tutorials
- 37 -
6
Exercise 5: Utilizing Pass-Through
Exercise5isbasedonExercise2andintroducesyoutotheconceptofpass-through.Componentsthatsupportpass-throughallowyoutomaponlythefieldsrequiredbytheComponent.Theremainingfields"pass-through"theComponentandareavailableforlinking downstream.SeethePass-throughsectionintheUserGuideformoreinformation. Note: TheTutorialPipelinesgeneratethesampleoutputfilesex_n_prospects.csv,(forexample,ex_1_prospects.csv,ex_2_prospects.csv).Whenyou runyourexercises,youwilloverwritethesefiles.Toavoidthis,youcanrename yourexerciseoutputfilestobeunique.Oryoucansimplyre-runthesamplepipelinestorecreatetheoutputfilesifyouneedtoexaminethem.
- 38 -
SnapLogic Tutorials
- 39 -
4. ClickOK. TheduplicatedLeadsResourceappearsinSnapLogic > User > Exercise_5. SincetheLeadsReaderwillfunctionexactlyasitdidinExercise2(thatis,readingin thefieldsfromtheleads.csvfile)wecanleaveitsProperties,Outputview,andParametersunchanged. 5. PerformthesamestepsasaboveforduplicatingtheProspectsresourcefromExercise 2.SpecifytheURIintheSaveAsdialogtobeSnapLogic/User/Exercise_ 5/Prospects.
- 40 -
SnapLogic Tutorials
- 41 -
Index
C
Components creating 8 data service 25 filter 17, 34 filter with pass-through 39 join 27 sort 35 writer 11 duplicating 19, 28, 34, 39
E
exercises overview 6, 16, 22, 32, 38
P
Pipelines building 12, 20, 29, 35, 40
T
tutorials environement 4 overview 5
- 42 -
SnapLogic Tutorials
- 43 -