Home

PostsRSS

CommentsRSS

Edit

Searchhere...

KnowInformatica

DataWareHousingETLProjectLifeCycle
PostedbySureshat1:08AM Labels:DataWarehouse,ETL,LifeCycle

Me

ThefollowingarethedifferentphasesinvolvedinaETLprojectdevelopmentlifecycle.

S UR ES H

1)RequirementGathering

VI EW M Y C OM PL ETE
PROF I L E

a)BusinessRequirementCollection(BRD)
b)SystemRequirementCollection(SRD)
2)DesignPhase
a)HighLevelDesignDocument(HRD)
b)LowlevelDesignDocument(LLD)
3)DevelopmentPhase
a)MappingDesign
b)CodeReview

Labels

c)PeerReview
4)Testing

DataWarehouse(3)

a)UnitTesting
b)SystemIntegrationTesting.

ETL(1)

c)USerAcceptanceTesting(UAT)
5)PreProduction
6)Production(GoLive)

RequirementGathering:

FactTable(1)

LifeCycle(1)

BusinessRequirementCollection(BRD):

SQL(2)

>ThebusinessrequirementgatheringstartbybusinessAnalyst,onsitetechnicalleadandclient
businessusers.
>Inthisphase,aBusinessAnalystpreparesBusinessRequirementDocument(BRD)(or)Business
RequirementSpecifications(BRS)
>BRcollectiontakesplaceatclientlocation.
>Theo/pfromBRAnalysisare

BlogArchive

>BRSBusinessRequirementSpecifications
>SRSSystemRequirementSpecifications

SystemRequirementCollection(SRD):
Seniortechnicalpeople(or)ETLarchitectwillpreparetheSRSwhichcontainss/wandh/wdetails

2011(6)
March(3)

TheSRSwillincludes
a)O/Stobeused(windowsorunix)
b)RDBMSrequiredtobuilddatabase(oracle,Teradataetc)
c)ETLtoolsrequired(Informatica,Datastage)

February(3)

DataWareHousingETLProjectLife

d)OLAPtoolsrequired(Cognos,BO)
TheSRSisalsocalledasTechnicalRequirementSpecifications(TRS)

DesignPhase:

>Theo/pfromdesignandplanningphaseis
a)HLD(HighLevelDesign)Document

PoweredbyBlogger.
FactTable

DataWarehouseDefinition

SearchThisBlog

b)LLD(LowLevelDesign)Document

Search

HLD(HighLevelDesign)Document:
AnETLArchitectandDWHArchitectparticipateindesigningasolutiontobuildaDWH.
AnHLDdocumentispreparedbasedonBusinessRequirement.

LLD(LowLevelDesign)Document:
BasedonHLD,aseniorETLdeveloperprepareLowLevelDesignDocument
TheLLDcontainsmoretechnicaldetailsofanETLSystem.
AnLLDcontainsdataflowdiagram(DFD),detailsofsourceandtargetsofeachmapping.

Resources

AnLLDalsocontainsinformationaboutfullandincrementalload.
AfterLLDthenDevelopmentPhasewillstart

1Keydata

DevelopmentPhase:

Mapping:
>BasedanLLD,theETLteamwillcreatemapping(ETLCode)
>Afterdesigningthemappings,thecode(Mappings)willbereviewedbydevelopers.

Followers

CodeReview:
>CodeReviewwillbedonebydeveloper.

Jointhissite

>Incodereview,thedeveloperwillreviewthecodeandthelogicbutnotthedata.

withGoogleFriendConnect

>Thefollowingactivitiestakesplaceincodereview

Members(2)

>Youhavetocheckthenamingstandardsoftransformation,mappingsofdataetc.
>Sourceandtargetmapping(Placedthecorrectlogicornotinmapping)
PeerReview:
>Thecodewillreviewedbyyourteammember(thirdpartydeveloper)

Testing:

Alreadyamember?Signin

Thefollowingvarioustypestestingcarriedoutintestingenvironment.
1)UnitTesting
2)DevelopmentIntegrationTesting
3)SystemIntegrationTesting

AddSubscribtion

4)UserAcceptanceTesting
UnitTesting:
>AunittestfortheDWHisawhiteBoxtesting,ItshouldchecktheETLprocedureandMappings.
>ThefollowingarethetestcasescanbeexecutedbyanETLdeveloper.
1)Verifydataloss
2)No.ofrecordsinthesourceandtarget
3)Dataload/Insert
4)Dataload/Update
5)Incrementalload
6)Dataaccuracy
7)verifyNamingstandards.
8)VerifycolumnMapping
>TheUnitTestwillbecarriedbyETLdeveloperindevelopmentphase.
>ETLdeveloperhastodothedatavalidationsalsointhisphase.
DevelopmentIntegrationTesting
>Runallthemappingsinthesequenceorder.
>FirstRunthesourcetostagemappings.
>Thenrunthemappingsrelatedtodimensionsandfacts.
SystemIntegrationTesting:
>Afterdevelopmentphase,wehavetomoveourcodetoQAenvironment.
>Inthisenvironment,wearegivingreadonlypermissiontotestingpeople.
>Theywilltestalltheworkflows.
>Andtheywilltestourcodeaccordingtotheirstandards.
UserAcceptanceTesting(UAT):
>Thistestiscarriedoutinthepresenceofclientsidetechnicaluserstoverifythedata
migrationfromsourcetodestination.

ProductionEnvironment:

>MigratethecodeintotheGoLiveenvironmentfromtestenvironment(QAEnvironment).

Thanks:shivakrishnas
2 0C OMME NTS :
Anonymoussaid...
thanksforsuchausefulpost,reallyappreciateit.
JUNE 14 , 2 011 AT 9 : 57 AM
Anonymoussaid...
GoodpostonETLlifecycle.
Mahesh
AUG US T 10, 2 011 AT 10: 4 0 PM
Sandyasaid...

Posts
Comments

Hi,Thislifecycleisreallyveryclearandnice
thankyouforthis.
AUG US T 17, 2 011 AT 9 : 52 PM
Sureshsaid...
ThanksforyourcommentsSandya...
O C TO B E R 14 , 2 011 AT 5: 16 PM
Anonymoussaid...
thanxforpostingithelpsalottome..........
thislifecycleissameforabinitioalso?
plzgivereplaytomemyemailidisrcreddy15.e@gmail.com.......
FE B RUARY 2 2 , 2 012 AT 1: 18 AM
Alekhyasaid...
ThanksforthePosting.....Ithelpsmelot...
Canupleasesaywatarethecomplexissuesusuallyfacewhilemapping....
Canupleasesharetheinformationtomygmailaccountalekhya.rudra@gmail.com
MARC H 4 , 2 012 AT 7: 07 PM
cmtopinkasaid...
What'sthedifferencebetweenUnittestingandDataValidation?
MAY 1, 2 012 AT 12 : 12 PM
Anonymoussaid...
HiAll,
Iamnewininformatica,canyoupleasesharewithmethesampledocumentslikeSRS,FS,HLD,LLD,etcon
mymailidisgajraj.meena@gmail.com
Thanksinadvance.
Gajraj
D E C E MB E R 3 1, 2 013 AT 11: 17 PM
Nitinsaid...
thankYouSoMuch
thispostisreallyveryuseful...
JANUARY 2 1, 2 014 AT 4 : 02 AM
NaveenKumarsaid...
ThankYouSuresh
FE B RUARY 5, 2 014 AT 9 : 04 AM
sruthisaid...
Reallyagreatwork,ithelpedmealot
FE B RUARY 9 , 2 014 AT 11: 2 1 PM
jakegeorgesaid...
ETLTestingOnlineTraining,ONLINETRAININGITSUPPORTCORPORATETRAINING
http://www.21cssindia.com/courses/etltestingonlinetraining249.htmlThe21stCenturySoftwareSolutions
ofIndiaoffersoneoftheLargestconglomerationsofSoftwareTraining,ITSupport,CorporateTraining
instituteinIndia+919000444287+917386622889Visakhapatnam,HyderabadETLTestingOnline
Training,ETLTestingTraining,ETLTesting,ETLTestingOnlineTraining|ETLTestingTraining|ETLTesting|
"Coursesat21stCenturySoftwareSolutions
TalendOnlineTrainingHyperionOnlineTrainingIBMUnicaOnlineTrainingSiteminderOnlineTraining
SharePointOnlineTrainingInformaticaOnlineTrainingSalesForceOnlineTrainingManymore|Call
Us+917386622889+919000444287contact@21cssindia.com
Visit:http://www.21cssindia.com/courses.html"

S E PTE MB E R 3 0, 2 014 AT 9 : 2 4 PM
DanielMasonsaid...
ItwasreallyanicearticleandIwasreallyimpressedbyreadingthisarticleWearealsogivingallsoftware
CourseOnlineTraining.TheDataWarehousingCoursesOnlineTrainingisoneoftheleadingOnlineTraining
instituteintheworld.
D E C E MB E R 2 6 , 2 014 AT 11: 18 AM
jhansiranisaid...
thankuSuresh,..reallyveryusefull..givemeurmailidplz
APRI L 2 , 2 015 AT 10: 10 PM
venkatesankalyanasundaramsaid...
ThankusomuchMrSureshforthewonderfularticle.
canusendthesampledocumentforBRD,SRD,HLD,LLDforreference(ssaivenkat89@gmail.com)

APRI L 6 , 2 015 AT 5: 08 AM
Bapisaid...
ThankyouSuresh,foralltheseValuableinfo.
Canyouplzzprovidemetherelevantmatrialsanddocumentstomyfollowingemailid:
bapisahoo194@gmail.com
JULY 6 , 2 015 AT 12 : 2 1 AM
CatMindssaid...
Thisinformationwhichyouprovidedisverymuchusefulforus.Itwasveryinterestingandusefulforqaonline
training.WealsoprovidingETLTestingOnlineTraininginstituteinUSA.
AUG US T 10, 2 015 AT 11: 54 AM
rajeshkannayadhavansaid...
I'mhappytojoiningthisblog,Itisaveryniceexperienceforme...Thankyouforallyourpostings.
TestingTrainingwithLiveProject
AUG US T 2 8, 2 015 AT 9 : 4 5 PM
Anonymoussaid...
HiSuresh,AsamnewfortheINFORMATICA,itwillbeveryhelpfulforme,thankyouforthisinformative
blog.AndplscanyousharethesampledocumentsofBRD,SRD,etc...wouldbesothankfullforthat.thankyou..
O C TO B E R 1, 2 015 AT 2 : 11 AM
Unknownsaid...
hiplsprovidedocumentsforhldandlld
ramesh.lokonda@gmail.com
D E C E MB E R 2 6 , 2 015 AT 6 : 4 7 PM

POS TAC OMME NT

Enteryourcomment...

Commentas:

Publish

GoogleAccount

Preview

LINK S TOTHIS POS T

CreateaLink
NewerPost

Home

Subscribeto:PostComments(Atom)

KnowInformatica
BloggerthemebyBloggerThemesDesignbyFreeThemeLayouts

OlderPost