Master Data On The Fly - MS

SAP COMMUNITY NETWORK scn.sap.
com
2012 SAP AG 1
Master Data on the fly - SAP
Planning and Consolidations,
version for the Microsoft Platform
Applies to:
SAP Planning and Consolidations, version for the Microsoft platform 7.5, and higher.
SAP Planning and Consolidations, version for the Microsoft platform 10, EPM Add-In, SP3 and higher.
For more information, visit the Enterprise Performance Management homepage.
Summary
This paper will describe how to create a SSIS package to automatically create master data based on the
transactional data values. This paper is based on an actual business requirement and has been published
with the permission and consent of the customer.
Authors: Daniel Jacinto and Pervendren Naidoo
Company: SAP Africa and Allied Electronics Corporation Limited (Altron)
Created on: August 1, 2012
Version: 0.02
Author Bio
Daniel Jacinto is a senior technical consultant with SAP South Africa in the Business Analytics division and is
currently involved with a wide range of projects and solutions in SAP Consulting. His current areas of focus
are working with SAP HANA and Planning and Consolidation Solutions
Pervendren Naidoo is a senior technical specialist at Altron Management Services in South Africa. He has
over 3 years experience with BPC MS and SAP Business Objects BI Solutions. He has over 8 years
experience with various Microsoft products and comes from a networking back office background. His
current areas of focus are Business Objects BI and BI Mobile platforms
Master Data on the fly - SAP Planning and Consolidations, version for the Microsoft Platform
SAP COMMUNITY NETWORK scn.sap.com
2012 SAP AG 2
Table of Contents
Executive Summary ............................................................................................................................................ 3
Business Requirements and Summary ........................................................................................................... 3
Technical Overview ......................................................................................................................................... 5
Package Overview .......................................................................................................................................... 5
Process Overview ........................................................................................................................................... 6
Package Information ....................................................................................................................................... 7
Process Flow: Update Master Data process flow ............................................................................................. 12
Reference Material ........................................................................................................................................... 30
Related Content ................................................................................................................................................ 32
Copyright........................................................................................................................................................... 33

2012 SAP AG 3
Executive Summary
This paper will detail how to create master data on the fly in SAP Business Objects Planning and
Consolidations, version for Microsoft. It details how this package was created to meet an actual business
requirement at a particular client. The requirement was to do Human Capital planning based on several
external HR payroll data sources, previously this was done manually and involved several hours of manual
manipulation of external data files to meet this requirement. This package has automated and streamlined
this process.
The package was built using an ETL framework which will automatically add and update the master data of
the Human Capital planning application / model based on the HR transactional data from the different
external data sources from the different subsidiaries.
This document is technical in nature and is assumed that the reader has an understanding of the concepts
and technologies; there has been no attempt to simplify the content of the document.

Some key highlights of this package:
Automate Master Data updates and additions
Automated Data Import of HR transactional data
Automatic updating of Dimension Member workbook
Built in Data Validations

Business Requirements and Summary
Allied Electronics Corporation Limited (Altron) is invested in telecommunications, power electronics and
multimedia and IT. With revenues of over US$2.8 billion, it employs more than 13,000 employees in over 150
subsidiaries. Altron implemented SAP BusinessObjects solutions to consolidate financial information from
its subsidiaries, gaining deeper, more accurate insight enterprise wide and the ability to better manage
assets and risk.
Altron is using HR payroll data from its subsidiaries to perform human capital planning in a Sustainability
Environment/ AppSet. Due to the federated nature of the Altrons business there is no single HR system
throughout the groups subsidiaries. There are several HR systems spanning from ORACLE, SAP and other
3
rd
party HR systems, which represents a major challenge as the structure and the information stored in the
respective subsidiaries HR systems differs and is non-standardized.
In terms of productivity, several hours were lost in simply formatting and ensuring that the import data files
are correct and contained the necessary master data values to ensure that the data import runs without any
error. This brought about a requirement to standardize and automate the way in which the HR data was
brought into SAP Business Objects Planning and Consolidation which would improve productivity and reduce
the amount of manual effort needed to import the data into the system.

An ETL framework was built in order to cater for any new automation or ETL requirements, part of this
framework involved the creation of a staging database in which any ETL related activities would take place in
the staging database and not in the Environment / Appset database. This approach also lessened the
security requirements as access to the Environment / Appset database could be restricted to object access
with limited rights.
The package is a standard SAP BusinessObjects Planning and Consolidation, version for Microsoft Example
IMPORT From SQL package that was modified to meet the business requirements. The core logic of the
package is built in Microsoft SQL Server Stored procedures which get executed from within the SSIS
framework.

2012 SAP AG 4

Figure 1: High level overview of the process

The core principle of the package is that it will import data from a staging database table, in which all of the
subsidiaries have inserted there relevant HR transactional data into. When the package runs, it will import
the data and perform basic checks to see if there are any transactional values that have no corresponding
master data values. If there are no corresponding master data values, it will then automatically update the
dimension master data and attempt to reimport the data with the updated dimension master data.

SAP HR
3
rd
Party
HR
ORACLE
HR
Subsidiary HR Systems

Staging DB
Environment DB
SAP BPC System
2012 SAP AG 5
Technical Overview
This package is built as an SSIS package and is executed from within the SAP BusinessObjects Planning
and Consolidations, version for Microsoft Data Manager Framework. The original package was taken from
the pre-built examples folder and modified accordingly to meet business requirements. It contains standard
BPC tasks which are delivered as part of the installation.
The core logic of the package is built using Microsoft SQL Server Stored Procedures; they perform the
logical checks to see if the transactional data has corresponding master data values. The package was built
on Microsoft SQL Server 2008 R2 on a Windows 2008 R2 platform. It has been tested on SAP
BusinessObjects Planning and Consolidations 7.5, version for Microsoft and BusinessObjects Planning and
Consolidations 10, version for Microsoft platform.

Package Overview
At a high level, the package gets the records from the staging database table in which the HR transactional
data has been inserted into and then checks to see if there are any values that dont have corresponding
master data values. If there are values or new master data records, it will flag the records and automatically
append the dimension with the new values. Once it has updated the dimension it will then re-import the
records into SAP BPC.

Figure 2 displays the package map with all of the defined tasks. All bpc_ tasks are predefined BPC tasks
and perform the activities within the BPC context.

Figure 2: Package Map

2012 SAP AG 6
Process Overview
The package will execute the Execute SQL task sp_BuildData which will identify if there are any new import
records. Once it has identified that there has been new transactional data, it will then check to see if there
are any values that dont have corresponding master data values. Based on the output of the Execute SQL
task sp_BuildData it will branch off the different package flows within the package. The tasks outlined in red
are the Update Dimension or Rebuild Master Data process flow, this process will update the dimension
master data. The tasks outlined in blue as the Import Data process flow, this process will simply import the
data using a defined transformation and conversion file in SAP Business Planning and Consolidations,
version for Microsoft.

Figure 3: Package process flow map

Note: The high level package flow is: First we check to see if there are any transactional values that dont have a
corresponding master data value. If there are any records that dont have corresponding master data, flag them,
and then perform an automated update of the dimension to append the flagged records (new master data values).
Once the dimension is updated with the new values re-import the transactional data.
2012 SAP AG 7
Package Information
This section will detail the process flow of the package. Each task will be highlighted and detailed with
rationale behind the task.

Figure 4: Package process flow map with detailed task information

Task name: sp_BuildData | Task type: [Execute SQL Task]
This Step executes the sp_BuildPayrollIMportData stored procedure with an OUTPUT parameter. The stored
procedure drops the temp tables created during the last package run, then the data is checked to see if there
are new records to be added or if the data will imported without new data. Based on the output parameter it
will then trigger the process flow will be executed within the package.

2012 SAP AG 8

Figure 5: sp_BuildData execute SQL task

Code Block SQL Stored Procedure { sp_BuildPayrollImportData }

USE [SustainabilityStaging]
GO
/****** Object: StoredProcedure [dbo].[sp_BuildPayrollImportData] Script Date:
05/30/2012 12:13:15 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER proc [dbo].[sp_BuildPayrollImportData]
@timeid nvarchar(10),
@rcode int output
as
begin
/*
-- This Section shows the Validation Check Codes used when deciding which route will
be taken in the package
Validation Checks:
Return Code Descriptions and Logic:
0 = Success and no master data needs to be added
1 = Master Data needs to be added and that will trigger the update / add master
data chain of tasks in the data manager package
*/
declare @strSQL nvarchar(MAX)
2012 SAP AG 9

--we set our timeid value accordingly
set @timeid = (select [timeid] from Sustainability.dbo.mbrTime
where [ID] = @timeid)

--This section will check if the temp table exists, if the table does exist then the
table will be dropped.
if object_id('tempDB..##tmpIDnum') is not null
begin
drop table ##tmpIDnum
end

--This Section checks the Staging Database for the tmpMDPayrollImport table. If the
table exists, the table is dropped.
if object_id('tmptblMDPayrollImport') is not null
begin
drop table SustainabilityStaging.dbo.tmptblMDPayrollImport
end

--We build the temp tables with the IDs of all the existing records in the BPC
application Dimension MBR file.
select distinct([id]) into ##tmpIDnum from sustainability.dbo.mbrHC_employees

--we need to flag our invalid records which have no ID or Passport numbers to be
input into the tblImportPayrollInvalidData Table
if exists(select [IDNumber], [PassportNumber] from
SustainabilityStaging.dbo.tblImportPayrollData
where ([IDNumber] is null or LEN([IDNumber]) = 0 )and ([PassportNumber] is null or
LEN([PassportNumber]) = 0 ))
begin
begin transaction trn_MoveInvalidRec
insert into SustainabilityStaging.dbo.tblImportPayrollInvalidData

([FirstName],[LastName],[CompanyName],[PersonnelArea],[PersonnelSubarea],[PersNo],
[Ethnicorigin],[Gender],[BirthDate],[IDNumber],[PassportNumber],[Nationality],[Disabi
lity],[OccupationLevel],[HireDate],[EmployeeSubgroup],[EmploymentIndicator],[Employee
Cnt])
select
[FirstName],[LastName],[CompanyName],[PersonnelArea],[PersonnelSubarea],[PersNo],[Eth
nicorigin],[Gender],[BirthDate],[IDNumber],[PassportNumber],[Nationality],[Disability
],[OccupationLevel],[HireDate],[EmployeeSubgroup],[EmploymentIndicator],[EmployeeCnt]
FROM [SustainabilityStaging].[dbo].[tblImportPayrollData]
where ([IDNumber] is null or LEN([IDNumber]) = 0 )and ([PassportNumber] is
null or LEN([PassportNumber]) = 0 )

commit transaction trn_MoveInvalidRec

delete from [SustainabilityStaging].[dbo].[tblImportPayrollData]
where ([IDNumber] is null or LEN([IDNumber]) = 0 )and ([PassportNumber] is null or
LEN([PassportNumber]) = 0 )
end

--we need to flag our records which are blank for OCCUPATIONLEVEL as this is Master
Data added to the mbr%Dim% table.
if exists(select [OccupationLevel] from
SustainabilityStaging.dbo.tblImportPayrollData
2012 SAP AG 10
where ([OccupationLevel] is null or LEN([OccupationLevel]) = 0 ))
begin
begin transaction trn_MoveInvalidRec
insert into SustainabilityStaging.dbo.tblImportPayrollInvalidData

([FirstName],[LastName],[CompanyName],[PersonnelArea],[PersonnelSubarea],[PersNo],
[Ethnicorigin],[Gender],[BirthDate],[IDNumber],[PassportNumber],[Nationality],[Disabi
lity],[OccupationLevel],[HireDate],[EmployeeSubgroup],[EmploymentIndicator],[Employee
Cnt])
select
[FirstName],[LastName],[CompanyName],[PersonnelArea],[PersonnelSubarea],[PersNo],[Eth
nicorigin],[Gender],[BirthDate],[IDNumber],[PassportNumber],[Nationality],[Disability
],[OccupationLevel],[HireDate],[EmployeeSubgroup],[EmploymentIndicator],[EmployeeCnt]
FROM [SustainabilityStaging].[dbo].[tblImportPayrollData]
where ([OccupationLevel] is null or LEN([OccupationLevel]) = 0 )

commit transaction trn_MoveInvalidRec

delete from [SustainabilityStaging].[dbo].[tblImportPayrollData]
where ([OccupationLevel] is null or LEN([OccupationLevel]) = 0 )

end

--We now do a validation check on the records we received from the source systems to
the records we just created in the temp table above. We build a new
tmptblMDPayrollImport table we dropped earlier so we dont have any duplicates in our
table to be imported. Any new records are then added to the tmptblMDPayrollImport
table.
select * into SustainabilityStaging.dbo.tmptblMDPayrollImport from
SustainabilityStaging.dbo.tblImportPayrollData where
([IDNumber] is null or [IDNumber] not in (select [id] from ##tmpIDnum)) and
([PassportNumber] is null or [PassportNumber] not in (select [id] from
##tmpIDnum))

--A Validation code is then generated. If there are records added to the
tmptblPayrollImport table the Validation code is set to 1. This will also start
the Package on the route definded on the Right shown in the Full Package Landscape.
if (select count(*) from SustainabilityStaging.dbo.tmptblMDPayrollImport) <> 0
begin
--we need to update master data
set @rcode = 1;
end
else

-- If there are no records in the above table then the Left Route in the Pacakge
Landscape is taken.
-- A check to find out if the tmptblTDPayrollImport table exists, if it does the
table is dropped.
-- A dynamic SQL statement(strSQL) is setup to create the tmptblTDPayrollImport
again.
begin
set @strSQL = '--we need to check if the table exists
if not exists (select * from sys.objects where object_id =
object_id(N''[dbo].[tmptblTDPayrollImport]'') and type in (N''U''))
begin
create table [dbo].[tmptblTDPayrollImport](
2012 SAP AG 11
[IDNumber] [varchar](50) null,
[Gender] [varchar](50) null,
[Time] [varchar](8) not null,
[EmployeeCnt] [int] not null,
[CompanyName] [varchar](255) not null,
[OccupationLevel] [varchar](150) not null,
[Ethnicorigin] [varchar] (50) not null
) on [PRIMARY]
end
else
begin
truncate table [dbo].[tmptblTDPayrollImport]
end '

exec(@strSQL)

--We run another SQL String to import the data from the tblImportPayrollData table
into the tmptblTDpayrollImport
if @timeid <> ''
begin
set @strSQL = 'set ansi_nulls off
Insert into [dbo].[tmptblTDPayrollImport]
([IDNumber],[Gender],[Time],[EmployeeCnt],[CompanyName],
[OccupationLevel],[Ethnicorigin])
select (case [IDNumber] when '' '' then [PassportNumber]
when null then [PassportNumber]
when '''' then [PassportNumber]
else [IDNumber] end) as [IDNumber],[Gender],''' + @timeid + ''' as [Time],1 as
[EmployeeCnt],[CompanyName],[OccupationLevel],[Ethnicorigin] from
SustainabilityStaging.dbo.tblImportPayrollData'
exec(@strSQL)
end
set @rcode = 0;
end

end

Based on the value of @rcode it will determine the process flow path. If the @rcode = 1, it will trigger the
update master data process flow. If the @rcode = 0 it will trigger the import data process flow. In order to
achieve this functionality attached to the execute SQL task is an expression constraint which will based on
the output of the stored procedure will determine which process flow to execute. In order to achieve the
different process flows, a conditional constraint was used with an expression.

2012 SAP AG 12
Process Flow: Update Master Data process flow
Based on the output value of the sp_BuildPayrollImportData stored procedure, the return code @rcode =
1, should trigger the Update Master Data process flow, which will the Update Master Data of the
HC_Employees dimension.

Figure 6: Update Master Data process flow

Task name: sp_UpdateLogStatus | Task type: [Execute SQL Task]
This task will insert a record into the tblStatus table, stating that the Update Master Data process flow is
being executed because there is invalid or new master data records. The table tblStatus contains event
records which will aid in troubleshooting and provide a mechanism of auditability.

2012 SAP AG 13
Figure 7: sp_UpdateLogStatus task

Task name: sp_BuildTempMDTables | Task type: [Execute SQL Task]
This task will create the tables needed for the BPC Admin Task which is used to update the dimension. It will
create two tables tbltmpMDHC_Employees and tblShadowMDC_Employees. The table
tbltmpMDHC_Employees will be used for the BPC Admin Task to update the dimension HC_Employees.
The table tblShadowMDHC_Employees will be a backup of the mbrHC_Employees table and in the event
any error, it will rebuild the dimension from the tblShadowMDHC_Employees table.

Figure 7: sp_BuildTempMDTables task

Task name: df_BuildMD | Task type: [Data Flow Task]
This data flow task will execute an OLE command. This is similar to a SQL Cursor in which, it applies an
OLE command to each record in the recordset. You could use a SQL cursor to apply the same logic and
principles, but the OLE command is faster and consumes fewer resources than an SQL Cursor.
This is a data flow task that includes two steps outlined below and is the heart of the Update Master Data
process flow.
2012 SAP AG 14

Figure 7: df_BuildMD dataflow task

In the dataflow task there are two tasks:
src_tmptblMDPayrollImport OLE DB Source
This is the source of the master data. It contains the contents of the mbr<DimensionName> table
inclusive with the newly flagged records. The newly flagged records were flagged in the
sp_BuildPayrollImportData stored procedure.

Figure 7: src_tbltblMDPayrollImport OLE DB Source task

Figure 7 represents the task that connects to the database through an OLEDB Source Connection
which maps to the tmptblMDPayrollImport table in the Staging Database.

sp_BuildPayrollMasterData OLE DB Command
This OLE DB command will execute the sp_BuildPayrollMasterData stored procedure. This stored
procedure will apply the Insert/Update methodology for master data. If the master data value exists,
it will update the record, if it doesnt exist, it will then insert a new value.

Figure 8: spBuildPayrollMasterData OLE DB command parameter mapping

2012 SAP AG 15

Figure 8 displays the column mapping from the OLE DB source [src_tmptblMDPayrollImport] to
the input parameters for the SQL stored procedure [sp_BuildPayrollMasterData]

Figure 9: spBuildPayrollMasterData OLE DB command statement

Figure 9 displays the statement that the OLE DB command sp_BuildPayrollMasterData executes.
It executes the SQL Stored procedure sp_BuildPayrollMasterData

Code Block SQL Stored Procedure { sp_BuildPayrollMasterData }

GO
/****** Object: StoredProcedure [dbo].[sp_BuildPayrollMasterData] Script Date:
05/30/2012 12:13:15 ******/
GO
CREATE PROC [dbo].[sp_BuildPayrollMasterData]
@idnumber nvarchar(255),
@Firstname nvarchar(255),
@LastName nvarchar(255),
@EthnicOrgin nvarchar(255),
@Nationality nvarchar(255),
@Gender nvarchar(255),
@Disability nvarchar(255),
2012 SAP AG 16
@Birthdate nvarchar(255),
@TempPerm nvarchar(255),
@CompanyName nvarchar(255),
@HireDate nvarchar(255),
@OccupationLevel nvarchar(255),
@PersNumber nvarchar(255),
@PassportNumber nvarchar(255)
as
begin
declare @seqid numeric(8,0)

--Set the SEQID(the counter for the amount of members in the dimension) to the last
one found and then one is added for the new record.
set @seqid = (select top 1 seq from Sustainability.dbo.tbltmpMDC_Employees
order by seq desc) + 1

--Data validation is done to check if the records exist, if not the record is added
to the shadow table. This is done in Step 2 above.
if not exists(select [id] from Sustainability.dbo.tbltmpMDHC_Employees where [id] =
@idnumber)

begin
--the user doesnt exists and so the record will need to be inserted
insert into [Sustainability].[dbo].[tbltmpMDC_Employees]
([ID],[NEWID],[EVDESCRIPTION],[PARENTH1],[ACCTYPE],[SCALING],[RATETYPE],[EthnicOrigin
],[Nationality],[Gender],[Disability],[IDNumber],[Birthdate],[TempPerm],[CompanyName]
,[HireDate],[OccupationLevel],[PersNumber],[PassportNumber],[CALC],[DIMCALC],[ISBASEM
EM],[HIR])
values (@idnumber,'',@FirstName + ' ' + @LastName,Unallocated',
'EXP','','',@EthnicOrgin,@Nationality,@Gender,@Disability, @IDnumber,
Birthdate,@TempPerm,@CompanyName,@HireDate,@OccupationLevel,@PersNumber,@PassportNumb
er, 'N','N','Y','H1' )
end

end

Task name: bpc_AdminMakeDim | Task type: [BPC task]
This is a BPC standard task; it is the Admin_Task which is responsible for several BPC actions and
processes. This task will create the dimension from SQL table after the addition or updates of the master
data records. If this process is successful the sp_RebuildData Execute SQL Task is run. If the process fails
the bpc_AdminMakeDimPlanB BPC task is run to rebuild the dimension from the shadow table. This is a
form of a rollback. This logic was done in order to ensure that the dimension is always in a consistent state.

2012 SAP AG 17
Figure 10: bpc_AdminMakeDim BPC task

This task will create the dimension from the tbltmpMDHC_Employees table. One of the limitations of this
task is that it will not see any tables outside of the AppSet / Environment DB. So in the task
sp_BuildTempMDTables it will create the tables in the AppSet / Environment DB in order for the
bpc_AdminMakeDim BPC task to use as a source for creating or building the dimension.
Task name: Replace_HC_Employees | Task type: [File System Task]
The File System Task is used to make a copy of the existing Excel dimension file and replace the file with a
template we created. We do this in order to keep the Dim<DimensionName> and Dimension Excel files in
sync. This is done to prevent the scenario in which the dimension will be updated, but in the event of a user
processing the dimension from the member sheet it will overwrite and cause the dimension to be out of sync.
Figure 11: Replace_HC_Employees File System Task

2012 SAP AG 18
Task name: sp_DumpToExcel | Task type: [Execute SQL Task]
This SQL Task executes a Stored Procedure that uses the OPENROWSET command to export all the
contents of the mbr<DimensionName> table to the Excel Template file which was copied by the
Replace_HC_Employees step. This step will ensure that the dimension file and the mbr<DimensionName>
table are in sync. This is done to prevent the scenario in which the dimension will be updated, but in the
event of a user processing the dimension from the member sheet it will overwrite and cause the dimension to
be out of sync.

Figure 11: sp_DumpToExcel Execute SQL task

Code Block SQL Stored Procedure { sp_Excel}

SET ANSI_NULLS ON
GO
GO
CREATE Procedure [dbo].[SP_Excel]
As
Begin
insert into OPENROWSET('Microsoft.ACE.OLEDB.12.0','Excel 12.0 XML;
HDR=YES;

Database=\\<ServerName>\d$\BPC\Data\Webfolders\Sustainability\ADMINAPP\hc_employe
es.xlsx',
'Select
ID,EVDESCRIPTION,NEWID,PARENTH1,ACCTYPE,SCALING,RATETYPE,ETHNICORIGIN,NATIONALITY,GE
2012 SAP AG 19
NDER,DISABILITY,IDNUMBER,

BIRTHDATE,TEMPPERM,COMPANYNAME,HIREDATE,OCCUPATIONLEVEL,PERSNUMBER,PASSPORTNUMBER
,EMPLDATE from [Members$]')

Select
ID,EVDESCRIPTION,NEWID,PARENTH1,ACCTYPE,SCALING,RATETYPE,ETHNICORIGIN,NATIONALITY,GE
NDER,DISABILITY,IDNUMBER,

BIRTHDATE,TEMPPERM,COMPANYNAME,HIREDATE,OCCUPATIONLEVEL,PERSNUMBER,PASSPORTNUMBER
,EMPLDATE from sustainability.dbo.mbrHC_Employees
End
GO

Task name: sp_ReBuildPayrollImportData | Task type: [Execute SQL Task]
This SQL Task executes a Stored Procedure sp_ReBuildPayrollImportData which will populate the
tmptblTDPayrollImport table which will be used to import the transactional data.

Figure 12: sp_ReBuildPayrollImportData Execute SQL task

Code Block SQL Stored Procedure { sp_ReBuildPayrollImportData }

GO
/****** Object: StoredProcedure [dbo].[sp_ReBuildPayrollImportData] Script Date:
05/30/2012 13:25:11 ******/
SET ANSI_NULLS ON
GO
GO
ALTER proc [dbo].[sp_ReBuildPayrollImportData]
@timeid nvarchar(10)
2012 SAP AG 20
as begin
set ANSI_NULLS OFF
--we set our timeid value accordingly
set @timeid = (select [timeid] from Sustainability.dbo.mbrTime
where [ID] = @timeid)
declare @strSQL nvarchar(MAX)

-- We check to see if the tmptblTDPayrollImport still exists, if not then the table
is created. If the table does exist, all records in the table will be truncated.

set @strSQL =
if not exists (select * from sys.objects where object_id =
object_id(N''[dbo].[tmptblTDPayrollImport]'') and type in (N''U''))
begin
Create table [dbo].[tmptblTDPayrollImport](
[IDNumber] [varchar](50) null,
[Gender] [varchar](50) null,
[Time] [varchar](8) not null,
[EmployeeCnt] [int] not null,
[CompanyName] [varchar](255) not null,
[OccupationLevel] [varchar](150) not null,
[Ethnicorigin] [varchar] (50) not null
) on [PRIMARY]
end
else
begin
truncate table [dbo].[tmptblTDPayrollImport]
end '

-- we then reimport the records from the tblImportPayrollData into the newly created
table.

exec(@strSQL)
if @timeid <> ''
Begin
set @strSQL = 'set ansi_nulls off
insert into [dbo].[tmptblTDPayrollImport]
([IDNumber],[Gender],[Time],[EmployeeCnt],[CompanyName], [OccupationLevel],
[Ethnicorigin])
select (case [IDNumber] when null then [PassportNumber]
when '' '' then [PassportNumber]
when '''' then [PassportNumber]
else [IDNumber]end) as [IDNumber],
[Gender],''' + @timeid + ''' as
[Time],1,[CompanyName],[OccupationLevel],[Ethnicorigin] from
SustainabilityStaging.dbo.tblImportPayrollData'
exec(@strSQL)
end

--Execute Dynamic String to change "," character so BPC import does not fail with
"Head Dimension count and data column count are not matched".
Begin
Set @strSQL = 'UPDATE SustainabilityStaging.dbo.tmptblTDPayrollImport
2012 SAP AG 21
SET occupationlevel = REPLACE(LTRIM(RTRIM(occupationlevel)), '','', ''_'')'
exec(@strSQL)
End
end

Rebuild Dimension: {Plan B} process flow
This process will trigger on the event if there is any error on bpc_AdminMakeDim BPC task. In the event of
an error it means that the building of a dimension has failed and has resulted in an error. There is an
OnError constraint and it will trigger the bpc_AdminMakeDimPlanB BPC task which will build the
dimension using the shadow table. This shadow table was created in the step sp_BuildTempMDTables

Figure 13: Rebuild Dimension Process Flow

Note: The values for the bpc_AdminMakeDimPlanB BPC task are defined at runtime. The values are passed through the
dynamic script. Please make reference to the Dynamic Script for additional details regarding the values being
passed to the BPC tasks.

2012 SAP AG 22
Task name: bpc_AdminMakeDimPlanB | Task type: [BPC Task]
This task will rebuild the dimension HC_Employees from the tblShadowMDHC_Employees table. This task
will execute if there is any errors in the task bpc_AdminMakeDim.

Figure 14: bpc_AdminMakeDimPlanB BPC task

Task name: sp_UpdateLogError | Task type: [Execute SQL Task]

Figure 15: sp_UpdateLogError Execute SQL Task

2012 SAP AG 23
Process Flow: Import Data process flow
Based on the output value of the sp_BuildPayrollImportData stored procedure, the return code @rcode = 0,
should trigger the Import data process flow, which will the import the transactional data.

Note: Although the package is called IMPORT From SQL, it still takes the contents of the SQL table and dumps it to a
text file, which will then be used for the bpc_ConvertData and bpc_LoadNProcess tasks. These tasks are the
same tasks used as the standard IMPORT package. Although there have been significant improvements to the
way that BPC loads data into the fact tables in version 10, if you are importing large amounts of data, you run the
risk of the process failing. It is quite common to for the bpc_LoadNProcess task to fail when importing millions of
records. If you are importing large amounts of records then try to break them into smaller amounts or batch them.

Figure 16: Import Data process flow

Task name: df_DumpData | Task type: [Data Flow Task]
This data flow task will dump the data from the tbltmpTDPayrollImport table which will be used for the BPC
Load and Process task to import the data into BPC. This data flow tasks has two tasks:
tmptblTDPaylrollImport and TDImportPayroll respectively.

2012 SAP AG 24
Figure 17: df_DumpData dataflow task

This is a data flow task that includes two steps outlined below.

Task name: tmptblTDPaylrollImport | Task type: [OLE DB Source]
This task will take the contents of the tmptblTDPayrollImport table in the staging database which contains
the transactional data which needs to be imported into SAP BPC.

Figure 18: OLE DB Source task

Task name: TDImportPayroll | Task type: [Flat File Destination]
This task will take the contents of the tmptblTDPayrollImport table which contains the transactional data
which needs to be imported into SAP BPC and dumps it into a flat file which will be used by the BPC task
Load and Process.

2012 SAP AG 25
Figure 19: Flat File Destination task

Figure 20: Flat File Mapping
.
The mapping section defines the column mappings which will be used to generate the flat file that will be
used for importing into SAP BPC. The Load and Process task is not able to take a SQL table as a source; it
still dumps it to a flat file and then loads it into SAP BPC.
These column mappings have a corresponding mapping entry in the transformation file.

Task name: bpc_ConvertData | Task type: [BPC Task]
This task will take the contents of the flat file in which the data flow task df_DumpData generated and then
performs conversions and transformations which are defined in the conversion and transformation file
respectively.

2012 SAP AG 26
Figure 21: BPC Task bpc_ConvertData

Note: The parameter values for the BPC tasks are taken from the Data Manager Dynamic Script. The values are
generated at runtime and are not stored as values in the task.
Task name: bpc_LoadNProcess | Task type: [BPC Task]
This task will take the contents of the flat file in which the data flow task df_DumpData generated and then
import it into the SAP BPC Environment / AppSet database fact table. This is the actual task that imports the
transactional data into SAP Business Planning and Consolidations, version for Microsoft.

Figure 22: BPC Task bpc_LoadNProcess

Note: The parameter values for the BPC tasks are taken from the Data Manager Dynamic Script. The values are
generated at runtime and are not stored as values in the task.

2012 SAP AG 27
Task name: sp_UpdateLog | Task type: [Execute SQL Task]
This task will update the tblStatusCode table in the Staging Database with a 0 code meaning that the import
of the data was a success and that no new data records were added.

Figure 23: sp_UpdateLog Execute SQL Task

Error checking process flow
This process will trigger a notification task to notify the administrators that there were invalid records or that
there were errors in processing of the package.

Figure 24: Error checking process flow
2012 SAP AG 28

Task name: check_InvalidRecords | Task type: [Execute SQL Task]
This task will check the table tblImportPayrollInvalidData for any records, if there are any records it means
that there were errors in processing the transactional data

2012 SAP AG 29
Figure 25: check_InvalidRecords Execute SQL task

Task name: SendMailTask | Task type: [SMTP Task]
This task will send an email to the desired recipients to notify them that there were some invalid records or
that there are errors in processing the package. This is meant for informational reporting purposes and to
ensure that there is a level proactive fault notification.
Figure 26: SendMailTask SMTP Task

2012 SAP AG 30

Reference Material
Package Dynamic Script
DEBUG(ON)
PROMPT(TEXT,%TIMEVAL%,"Please enter the correct time period to post the data
to..",,"")
PROMPT(TRANSFORMATION,%TRANSFORMATION%,"Transformation file:",,,Import.xls)
PROMPT(RADIOBUTTON,%CLEARDATA%,"Select the method for importing the data from the
source file to the destination database",0,{"Merge data values (Imports all records,
leaving all remaining records in the destination intact)","Replace && clear
data values (Clears the data values for any existing records that mirror each
entity/category/time combination defined in the source, then imports the source
records)"},{"0","1"})
PROMPT(RADIOBUTTON,%RUNLOGIC%,"Select whether to run default logic for stored values
after importing",1,{"Yes","No"},{"1","0"})
PROMPT(RADIOBUTTON,%PROCESSCUBE%,"Select whether to run process cube after
importing",1,{"Yes","No"},{"1","0"})
PROMPT(RADIOBUTTON,%CHECKLCK%,"Select whether to check work status settings when
running logic.",1,{"Yes, check for work status settings before running logic","No,
do not check work status settings"},{"1","0"})
PROMPT(RADIOBUTTON,%PROCESS%,"Perform a full process the application after creating
the dimension",1,{"Yes","No"},{"1","0"})
PROMPT(RADIOBUTTON,%AVAILABLE%,"Make system available after creating the
dimension",1,{"Yes","No"},{"1","0"})
PROMPT(RADIOBUTTON,%SECURITY%,"Process security after creating the
dimension",1,{"Yes","No"},{"1","0"})
PROMPT(MESSAGE,"When this package is running, end user functionality will be
affected (i.e. data cannot be sent to the database or incorrect numbers can be
retrieved from the database while this package is running)")
INFO(%FILE%,%TEMPPATH%%RANDOMFILE%)
INFO(%TEMPFILE%,%TEMPPATH%%RANDOMFILE%)
INFO(%STAGINGDB%,SustainabilityStaging)
INFO(%DIMENSION%,HC_Employees)
INFO(%SRCTBL%,tbltmpMDC_Employees)
INFO(%SHADOWSRCETBL%,tblShadowMDC_Employees)
CONNECTION(FileConn,ConnectionString,%FILE%)
CONNECTION(SQLServerConn,ConnectionString,Data
Source=%SQLSERVER%;Provider=SQLNCLI10.1;Integrated Security=SSPI;Auto
Translate=False;)
CONNECTION(SQLServerConn,InitialCatalog,%STAGINGDB%)
CONNECTION(SQLServerConn,ServerName,%SQLSERVER%)
TASK(sp_BuildData,SqlStatementSource,execute
SustainabilityStaging.dbo.sp_BuildPayrollImportData '%TIMEVAL%',? OUTPUT)
TASK(sp_RebuildData,SqlStatementSource,execute
SustainabilityStaging.dbo.sp_ReBuildPayrollImportData '%TIMEVAL%')
TASK(bpc_ConvertData,INPUTFILE,%FILE%)
TASK(bpc_ConvertData,OUTPUTFILE,%TEMPFILE%)
TASK(bpc_ConvertData,CONVERSIONFILE,%TRANSFORMATION%)
TASK(bpc_ConvertData,STRAPPSET,%APPSET%)
TASK(bpc_ConvertData,STRAPP,%APP%)
TASK(bpc_ConvertData,STRUSERNAME,%USER%)
TASK(bpc_LoadNProcess,APPSET,%APPSET%)
TASK(bpc_LoadNProcess,APP,%APP%)
TASK(bpc_LoadNProcess,USER,%USER%)
2012 SAP AG 31
TASK(bpc_LoadNProcess,DATATRANSFERMODE,1)
TASK(bpc_LoadNProcess,CLEARDATA,%CLEARDATA%)
TASK(bpc_LoadNProcess,PROCESSCUBE,%PROCESSCUBE%)
TASK(bpc_LoadNProcess,PROCESSMODE,2)
TASK(bpc_LoadNProcess,FILE,%TEMPFILE%)
TASK(bpc_AdminMakeDim,DESCRIPTION,bpc_AdminMakeDim)
TASK(bpc_AdminMakeDim,APPSET,%APPSET%)
TASK(bpc_AdminMakeDim,APP,%APP%)
TASK(bpc_AdminMakeDim,USERID,%USER%)
TASK(bpc_AdminMakeDim,PROCESSMODE,4)
TASK(bpc_AdminMakeDim,DIMENSIONNAME,%DIMENSION%)
TASK(bpc_AdminMakeDim,SOURCETABLE,%SRCTBL%)
TASK(bpc_AdminMakeDim,BAPPLICATIONPROCESS,%PROCESS%)
TASK(bpc_AdminMakeDim,BTAKESYSTEMAVAILABLE,%AVAILABLE%)
TASK(bpc_AdminMakeDim,BSECURITYPROCESS,%SECURITY%)
TASK(bpc_AdminMakeDimPlanB,DESCRIPTION,bpc_AdminMakeDimPlanB)
TASK(bpc_AdminMakeDimPlanB,APPSET,%APPSET%)
TASK(bpc_AdminMakeDimPlanB,APP,%APP%)
TASK(bpc_AdminMakeDimPlanB,USERID,%USER%)
TASK(bpc_AdminMakeDimPlanB,PROCESSMODE,4)
TASK(bpc_AdminMakeDimPlanB,DIMENSIONNAME,%DIMENSION%)
TASK(bpc_AdminMakeDimPlanB,SOURCETABLE,%SHADOWSRCETBL%)
TASK(bpc_AdminMakeDimPlanB,BAPPLICATIONPROCESS,%PROCESS%)
TASK(bpc_AdminMakeDimPlanB,BTAKESYSTEMAVAILABLE,%AVAILABLE%)
TASK(bpc_AdminMakeDimPlanB,BSECURITYPROCESS,%SECURITY%)

2012 SAP AG 32
Related Content
Reference SQL Code
http://db.tt/RkMZD35y

SCN forum discussions for Updating of Master Data
http://scn.sap.com/thread/1688034
http://scn.sap.com/message/13442330#13442330

How to Build SSIS Packages for Business Planning and Consolidation 5.0
http://scn.sap.com/docs/DOC-4131

Microsoft SQL Server Integration Services Tutorials
http://msdn.microsoft.com/en-us/library/ms169917(v=sql.105).aspx
http://www.accelebrate.com/sql_training/ssis_2008_tutorial.htm

2012 SAP AG 33
Copyright
Copyright 2012 SAP AG. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG.
The information contained herein may be changed without prior notice.
Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.
Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.
IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, System z9, z10, z9,
iSeries, pSeries, xSeries, zSeries, eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server,
PowerVM, Power Architecture, POWER6+, POWER6, POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes,
BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX,
Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM Corporation.
Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems
Incorporated in the United States and/or other countries.
Oracle is a registered trademark of Oracle Corporation.
UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.
Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of
Citrix Systems, Inc.
HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C, World Wide Web Consortium, Massachusetts
Institute of Technology.
Java is a registered trademark of Oracle Corporation.
JavaScript is a registered trademark of Oracle Corporation, used under license for technology invented and implemented by Netscape.
SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP Business ByDesign, and other SAP products and services mentioned
herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.
Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and
other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered
trademarks of Business Objects S.A. in the United States and in other countries. Business Objects is an SAP company.
All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document
serves informational purposes only. National product specifications may vary.
These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies ("SAP
Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or
omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the
express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an
additional warranty.

Master Data On The Fly - MS

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Master Data On The Fly - MS

Enviado por

Direitos autorais:

Formatos disponíveis

SAP COMMUNITY NETWORK scn.sap.

Você também pode gostar