Escolar Documentos
Profissional Documentos
Cultura Documentos
BI
Fundamentos
Adquisición de datos
Scope Part 2.
The first lesson describes the flow of data between BI
and source systems that contain data.
The second lesson shows the procedure for loading
master data (attributes and texts) from an SAP system.
On the third lesson we will discuss the data transfer
process with more complexity and more details. We
will discuss the available transformation rule types and
more advanced start and end routines. In addition, we
will visualize our data in the InfoCube upon
completion.
Generic Data Warehouse Positioning of the
Data Flow
The ETL process, sometimes called the data flow is a list of the steps that raw (source)
data must follow to be extracted, transformed, and loaded into targets in the BI system
BI Architecture: Positioning of the ETL
Process
BI Data Flow Details
Source Systems and DataSource
Once the data arrives in the PSA, you then to cleanse / transform it prior to physical storage in your
targets. These targets include InfoObjects (master data), InfoCubes and DataStore Objects.
Optional BI InfoSources
InfoPackages and Data Transfer Processes 1
The first process is loading the data from the source system. This
involves multiple steps that differ depending on which source
system is involved. For example, if it is a SAP source system, a
function call must be made to the other system, and an extractor
program associated with the DataSource might be initiated. An
InfoPackage is the BI object that contains all the settings directing
exactly how this data should be uploaded from the source system.
The target of the InfoPackage is the PSA table tied to the specific
DataSource associated with the InfoPackage. In a production
environment, the same data in the same source system should only
be extracted once, with one InfoPackage; from there, as many data
transfer processes as necessary can push this data to as many
InfoProviders as necessary.
InfoPackages and Data Transfer Processes
Initiate the Data Flow
InfoPackages and Data Transfer Processes 3
The second process identified in the figure is the data transfer process.
It is this object that controls the actual data flow (filters, update
mode (delta or full) for a specific transformation. You might have
more than one data transfer process if you have more than one
transformation step or target in the ETL flow. This more complex
situation is shown below. Note if you involve more than one
InfoProvider, you need more than one data transfer process.
Sometime necessity drives very complex architectures.
More Complex ETL: Multiple InfoProviders
and InfoSource Use
Loading SAP source system Master Data
Scenario
Global Transfer Routines
In this first load process, we are trying to keep it simple. Since we added some custom global transfer
logic directly to our InfoObject, we just need field-to-field mapping for our third step:Transformation.
Transformation GUI Master Data
InfoPackage: Loading Source Data to the
PSA
Creation and Monitoring of the Data
Transfer Process
Complete Scenario: Transaction Load from
mySAP ERP
Emulated DataSources
Issues Relating to 3.x DatasSources
Using the Graphical Transformation GUI
The Transformation Process: Technical
Perspective
Start Routine 1
Start Routine 2
Transformation Rules: Rule Detail
Transformation Rules: Options and Features
Transformation: Rule Groups
A rule group is a group of transformation rules. It contains one transformation rule for each key field
of the target. A transformation can contain multiple rule groups. Rule groups allow you to combine
various rules. This means that you can create different rules for different key figures for a
Transformation Groups: Details
End Routine
Data Acquisition Layer
Extraction using DB Connect and UD
Connect
UD Connect Extraction Highlights
DB Connect Extraction
Technical View of DB Connect
XML Extraction
XML Purchase Order Example
XML Extraction Highlights
Loading Data from Flat Files: Complete
Scenario
Flat File Sources
Features of the BI File Adapter and File-
Based DataSources
Basically a DataSource based on a flat file is an object that contains all the settings necessary to load
and parse the file when it is initiated by the InfoPackage. Some of features of the BI file adapter are
listed below.
File System DataSource: Extraction Tab
File System DataSource: Proposal Tab
File System DataSource: Fields tab
File System DataSource: Preview Tab
BI Flexible InfoSources
A New BI InfoSource in the Data Flow
Complex ETL: DataSource Objects and
InfoSources
DTP: Filtering Data
Error Handling
The data transfer process supports you in handling data records with
errors. The data transfer process also supports error handling for
DataStore objects. You can determine how the system responds if
errors occur. At runtime, the incorrect data records are sorted and
can be written to an error stack (request-based database table). In
addition, another feature supports debugging bad transformations. It
is called temporary storage.
Error Processing
Features of Error Processing
More Error Handling Features
DTP Temporary Storage Features
Access to the Error Stack and Temporary
Storage via the DTP Monitor
Loading and Activation in DataStore Objects
DataStore objects
InfoObjects
InfoSets
Planning)
MultiProvider Concept
Advantages of the MultiProvider
The Manage function allows you to display the contents of the fact
table or the content with selected characteristic values (through a
view of the tables provided by the Data Browser). You can also
repair and reconstruct indexes, delete requests that have been loaded
with errors, roll up requests in the aggregates, and compress the
contents of the fact table. Select the InfoCube that you want to
manage and choose Manage from the context menu. Six tab pages
appear:
Contents
Performance
Requests
Roll-Up
Compress
Reconstruct ( Only valid with 3.x data flow objects)
Managing InfoCubes
Requests in InfoCubes
Compressing InfoCubes
Management Functions of DataStore Objects
The Query icon, indicating readability by BEx queries, is set when activation is started for a
request. The system does not check whether the data has been successfully activated.
DataStore Object Change Log: Maintenance
Required