Escolar Documentos
Profissional Documentos
Cultura Documentos
juergen.haupt@sap.com
SAP (SAP AG and SAP America, Inc.) assumes no responsibility for errors or omissions in these materials.
These materials are provided “as is” without a warranty of any kind, either express or implied, including but not limited to, the
implied warranties of merchantability, fitness for a particular purpose, or non-infringement.
SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, or consequential damages
that may result from the use of these materials.
SAP does not warrant the accuracy or completeness of the information, text, graphics, links or other items contained within
these materials. SAP has no control over the information that you may access through the use of hot links contained in these
materials and does not endorse your use of third party web pages nor provide any warranty whatsoever relating to third party
web pages.
ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0
Table of Contents
ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW ....................................... 1
1 INTRODUCTION........................................................................................................................... 1
1.1 SUPPORTED SOFTWARE VERSIONS ........................................................................................... 1
1.2 THIS WHITE PAPER .................................................................................................................. 1
2 AIMS OF THE WHITE PAPER ..................................................................................................... 1
3 MOTIVATION ................................................................................................................................ 2
3.1 THE MARKET ............................................................................................................................ 2
3.2 WHY DO YOU NEED A CORPORATE BW STRATEGY?................................................................. 3
4 ENTERPRISE DATA WAREHOUSING WITH SAP BW.............................................................. 5
5 BW ENTERPRISE ARCHITECTURE........................................................................................... 6
5.1 ASPECTS OF A DATA WAREHOUSE ARCHITECTURE .................................................................... 6
5.2 DATA STORE ARCHITECTURE - BW DATA LAYER ..................................................................... 10
5.2.1 BW Architected Data Mart Layer .................................................................................. 11
5.2.2 BW Data Warehouse Layer .......................................................................................... 14
5.2.3 BW Extraction and Staging........................................................................................... 19
5.2.4 BW Support for Operational / Real Time Reporting ..................................................... 20
5.3 DATA ARCHITECTURE - BW DATA MODEL ............................................................................... 24
5.3.1 Operative Data Models and Data Warehouse Data Model .......................................... 24
5.3.2 The BW Data Model ..................................................................................................... 25
5.4 TOPOLOGIES – BW LANDSCAPES ........................................................................................... 29
5.4.1 Consistency in a distributed BW Landscape ................................................................ 30
5.4.2 BW Landscapes und Technical Parameters ................................................................ 31
5.4.3 Inside-Out Landscape Architecture - BW as Central Enterprise Data Warehouse...... 32
5.4.4 Outside-In Landscape Architecture - Decentralized BW Data Warehouses................ 34
6 CENTRAL BW APPLICATION DEVELOPMENT ...................................................................... 38
1 Introduction
3 Motivation
3.1 The Market
At the time of writing, there are more than 6000 BW installations within the market.
The fundamental advantages of BW have led more and more customers to center their corporate
data warehousing strategy around BW. Customers frequently stress the end-to-end conception of the
BW in this context. In comparison to fragmented technologies, the integrated metadata concept, from
data integration right through to analysis, leads to lower total costs for the project overall.
The BW has moved away from isolated instances to incorporate an architecturally-based view. In this
way, the BW has come to set new standards of quality in terms of its positioning within an enterprise.
The following graphic shows various rudiments of this strategy:
Evolution of SAP BW
BW1
BW2 ‘Headquarter’
BW1 Global
Reporting
BW
BWn
Scenario
Others
BW2
‘Local’
BW
Architected
Global
‘Local’ BW Landscape
BW3 BW
BW Scenario
‘Local’
BWn BW
Spoke BW Enterprise
Issues: BW Data
Synergy Enterprise
BW Warehouse
Integration Spoke Scenario
Consistency BW
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 3
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 3
This leads to one main motivation for this paper. It aims to support the evolution-process of BW
offering an overview of the essential criteria for corporate architected BW implementations for
customers and consultants.
On the other hand, there is still a large number of customers who do not have a corporate data
warehouse strategy and who have implemented BW in a more isolated and restricted way.
Thus, another motive is the hope of making people aware of the prospects and benefits of a
business-wide, architecturally-based BW implementation.
Real World
U V W .... Requirements
Information
(grouped to Scopes)
BW Application Project Team
Extraction
Sources
SuccessfulData
Successful DataWarehousing
Warehousingmeans
means ‘Controlled
‘ControlledRedundancy’
Redundancy’!!
Source Systems
SuccessfulData
Successful DataWarehousing
Warehousingmeans
means‘Controlled
‘ControlledRedundancy’
Redundancy’ !!
The more complex an enterprise’s structures, the more important it is to have corporate-wide
guidelines and recommendations for the implementation of BW. They guarantee long-term
consistency and reduced costs.
5 BW Enterprise Architecture
An architecture can be seen as a system design decision that has a specified duration. In other
words, once it has been implemented it is not easy to change.
In defining a corporate BW architecture it is clear that the term ‘architecture’ is used in various
senses, and that aspects such as ‘where’, ‘what’ and ‘how’ often become confused. This leads to
confusion and is dangerous in that essential factors may be considered from a lop-sided perspective,
or not at all.
Persistent Infor-
Staging mation
Area Archi- Access
Data tected
SAP Warehouse Data
Source Marts
Repeatedly saving raw data in various refinement situations means on the one hand side
redundancy. On the other hand side means a consistent layer architecture the only way to control the
redundancy of data warehousing, which derives from its very nature. This has to be taken into
consideration with the layer data store schemas.
However, a suitable layer support is not the only necessary condition for a corporate-wide data
warehouse architecture that can secure consistency in the mid- and long-term.
This can only be achieved if the operative data models are mapped consistently to the model of the
data warehouse layer and the model of the data mart layer. Such a consistent data (model)
architecture therefore describes the model of each layer and how the models relate to each other.
As such we need a data architecture that guarantees, for example, that operative material terms, as
they are found in the respective sales or distribution area, or in materials management, are
reproduced consistently in the material subject area.
Ideally you would have a single operative enterprise data model and use that to derive the model for
each layer. However, a corporate data model of this sort does not normally exist, particularly if legacy
systems are being used.
This means that often no or only limited guidelines and rules exist how to map the operative models
to a single data warehouse layer model and a single data mart layer model because of the difficulties
to achieve this. The result is unsynchronized isolated solutions (‘stovepipe solutions’).
This is illustrated in the following graphic:
Inconsistent data warehouse data models ⇒ Consistent data warehouse data model ⇒
stovepipe solutions long term success
A data warehouse
consistency challenge
A missing data model architecture is one reason why corporate data warehouse strategies fail.
Only when there is clarity regarding the data layer and data model does it make sense to discuss the
corporate-wide topological aspects of a BW architecture. Because topological aspects are
influenced by the layer architecture.
Two basic topologies are differentiated:
• The BW enterprise data warehouse landscape with BW as the central information hub
• Landscape based on ‘local’ BWs with a higher level ‘global’ BW
Inside-
Inside-Out Others Others Outside-
Outside-IN
BW Enterprise BW
Data Landscape
Spoke
Warehouse ‘Local’
BW BW
Enterprise
Spoke Global
BW ‘Local’
BW BW
BW
Spoke
‘Local’
BW
BW
When determining the BW topology, political, organizational and technical factors play an important
role. As a result you cannot really talk about a generally valid optimal BW topology.
We summarize that it is necessary to make definitions that are valid enterprise-wide in the following
areas:
Data Architected
Warehouse Data Marts
Extended Star Schemas
Extraction / Open Staging
Master
Reference
Open Distribution
Data
Finance
any source
any target
Transaction
Data
Logistic
ODS
Flow Control
Meta Data
Material
Local Part of
Material Dimension Dimension
Material Master Table
Material Number
Material Number
Material Type
Material_Dimension_ID
Material Text Table
Material Number
Material Number
Material Dimension Material Number
InfoCube Table Language Code
Language
Material Code
Name
Material_Dimension_ID Material Hierarchy Table
SalesOrg_Dimension_ID Vertriebsorganisation
Customer_Dimension_ID Material
Bezirk 1Bezirk 2 BezirkGroup
3 Bezirk 4 Bezirk 5
Sales Amount
Gebiet 1 Gebiet 2 Gebiet 3 Gebiet 3a Gebiet 4 Gebiet 5 Gebiet 6 Gebiet 7 Gebiet 8
Quantity
FACT Table
Shared, global
Part of
BW Extended Star Schema Material Dimension
CUSTOMER
master
Conformed dimensions essentially support horizontal consistency within the data mart layer. They
are part of the BW concept and help prevent the occurrence of isolated solutions (‘stovepipes’). The
BW term for these conformed dimensions is ‘master data’.
This is one reason to qualify the BW data marts as Architected data marts.
The effectiveness of the BW InfoCube schema does not just support consistency but is obviously also
essential for mapping end-users’ demands. Above all we have to mention here the ability to map
entities from different historical views to the same key figure.
By using surrogate keys in the InfoCube and because of the master data/ conformed dimension
concept, the BW makes it possible to simultaneously map different views to key figures in one
InfoCube.
Constellation 09/1998
Customer Mother-Company
Report using today‘s (10/98) constellation
AAA X Mother-Comp Rev 09/98 Rev 10/98
BBB X X 100 0
CCC Y Y 200 200
• 'Today is Yesterday' : Show key figure values, including historical key figure values, from the
currently valid perspectives (current view of dimension attributes)
• 'Yesterday is Today' : Show key figure values from user-defined perspectives (key date oriented
view of dimension attributes)
• 'Historical Truth' : Show key figures (transactions) in the same state (historicization of dimension
attributes), as at the time of posting – historically correct
Scenarios can also be presented in which only key figure values with unchanged dimension attribute
constellations are reported.
In order to reduce redundancy and counteract the ‘one report is equal to one InfoCube’ error, virtual
multidimensional structures are supported for persistent InfoCubes. These are called BW
MultiProviders and BW Queries.
Multi-dimensional Schemas in BW
End-
End-User
Reporting & Analysis need
BW BEX Report
Multi-
Multi-dimensional model end-user requirement
specific
Despite the capabilities of BW data mart layers, the limitations and dangers of a data mart-focused
implementation are also evident:
Limited reusability of data and loss of historical conditions
• Data is dealt with in the data mart layer in a scope-specific manner: Aggregation/
manipulation/ overwriting of data is normal. It follows that the usage of these data for other
purposes is limited.
• The data mart layer is the direct access layer for the end-user. Performance, therefore, plays
a decisive role. For this reason only data that is actually needed is retained in the data mart
layer. Saving data ‘just in case’ will quickly lead to a large data volume and impact negatively
on performance. Overwriting old images, as normally happens in the conformed dimensions
(master data), is therefore valid and indeed desired. However, this means that history is lost.
Redundant data storage (the same data is held in various InfoCubes and often in different
degrees of granularity)
Redundant data acquisition (staging)
Redundant data extraction
Limited ability to trace data.
Data staging is required by the BW layer architecture here.
Factors arising from contact with the data mart data model that threaten consistency are discussed in
the chapter on data model architecture.
Data Architected
Warehouse Data Marts
Extended Star Schemas
Extraction / Open Staging
Master
Reference Open Distribution
Data
Finance
any source
any target
Transaction
Data
Logistic
ODS
Flow Control
Meta Data
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 41
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 41
The following description of a BW data warehouse layer is based on William H. Inmon’s definition:
A BW data warehouse layer offers:
o Subject-oriented, integrated data
o Original data that will not be changed in relation to the goals of a particular project
(‘unflavored data’)
o Granular data
Non-aggregated data
o Historical data (that cannot be overwritten or deleted)
A BW data warehouse (layer) is an enterprise’s memory, a repository for information for the
whole enterprise.
In addition to offering horizontal consistency within the BW data mart layer, the BW data warehouse
layer offers a sort of vertical consistency that the data mart layer alone is not able to guarantee.
CUSTOMER
master
MATERIAL
master
Vertical Consistency:
Single Point of Truth ¨
BW DWH Layer ODS-
ODS-Objects
The question of how to organize data warehouse layers often causes arguments. Extreme positions
on this range from 3NF to de-normalized star schemas. We do not want to give a definite answer
here, but suggest taking a pragmatic approach. As the data warehouse layer forms the basis for
supplying the data marts, data should be saved so that it can be easily transported into conformed
dimensions and InfoCubes. This means complex join operations for normalized data warehouse
objects on the way to the data marts should be avoided. It is therefore, for example, legitimate to
store position data and header information for a sales order together in the data warehouse layer.
To the question, "what kind of database design is appropriate for a data warehouse?", W.H. Inmon
offers the following answer:
‘For a data warehouse, it is always better to have a lightly normalized design. Star schemas fit
elsewhere (s. Data Marts t. author). The data warehouse design is not perfectly normalized because
a few alterations are made for common usage, such as creating arrays of data when data is always
used together, creating a small and selective amount of redundancy where appropriate, merging
tables that are commonly used together, and so forth. At the end of the day, a data warehouse
design is distinctively normalized.
This answer presumes that you know the difference between a data warehouse and a data mart.
Restatement or regeneration of some of these higher levels of information requires that we keep
available a detailed historical data store. This should be ‘clean’ (free of errors and conformed to the
Data Warehouse master data model) to avoid unnecessary reprocessing of source data.’
(W. H. Inmon, Architecture 2002)
The BW does not offer a data store specific for the data warehouse layer. Transaction data is saved
in flat ODS objects, master data, reference data, in InfoObjects or ODS objects. Handling history is
supported in BW by staging processes (transfer/ update rules) and staging ODS objects.
DWH Object
Customer CustomerGroup
A X 1st Load: 20020101
Sourcesystem S1
B Z
2nd Load: 20020401
A Y
With master data, the source system normally does not deliver versioned or historical data images.
Furthermore full-loads are often provided for master data. If the source system does not provide any
historization, the BW staging processes must provide this information. The management of activity
time slots is supported by BW for data warehouse InfoObjects automatically. For data warehouse
ODS objects activity time slots must be maintained during staging. It is also the task of the BW with
‘full loads’ to determine the changed records. This is done using special staging ODS objects.
These tasks are omitted when SAP Master Data Management (SAP MDM) historicized Delta images
are provided.
Handling transaction data is easier, as long as the transaction is not to be posted again. With
changeable transactions a simple time stamp is often not enough to prevent it from being overwritten.
InfoCube
Customer Dimension Material Dimension
A 4712
4713
Amount
Customer Material Time Company
Currency
A 4712 200211 100
BW Architected Data Mart Layer A 4713 200211 150
Time Dimension
other InfoCube other InfoCube
200211
daily
Sourcesystem Amount
Customer Time DocNo Pos Material Local
Currency
A 20021107-10am 1 10 4711 100 New booking
A 20021107-3pm 1 10 4712 200 Correction booking
A 20021107-4pm 2 10 4713 300 New booking
The quantity of data in a BW data warehouse layer mounts quickly. The BW/ SAP Archiving
Development Kid (ADK) provides the prerequisites for the data warehouse layer to cope with the
demand, without the memory space used escalating exorbitantly.
If a large data volume is to be made available online, the BW supports the usage of nearline storage.
Staging Cleansing
Area: Transform
PSA
BW Data
Extraction / Open Staging
Warehouse
any source
BW ODS
Whether and which data statuses between raw and integrated data is saved persistently has to be
decided on a case by case basis. This decision depends on the quality of data and the outlay for
potential data harmonization. BW ODS objects are used as data stores.
Architected
Data Data Marts
Warehouse
Extraction / Open Staging
Tactical / Strategic
Data
Master
Mart
Reference
Data Finance
Transaction
Data
Data
Mart
Logistic
Alongside the classic layers (staging, data warehouse and data mart) an additional data area is often
postulated: The Operational Data Store (ODS). The operational data store supports operative
reporting, whereas data marts are data structures for tactical and strategic reporting and analysis.
In practice, if operative reporting does not demand real-time data, the physical division between the
data mart and the ODS becomes artificial.
Mostly loading data two or three times daily is sufficient for operative reporting. Updating data on a
daily basis may even be enough. The greater the ‘latency’ (the time difference between the operative
event and the acquisition of this event for reporting purposes), the more sensible it is to store granular
data in data mart InfoCubes too, and to use this both for tactical analysis and operative reporting. Flat
BW ODS objects are better suited to purely listing data.
In this way, most BW implementations already support operative reporting and so relieve the
pressure on operative systems.
ODS-Object
Sales Orders
BW supports this tactical analysis and operational reporting on the same InfoCube through pre-
calculated aggregates, ‘aggregation awareness’ in the OLAP Engine, and ‘query catching’.
However it is to be pointed out that the utilization profile of data for operative usage is different in
regard to both its history and its granularity to data for analytical usage. With large volumes of
transaction data this has to be accounted for in the schema modeling.
With delayed operative reporting, which follows the path of ‘classic data warehousing’, there is no
need to define a dedicated BW operational data store. You do then have to be aware of the possible
consequences of basing analysis and operative reporting on the same structures.
One the other hand, if data is requested close to the event time in the operative system, the ‘classic’
pull principle of the BW data warehousing boundaries are set. In this case we talk about ‘real-time’ or
‘near real time operational reporting’. The data extracted (in real time) is written straight to ODS
objects, without going through the data warehouse layer. These objects form the BW operational data
store.
Data Architected
Warehouse Data Marts
Master
Reference
Open Distribution
Data
Finance
any source
any target
Transaction
Data
Logistic
ODS
Flow Control
Meta Data
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 83
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 83
Illustration 16: BW operational data store: Near real time data warehousing
Data from an operational data store is not to be processed directly in the data mart layer for reasons
of data consistency. If data from an operational data store is needed in a data mart scenario, you
should always go through the (enterprise) data warehouse.
To summarize, the functionalities planned and offered by BW to support real time reporting scenarios
are:
BW Remote InfoCubes – external InfoCube fact tables
BW allows you to define InfoCubes whose fact tables are themselves not persistent in the
BW. The data is then read by an extractor during the query runtime and subjected to ‘normal’
InfoSource treatment as far as transfer and update rules are concerned. It can then be made
available to the OLAP processor.
All data sources that can be addressed by BW can make external fact tables available in this
way.
BW Virtual InfoCubes – a program that behaves like a fact table
BW ‘near real time data warehousing’ (next main BW release)
With the next main BW release it will be possible to ‘push’ data into BW ODS objects directly
after either its emergence or the change event.
Direct reporting to operative structures (next main BW release)
Reporting directly to operative structures will also be an option if integration with other data is
not required. However the burden produced (lack of data integration and data quality
controls) means tight limitations are set for this option.
Key-Questions:
Data Latency ?
Data Integration ?
Workload on OLTP Systems ?
Ul
Apollo Apollo BW
BW Near real time BW remote BW virtual BW ‘classic’
data warehousing InfoCube InfoCube data warehousing
OLTP
Inconsistent data warehouse data models ⇒ Consistent data warehouse data model ⇒
stovepipe solutions long term success
A data warehouse
consistency challenge
Illustration 18: Operational data model and data warehouse data model
Generating a business-wide data warehouse data model based on operative data models, which are
not harmonized, is a complex and political procedure. This is why the data warehouse data model
architecture is often neglected. The result is isolated applications or ‘stovepipe’ data marts that are
not integrated.
In this regard BW customers finds themselves in a good position.
BW offers an extensible data warehouse data model that represents the operative models of the SAP
applications and a large number of industry-specific solutions consistently as part of the BW Business
Content. Data models also exist for specific applications from the competitive environment (Oracle
Financials, for example).
BW customers with a high operative degree of coverage from SAP applications can enjoy being in
the situation illustrated in the following graphic:
B W C o n s is te n t d a ta w a re h o u s e d a ta m o d e l fo r
S A P A p p lic a tio n s a n d o th e rs ⇒
lo n g te rm s u c c e s s
A d a ta w a re h o u s e
c o n s is te n c y c h a lle n g e
N o t a lig n e d o p e ra tio n a l
d a ta m o d e ls S A P e n te rp ris e d a ta m o d e l
Illustration 19: Data warehouse data models and the situation for BW customers
The BW Business Content data model watches over the incremental generation of InfoCube
solutions and ensures that they match, often without the customer even having to be aware of this.
This is the true value of BW Business Content from an architecture point of view, and one reason for
the success of BW.
0MATTYPE
0AMOUNT
0CUSTOMER 0SALESORG
BW Data Model
0SALESPERS
0MATERIAL
0MATGR 0DOCNO
Materialgroup Sales
MaterialType
Enterprise Data Model: ORG
e.g. mySAP Component
Object Models
Customer Material Sales Person
Sales
Transaction
Illustration 20: Enterprise data model and enhanceable BW data model: InfoObjects
Alongside technical properties like format and length, InfoObjects carry data warehousing-relevant
properties:
• Aggregation behavior (key figures)
• Standard displays (key figures)
• Textual description
• Language-dependant descriptions
• External hierarchies
•…
Clustering operative entities and attributes to subject areas takes place through InfoSources. Master
data and transaction data InfoSources are differentiated:
BW
BW
Data Model definedby
Data Model defined by
BusinessContent
Business Content
BCT InfoSources ≡ 0MATERIAL 0MATGR 0MATTYPE 0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT
Subject Areas
Master Data Transaction Data
Materialgroup Sales
MaterialType
ORG
InfoSources always represent a set of InfoObjects and describe relations between these.
InfoSources are the central bridge between the operative and the BW data model.
In the operative system they correspond in form to a DataSource that, in turn, represents an
extractor’s result structure.
In the BW data mart layer an InfoSource either defines a conformed dimension (master data
InfoSources), or a relation between conformed dimensions (transaction data InfoSources), in
other words, the basis for InfoCube schemata.
The following graphic illustrates the role of BW InfoSources with regard to the data mart layer data
model:
BWExtended
BW ExtendedStar
StarSchemas
Schemas
0MATERIAL
0CUSTOMER 0SALESPERS
0MATGR
00CUSTGR 0SALESORG 0CUSTOMER
0MATTYPE
...... ..... 0MATERIAL
......
0AMOUNT
Shared/Conformed
Shared/ ConformedDimensions
Dimensions InfoCubeswith
InfoCubes withLocal
LocalDimensions
Dimensions
BWData
BW DataMart
MartLayer
Layer
Data Model defined by
Data Model defined by
Business Content
Business Content
BCT InfoSources ≡ 0MATERIAL 0MATGR 0MATTYPE 0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT
Subject Areas
Master Data Transaction Data
Materialgroup Sales
MaterialType
ORG
In the BW (enterprise) data warehouse a master data InfoSource and additional time-slice or
time-stamp attributes defines a data warehouse ODS object, which is able to store different
versions of a subject area, which occur over time. Time-slices or time-stamps for master data are
normally not offered by the source systems. This information has to be provided in the transfer or
update rules during staging. As a rule of thumb a transactional-InfoSources define the structure
of a data warehouse ODS object for transactional data (see following graphic):
BWData
BW DataWarehouse
WarehouseLayer
Layer
+ Insert Timestamp/ Data Model defined by
Data Model defined by
If Overwrite:
Activity Time Slice Unique Identifier
+ Source BusinessContent
Business Content +Source
BCT InfoSources ≡ 0MATERIAL 0MATGR 0MATTYPE 0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT
Subject Areas
Master Data Transaction Data
Customer
Material
Sales Person EnterpriseData
Enterprise DataModel:
Model:
e.g. mySAP Component
e.g. mySAP Component
ObjectModels
Object Models
Sales
Transaction
BW Business Content offers a pre-defined extensible data model for SAP components based on
InfoObjects and InfoSources. This data model describes the BW data mart layer and also extends
the BW data warehouse layer to include time slices.
If the Business Content data model for mySAP components is used throughout, data mart InfoCube
scenarios for a particular themed area can be built without producing isolated applications.
However, the discussion in the previous chapters has made it clear that focusing on landscape
aspects alone is superficial and in no way productive if strategies regarding the persistent layer and
its data models are not also developed. If this is now clear, we can pose the question:
Enterprise-wide Strategies
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 109
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 109
88pp m
m
Data actuality SS aa 1122aa m
m
88aa m
m
System upgrades and maintenance
1122pp m
m
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 114
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 114
BW as Central Enterprise
Spoke
BW BW Enterprise
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 111
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 111
All data that acts as the basis for tactical and strategic data marts has to go through the enterprise
data warehouse.
Extraction straight into data marts, ignoring the enterprise data warehouse, is not permitted.
Exceptions have to be controlled. There must be a strategy in place for using these data marts as the
basis for operative reporting, as well as for using the enterprise data warehouse itself for operative
reporting.
A landscape of this sort with the inherent demands
• „single point of truth” and
• “extract once, deploy many”
offers the ideal prerequisites for controlling redundancy at corporate level.
It is often assumed that this solution, although ideal, is practically unrealizable. Without going into the
arguments in detail, it is sufficient to say that this is not necessarily the case for BW customers.
Many large companies that operate globally make a large investment in order to unify their operative
landscapes by implementing SAP R/3. As a result, the ideal prerequisites for a central enterprise data
warehouse approach emerge: integration of data (common coding), integration of data models into
the SAP R/3 data model, the building of an operative R/3 landscape, which takes the company
organizational and technical conditions into account.
In other words, a centralized approach in the operative environment leads directly to a solution option
for the enterprise data warehouse topology. This type of ’ideal’ environment is often found in ’single
division’ enterprises that find themselves in strongly competitive environments and as a result, have a
high sense of awareness for the usage of more reliable and efficient IT solutions. This awareness
mostly involves a high degree of support from the highest management level for a BW enterprise data
warehouse solution.
Inside-Out: Enterprise
Spoke
BW BW Enterprise
Inside-Out:
Central EDW BW Instance Covers All
PSA
BW Architected Data Marts
BW ODS
operational/ near real time
reporting
granular
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 110
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 110
A BW enterprise data warehouse based landscape will be implemented incrementally for cost
reasons. Often, first a BW instance will be built that offers data warehouse layer as well as data mart
layer services, including operative-level reporting.
Obviously the aim of the enterprise data warehousing strategy also has to be to source all external
data marts with data from the BW enterprise data warehouse only!
It is then possible to separate reporting and analysis services as the load increases.
Inside-Out:
Central EDW BW Instance as Information HUB
Architected Data Marts Spoke Instances
PSA BW ODS
operational/ near real time
reporting
BW Ogranula
DS r
operational reporting
BW ODS
Architected
BW granular Data Marts
operational reporting
granular tactical/
operational like
PSA reporting
Corporate BW
less granular Data Mart
Staging Area
tactical/ strategic
aggregation
BW EDW operational like
reporting
reporting
integrated
consistent aggregated
granular
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 112
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 112
To summarize, the conditions for a central BW enterprise data warehouse solution are always most
favorable within a ‘single division company’ with ‘strong’ headquarter.
Even where it is preferable to have a single BW enterprise data warehouse for the whole enterprise,
the complexity of the different business areas in a company that operates globally can lead to shared
BW landscapes. On this issue William H. Inmon says:
‘Simply building a central single data warehouse does not address the size and complexity of
the problem posed by the need for integrated, historical easily accessible data across a
complex global enterprise.’
W.H. Inmon ‘Managing Multiple Data Warehouse Development‘
Often political circumstances or a company’s lack of an over-arching concept lead to shared BW
landscapes too, even when the business conditions seem to be favorable.
Enterprises that choose diversification should generally check that the outlay involved in integrating
data on a granular level, as required with the enterprise data warehouse approach, is not greater
than the benefit.
Even when the ideal prerequisites exist such as for a ‘single division’ enterprise, certain conditions (a
regional enterprise structure, for example, without process integration between the regions), coupled
with technical and political factors, can lead to an enterprise moving away from the central BW
enterprise data warehouse concept.
This leads to a landscape that comprises several ‘local’ BWs. ‘Local’ here means that one of these
BWs covers only one part of the organization. The description of what ‘local’ can mean in individual
cases depends on the size and the complexity of the company.
Each ‘local’ BW should operate like an enterprise data warehouse for its own area. In this way, the
sum of the ‘local’ enterprise data warehouses forms a quasi virtual enterprise data warehouse.
Unlike the central enterprise data warehouse that requires all corporate data to be integrated at
granular data level, this is not necessary with ‘local’ enterprise data warehouses, although it is still
preferable. Each ‘local’ enterprise data warehouse necessarily ensures a ‘local’ granular data
integration.
Having a landscape architecture based on ‘local’ BWs means that for over-arching reporting and
analysis, at least one additional integration layer is required to consolidate the data from the ‘local’
BWs.
A BW that consolidates the data from other BWs and offers this in an integrated form is called a
‘global’ BW. Again, the term ‘global’, and the scope of a particular ‘global’ BW can only be defined on
an individual, company-specific basis.
Global BW
Headquarter
Divisional
Global BW
Division A
Data Marts ODS Data Marts ODS Data Marts ODS virtual
Data Warehouse Data Warehouse Data Warehouse
BW Enterprise
Staging
Data
Staging Staging Staging
Warehouse
Local
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 128
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 128
In terms of an enterprise’s data integration strategy, it is important that master data harmonization
takes place on an aggregated level (global BW)1.
1
Integration is then only demanded at the higher hierarchy levels of a ’subject area’ or dimension.
This is generally easier to achieve than integration at granular level. As a result, a topology of this
kind often mirrors the integration strategy of an enterprise that does not choose granular ‘common
coding’.
The situation of ’local’ BWs or legacy systems with respect to harmonization determines the
integration efforts at global BW site:
1. ‘Local’ BWs are to be integrated
I. It is the task of the ’global’ BW to gather data from ‘local’ BWs
(Data values are already ’common coded’, local-global data models are integrated, a
data model architecture exists)
II. It is the task of the ’global’ BW to consolidate and harmonize data (mapping)
(Data values are not ’common coded’, local-global data models are integrated, data
model architecture exists)
III. It is the task of the ’global’ BW to consolidate and harmonize data (mapping) and to
integrate the local-global data models.
(A data model architecture either does not exist, or is not controlled. Statements about
model consistency cannot be made without doing individual checks.)
2. ’Local’ BWs and legacy systems are to be integrated
I. For legacy systems data and data model integration is necessary.
These scenarios copy the different corporate data warehousing strategies. The essential difference
lies in whether ‘local’ and ‘global’ BWs are developed according to common architecture guidelines
(scenario 1.I and 1.II). If this is the case we can talk about an ‘architected outside-in BW landscape’.
This harmonized, architecture-based interaction of local and global BW instances is a serious option
for building a BW based corporate data warehousing strategy from the very beginning. It is supported
by the central development of BW templates for ‘local’ reporting and analysis.
Global BW
Headquarter
Divisional
Global BW
Division A
BW template
roll-out
BW Europe BW Americas BW Asia
Regional
Data Marts ODS Data Marts ODS Data Marts ODS virtual
BW Enterprise
Data Warehouse Data Warehouse Data Warehouse Data Staging
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 139
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 139
However, if dealing with a mature BW landscape, you will probably echo scenario 1.III and scenario
2. There is obviously only a restricted corporate strategy for ‘local’ BWs and it is the task of the
‘global’ BW to take the first step towards a corporate strategy (‘catch the BWs scenario’).
BW
BW BW
BW others
local local others
local local
2
InfoProvider: InfoCube, ODS objects; MultiProvider: Union of InfoCubes and/ or ODS objects.
M(odified) 11 44
A(ctive) 11 11
D(elivery) 11 11 11
standardized
source for corporate defined
corporate reporting local reporting
distilled corporate
local defined
local local reporting
Local BW
Data Structures
BW-KEY
0MATERIAL Date Quantity
M1
S2
20020601
20020601
100
200
?
InfoCube Fact-table
* the technical BW implementation is more sophisticated - the key issue exist anyhow
3
With restricted BW implementations data harmonization is often not a central issue. You may
therefore only come up against the data harmonization task later when the BW is implemented as an
enterprise-wide data warehouse solution.
There are several ways in which this situation can be controlled with BW. For one, by using name
spaces different data models can be maintained in BW for the different operative systems that cannot
be integrated. Data stores are then separated automatically too.
This approach is not recommended for an enterprise, however, it is an option for application centers
(which offer BW services for different enterprises).
As separating data using different data models is only advisable in odd cases, data should actually be
separated in data structures defined by a common data model. This is done by extending the key
structure of the BW InfoObject affected using a ‘separator’. In the most straightforward of cases this
separator will contain origin information (source systems).
BW supports separation through concatenation and compounding.
Concatenation
BW-Key
U1 Separator U2
Source-Value
004711 AUDI 004711 BMW
Source-System 1 Source-System 2
Global and
consolidated views
0PRODUCT on products,
independent from
source component
DataSources
R/3 Material R/3 Service EBP Product CRM Product
Source Components
individual data
R/3
R/3 R/3
R/3 EBP
R/3 CRM
R/3
R/3 R/3 R/3 R/3 models
This obviously does not resolve the challenge of making a common coded key available for the
consolidated view 0PRODUCT. SAP MDM functionalities offer support in this respect.
Content Consolidation
MDM
MDM Core Process Steps:
Client
SAP 1. Loading identifying attributes of
master data objects as they were
maintained in their local
Client applications (clients)
non SAP
Load
Client
non SAP ?
2. Matching of uploaded master data
Matching objects to identify duplicates
Client
SAP
= 3. Providing ID Mapping Information
ID Mapping based on matching results for
subsequent usage (I.e. in
Client Business Intelligence)
non SAP
The mapping information (group ID) is then a navigation attribute of the InfoObject to be harmonized
in BW (see Illustration 37: Navigation attributes support the flexibility of BW solutions). This can also
be added afterwards without interfering with the existing schemata.
4
Integration between BW and MDM will be offered as of Release BW 3.5.
SAP MDM supports the model consolidation scenario described in 7.2 (Data Model Integration)
through ’master data harmonization’ by means of consolidated InfoObjects for an analysis that spans
components.
Keys of different InfoObjects are checked in MDM for commonalities, mapped to a group ID, and
maintained centrally as attributes that span InfoObjects.
MDM
MDM Master data harmonization
Client
SAP Core Process Steps:
1. Central Creation of master data
Local
+ Creation
objects just covering the global
information of the master data object
Client
non SAP 2. Local Creation within the master data
Central
Local environment of the application
Creation
Completion system (client) and distributing their
1.234.237 € global information to MDM
=? 6.674.288 €
737.108 €
identify duplicates and result in
+ ID Mapping = 13.282.401 €
ID Mapping between them
Client 4. Distribution of global master data
SAP information to the various clients
5. Local Completion of master data
Distribution within the local environment
Client 6. Use of ID Mapping information for
non SAP Analytics MDM Analytics like business-wide
analyses etc.
Description of a master data object:
Globally relevant Data
SAP AG 2003, Corporate Data Warehousing based on SAP BW, J. Haupt 46(Client) specific Data
SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 46
As well as active master data management, MDM supports key consolidation and complex master
data harmonization through deferred (asynchronous) consolidation. The flexibility of the conformed
dimension in BW allows the ’extended star schemas’ to be subsequently enhanced with information
from the MDM, without interfering with existing solutions.
8 Summary
Important elements of a corporate BW strategy are architecture, consistent application development
and data integration concepts.
A corporate BW architecture is essential to mid- and long-term success. An architecture of this kind
manifests itself in design and procedural guidelines that are valid enterprise-wide, and in their control.
BW supports various aspects of the architecture through a multitude of functionalities that secure
consistency. It is up to each individual enterprise to determine how and to what extent these
functionalities are used.
A central development of BW application templates is an adequate way to support a corporate wide
BW architecture and harmonization.
Integration of data is an ongoing challenge. BW is tightly integrated with SAP’s MDM and offers in
addition various design options to meet integration requirements. This has to be considered in a BW
strategy.
Coming to the end we want to repeat once more the main statement of this paper:
The top priority of any corporate BW strategy must always be to control redundancy. This is
true for each individual BW, and for the interaction of all the BWs within an enterprise!
Table of Illustrations
Illustration 1: Evolution of the SAP BW ................................................................................................ 2
Illustration 2: Redundancy and Solution focus ..................................................................................... 3
Illustration 2: Redundancy and multiple BW landscape ....................................................................... 4
Illustration 3: The conceptual layer architecture of data warehousing ................................................. 7
Illustration 4: The BW extended star schema .................................................................................... 11
Illustration 5: Horizontal consistency in the BW architected data mart layer ..................................... 11
Illustration 6: BW and slowly changing dimensions ........................................................................... 12
Illustration 7: Persistent and virtual multidimensional structures in BW............................................. 13
Illustration 8: SAP BW data warehouse layer..................................................................................... 14
Illustration 9: BW data warehouse layer and vertical consistency ..................................................... 16
Illustration 10: Example - BW data warehouse: Historization of master data .................................... 17
Illustration 11: Example - BW data warehouse: Transaction data ..................................................... 18
Illustration 12: BW staging.................................................................................................................. 19
Illustration 13: Classic data warehousing ........................................................................................... 20
Illustration 14: Classic data warehousing supports operatives reporting........................................... 21
Illustration 15: BW operational data store: Near real time data warehousing.................................... 22
Illustration 16: BW and operational / real time reporting .................................................................... 23
Illustration 17: Operational data model and data warehouse data model......................................... 24
Illustration 18: Data warehouse data models and the situation for BW customers............................ 25
Illustration 19: Enterprise data model and enhanceable BW data model: InfoObjects...................... 25
Illustration 20: BW data model: InfoSources and subject areas ....................................................... 26
Illustration 21: BW data model: data mart layer data model ............................................................. 27
Illustration 22: BW data model: Data warehouse layer data model .................................................. 28
Illustration 23: The ideal corporate BW landscape?........................................................................... 29
Illustration 24: Enterprise-wide strategies and BW architecture ........................................................ 30
Illustration 25: BW landscape and availability / maintenance ............................................................ 31
Illustration 26: BW as Corporate Information Factory ........................................................................ 32
Illustration 27: Scalability of a BW Corporate Information Factory I................................................... 33
Illustration 28: Scalability of a BW Corporate Information Factory II.................................................. 34
Illustration 29: ‘Local - Global’ BW Landscape................................................................................... 35
Illustration 30: Architected Multiple BW Landscape and BW Template roll-out................................. 36
Illustration 31: Global BW as corporate Integrator ............................................................................. 37
Illustration 32: BW Customer Content ................................................................................................ 39
Illustration 33: BW Customer Content ................................................................................................ 39
Illustration 34: Source system key as BW key? ................................................................................. 41