SAP BW on HANA & HANA Smart Data Access

BEx Query Execution

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION

TABLE OF CONTENTS
DOCUMENT VERSION ............................................................................................................................... 3
DESCRIPTION ............................................................................................................................................ 3
WHERE ARE CROSS-SYSTEM-JOINS EXECUTED? ................................................................................ 5
WHAT IS A SEMI-JOIN? ............................................................................................................................. 5
HOW ABOUT A SEMI-JOIN EXAMPLE? .................................................................................................... 6
WHAT IS JOIN RELOCATION? .................................................................................................................11
HOW ABOUT A JOIN RELOCATION EXAMPLE? .....................................................................................11
WHICH OPTIMIZATION TYPES ARE AVAILABLE FOR WHICH KIND OF REMOTE SOURCE? ..............16
PERFORMANCE - CLOSING REMARK.....................................................................................................17

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION

Document history
DOCUMENT VERSION

DESCRIPTION

DATE

1.0
1.1

First Version
Version with the following updates:
What is a semi-join?:Picture 1, steps, statement.
Optimization types per remote source.
Closing remarks.

14.03.2014
12.08.2014

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
This article describes how BEx queries are executed on remote sources as of SAP BW 7.40 SP5 and SAP
HANA SP7.
The following analysis and tracing options and tools help to understand how a query is executed and how to
analyze potential performance problems.
(1) Query SQL
When a query is executed in the BW backend with transaction RSRT it is possible to display the SQL/
HANA Query which is sent to the HANA database (see debug options).
(2) Query statistics
When a query is executed in the BW backend with transaction RSRT and statistics-display option is
selected, the BW statistics display the access to the Virtual Table as any other BW Provider with the
runtime and number of records selected/transferred.
(3) Federation Trace
In the indexserver.ini [section trace] federation traces can be activated. For analysis of a query
execution the parameter federationexecution and fedtrace can be set to debug. Then the federation
execution is traced with details which can be found in the indexserver trace. Please switch-off the traces
again after executing the query in order to prevent slowing down the HANA system through massive
tracing.
(4) Explain Plan
In the SQL console of the HANA studio the SQL/HANA Query (see (1)) can be executed with option
Explain plan. It shows the execution steps, details like the sql statement sent to the remote source and
information on optimizations as well as the engine where the step has been executed.
(5) Visualize Plan
In the SQL console of the HANA studio the SQL/HANA Query (see (1)) can be executed with option
Visualize plan. It shows the execution in a plan graph in a good overview and e.g. with more details
about the join parts.
(6) Provisioning
In the HANA Studio under Folder Provisioning the executed statements on the remote source database
can be found including information on runtime and result rows.

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
WHERE ARE CROSS-SYSTEM-JOINS EXECUTED?
Inside HANA the Federation Layer component has been introduced which provides a uniform interface
regardless of the type of remote source. The connection to a remote source is handled via a Virtual Table in
HANA which is treated as any other HANA table (Some restrictions on Virtual Tables apply, e.g. they cannot
be included in Analytic or Attribute Views).
When executing a JOIN between a local and a remote table a general problem in federation is if it is better
from a performance perspective to fetch the data from the remote source and transfer it to the local HANA or
to transfer the data to the remote source in order to execute the join there. This is decided by the sql
optimizer depending on estimations based on statistics. In general as much SQL functionality as possible is
pushed to the remote source in order to reduce the amount of data to be moved across the network to
HANA.
Currently there are two main optimization methods available. These are the so-called semi- join and its
enhancement, the join relocation. The goal of both is the reduction of the data transfer between the remote
source and the local HANA.
WHAT IS A SEMI-JOIN?
The semi-join calculates the share of a natural join, which remains after reduction on the left relation (see
WIKIPEDIA).
On the table in the local HANA matching values (from the join condition) are selected and filters applied.
These values are shipped to the remote source. In order to reduce the data transfer from HANA to the
remote source table only distinct values are sent. The result of this selection (reduction) is sent back to the
local HANA and the join is executed inside HANA.

Local
HANA

Remote
Source

/BIC/YSHSTORE

CH05

/BIC/YSHSTORE

SID

STORE

COSTWT

[]

[]

[]

[]

CH04

15

CH04

100

CH05

16

CH05

200

CH06

17

CH06

300

[]

[]

[]

[]

Reduce Table 2 by eliminating tuples

Filter on SID = 16

Reduce

where attribute /BIC/YSHSTORE is not

matching any value in Table 1

STORE

COSTWT

CH05

200

Picture 1: Execution of semi-join

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
HOW ABOUT A SEMI-JOIN EXAMPLE?
A query is executed on a CompositeProvider containing an Open ODS View of type Virtual Table on remote
source BZH_ld7597 with semi-join optimization. We are selecting key figure COSTWT values for
characteristics named STORE and CURRENCY from the Virtual Table called /BIC/EXSB_01S_BZH. There
is a filter applied on STORE selecting one characteristic value. The SQL/BW HANA query looks like this:
Query execution in BW

Picture 2: BEx Query execution with execute&debug and display SQL/HANA Query

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION

Picture 3: BEx Query result

The BW statistics show the execution on the CompositeProvider (column InfoProvider XSB_02S_COPR)
and the Open ODS View in column (Basic InfoProvider XSB_01S_BZH).

Picture 4: BEx Query statistics Aggregation Layer

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
The steps in this example in detail are:
(1) Filter SID = 16 is applied on local HANA table BIC/SYSHSTORE
(2) Distinct values of CURRENCY are determined
(3) Distinct values of STORE are determined
(4) Unique values for STORE are sent to the remote source via the Virtual Table
(5) It is estimated that CURRENCY does not reduce the result set because there is no filter to apply
(6) It is estimated that STORE reduces the result set because only a single value is joined. In the federation
trace this is shown: Results for reducer STORE: 1 (see FEDTRACE_SEMIJOIN.txt below)
(7) The query is executed with semi-join execution strategies applied in the remote source (see Federation
trace extract below):
Execute BZH_ld7597:
SELECT
SQ.*
FROM (SELECT
"W1"."STORE" AS "STORE",
"W1"."DOC_CURRENCY" AS "DOC_CURRENCY",
SUM("W1"."COSTWT") AS COL0
FROM
"SAPKIT"."YSB_GEN_TABLE2" "W1"
GROUP BY
"W1"."STORE",
"W1"."DOC_CURRENCY" ) SQ
WHERE SQ."STORE" IN ('CH05')
Such a strategy can be an IN clause or a temporary table see the configuration in the indexserver.ini
smart_data_access
semi_join_execution_strategies = IT (I stands for IN clause, T stands for
temporary table).

Picture 5: Federation trace extract

There are two further places in the HANA Studio where the information about the semi-join execution
can be found. You can generate the explain plan of the SQL statement and have a look at the operator
details for the remote column scan or go to the Smart Data Access folder in the HANA Studio:

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
(a) Explain Plan

Picture 6: Result of explain plan

The remote column scan with semi-join optimization (see column operator details) is executed on the
remote source (see column EXECUTION_ENGINE = EXTERNAL). All other steps are executed on the
local HANA.
Operator Details in operator column Remote Column Scan show the select statement with the semi-join
optimization:
SELECT

FROM

"W1"."STORE" AS "STORE",
"W1"."DOC_CURRENCY" AS "DOC_CURRENCY",
SUM("W1"."COSTWT") AS COL0

"SAPKIT"."YSB_GEN_TABLE2" "W1"
GROUP BY
"W1"."STORE",
"W1"."DOC_CURRENCY" ,
Semijoin on [STORE,99.76]
(b) Smart Data Access
In the HANA Studio under Provisioning
Smart Data Access, the column SQL Text shows the same
SQL with the semi-join information.
Additionally the column rows shows the number of result rows. With semi-join applied in this example
the number of rows is 1, whereas without semi-join 406 result rows are transferred:
With semi-join

Without semi-join

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
(8) The result is sent back to the local HANA and the join is executed there as shown in the visualized plan:

Picture 7: Visualized plan

Column search detail shows join in HANA

10

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
WHAT IS JOIN RELOCATION?
Joins between a local and a remote table can be relocated to the remote source. In the remote database a
temporary table is created and bulk inserts (with a default bulk size of 65.535 rows) take place (currently not
in parallel). The temporary table is dropped after the query execution has finished. In the remote source the
join is executed and returns the result set back to the local system. This can avoid a significant amount of
network traffic. But: the query performance is then dominated by the JOIN-performance of the remote
source.
HOW ABOUT A JOIN RELOCATION EXAMPLE?
Query execution in BW
A query is executed on the same CompositeProvider as above containing another Open ODS View of type
Virtual Table on remote source IQ_SDA with join relocation optimization. Again we are selecting key figure
COSTWT values, this time for characteristics named STORE, PRODUCT and CURRENCY from the Virtual
Table called /BIC/EXSB_01B_IQ. There is a filter applied on STORE selecting one characteristic value. The
SQL/BW HANA query looks like this:

Picture 8: BEx Query execution with execute&debug and display SQL/HANA Query

11

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION

Picture 9: BEx Query result

Picture 10: BEx Query statistics Aggregation Layer

12

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
In this example the query is executed with join relocation applied on the remote source. The main steps in
detail are:
(1) Filters are applied to SID table /BIC/YSYHSTORE (SID = 16). The columns that are needed for the join
are projected. In this example only field /BIC/YSYHSTORE is needed.

Picture 11: Visualized Plan Column Search

13

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
(2) The result set from step (1) is pushed to the remote source as a temporary tables named
#JRT_<number>.

Picture 12: Visualized Plan Remote Push

(3) The join between the temporary table and the fact table is executed in the remote source and afterwards
aggregated as per the GROUP BY clause.

Picture 13: Visualized Plan Join in the remote source

14

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
The SQL statement can be found in the indexserver trace when the debug options (see beginning of this
document) are set (see FEDTRACE_JOINRELOC_EXTRACT.txt):
Execute IQ_SDA:
SELECT

"W1"."PRODUCT",
"W1"."DOC_CURRENCY",
SUM("W1"."COSTWT")
FROM "SAPKIT_SDA"."YSB_50MIO" "W1"
INNER JOIN
(SELECT
IDX_0."c_1" AS V1
FROM
#JRT_5 IDX_0 ) "W11"
ON ("W1"."STORE" = CAST("W11"."V1" AS VARCHAR(40)))
GROUP BY
"W1"."PRODUCT",
"W1"."DOC_CURRENCY"

Picture 14: Federation trace extract

This information can also be found in the explain plan as well as in the smart data access folder see the
explanations for the semi-join optimization. Also the visualized plan gives hints about the join relocation
optimization (Remote Push and Inner Join) see picture 15.

15

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
(4) After the join in the remote source has been executed, the reduced result set is transferred back to
HANA. Here the join with master data PRODUCT, CURRENCY and the reduced facts for STORE takes
place.

Picture 15: Visualized Plan Final join in the local HANA

WHICH OPTIMIZATION TYPES ARE AVAILABLE FOR WHICH KIND OF REMOTE SOURCE?
The semi-join optimization is available for the most important remote sources, such as HANA, SAP IQ, MS
SQL, Oracle and Teradata. Further remote sources are gradually added with future HANA SPs.
Join relocation is available for SAP IQ, SAP ASE, Teradata and for HANA (HANA as of Revision 74.01).
Currently the system-wide parameter to force join-relocation is not switched-on per default, but should be
activated to benefit from further query optimization techniques.
For MS SQL Server, IBM DB2 and Oracle join relocation is available as of Revision 81.

16

SAP BW ON HANA & HANA SMART DATA ACCESS BEX QUERY EXECUTION
PERFORMANCE - CLOSING REMARK
A principal problem of federation optimization however cannot be solved with these JOIN optimizations:
Joining two very large tables on a high cardinality column will always require sending one of the tables over
the network, which will most likely dominate the runtime. If this runtime is not acceptable, such problems
need to be addressed by e.g. loading the remote data to HANA.
Below, a very simple example illustrates this conceptual limitation. A master data table contains records of
10 million customers (Table CUSTRATE). This table resides on HANA. The transactional sales data resides
in a remote database and may contain several records for each customer (Table CUSTSALES).
The query below calculates the aggregated revenue on a customer master data attribute:

HANA

SELECT

CUSTRATE
CustID

ABC

C00000001

C00000002

C00000003

CUSTSALES_VIRT

C10000000

CustID

Remote DB

FROM

Revenue

CUSTSALES
CustID

Revenue

C00000001

100

C00000002

120

C00000003

150

C10000000

200

T2.ABC,
SUM(T1.Revenue)

CUSTSALES_VIRT AS T1
JOIN CUSTRATE AS T2
ON
T1.CUSTID = T2.CUSTID
GROUP BY
T2.ABC

The result will only have 3 rows but the JOIN has to be
executed locally on 10 million records.
The CUSTRATE or CUSTSALES table has to be completely
transferred over the network in order to execute the join locally
or in the remote database (JOIN-relocation).

17

www.sap.com

2014 SAP SE. All rights reserved.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP
BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP
products and services mentioned herein as well as their respective
logos are trademarks or registered trademarks of SAP SE in Germany
and other countries.
Business Objects and the Business Objects logo, BusinessObjects,
Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and
other Business Objects products and services mentioned herein as
well as their respective logos are trademarks or registered trademarks
of Business Objects Software Ltd. Business Objects is an SAP
company.
Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL
Anywhere, and other Sybase products and services mentioned herein
as well as their respective logos are trademarks or registered
trademarks of Sybase Inc. Sybase is an SAP company.
Crossgate, m@gic EDDY, B2B 360, and B2B 360 Services are
registered trademarks of Crossgate AG in Germany and other
countries. Crossgate is an SAP company.
All other product and service names mentioned are the trademarks of
their respective companies. Data contained in this document serves
informational purposes only. National product specifications may vary.
These materials are subject to change without notice. These materials
are provided by SAP SE and its affiliated companies ("SAP Group")
for informational purposes only, without representation or warranty of
any kind, and SAP Group shall not be liable for errors or omissions
with respect to the materials. The only warranties for SAP Group
products and services are those that are set forth in the express
warranty statements accompanying such products and services, if
any. Nothing herein should be construed as constituting an additional
warranty.