Use Semantic Groups in DTPs for Flexible

Splitting of Data into Separate Packages

by Gary Nolan, SAP BI Strategic Architect, Sapiex Associates LLC
March 19, 2012
Share on facebookShare on twitterShare on emailShare on printMore Sharing Services
Avoid inconsistent or unexpected results in data loads by ensuring that data that needs to be
grouped stays together in data packages during loading.
Key Concept
You can use the semantic key to aid in grouping data during loading to ensure like data is processed
and loaded in the same data package.
When loading data into an InfoProvider, an SAP system groups the data into packages in order to load
the data into the source. The size and grouping of the packages are dictated by the system at the time of
extraction. The extractor uses the packet size and source data structure to assemble the data into
coherent loading packets in order to group the source data. The extraction is executed in individual
packages rather than in one large full data package because large data volumes can overwhelm system
resources if the system tries to extract large data volumes as one unit.
In most data loads, the contents and grouping of the data into packages is not relevant. The system
chooses the first package, loads this package, then the next. The contents of the packages are based on
the contents of the source data. The system is careful to cut off the data when the package size limit is
reached. In order to see the package size, double click on the data transfer process (DTP) and go to the
Extraction tab. (Figure 1).

Figure 1
The Extraction tab in the DTP that shows the package size for the process

In some cases, however, the contents of the specific packages are relevant. This is typically because
there is some start routine coding involved in the transformation that loops through the source data. Since
the system does not bring in the entire data set at once, the start routine is run for each package that is
loaded. Therefore, if data is split across packages, the results of the start routine can be inconsistent.
For example, a flag is needed in the source to note if any material on the sales order comes from division
10. This flag requires a looping through all the lines on the order to determine if any lines are division 10.
If so, all lines are flagged with a division 10 indicator. Based on the way the system gathers the data from
the source, there is no assurance that all the items from the sales order will be loaded into one package.
Thus, if the start routine ABAP code loops through all the records to determine if any are division 10, and
some of the lines are split into other packages, you could get inconsistent results. This is because not all
the data from the sales order line is contained in this package. The sales order lines for one order could
be split among multiple packages.
The best way to ensure that all the relevant data is contained in one package is to use the semantic
groups in the DTP. This option allows you to choose the key fields for the extraction so the data packages
are grouped. Note the semantic key does not need to correspond to the key in either the source or
destination InfoProvider.
To set the semantic key, click the Semantic Groups button in the Extraction tab of the DTP (Figure 2). If
semantic groups have been chosen, a green checkmark is shown next to the button to indicate the
presence of the semantic group.

Figure 2
Select the Semantic Groups button to choose fields

Once you click the Semantic Groups button, the system provides you with the list of characteristics from
the source (Figure 3). You can select one or many fields to use. The system uses this grouping any time
this DTP is used to load the data. In this case, by choosing the sales document and item, you are assured
that the system always has all order lines for one order in the same package. They will not be split among
multiple packages.

Figure 3
List of possible fields to select when you click the Semantic Groups button

The system also uses the semantic key to set the key structure of the error log. If error handling is turned
on via the option on the update screen of the DTP, the system shows the error log using the semantic
key. This means that any errors are grouped and displayed based on the semantic key when looking at
the error log.
The error log also works without choosing a semantic key. If no semantic key is chosen, the system
shows the error log grouped by the package order. If a semantic group is chosen, it uses that key
structure to show the error log.
Performance can be affected with the addition of a semantic key to the DTP. This is because the system
must read and gather the data in a specific key set. This can slow performance, especially if no index
exists on the selected field grouping in the source InfoProvider. Thus, unless there is a specific reason to
use the semantic group (i.e., a start routine that needs the data grouped), do not include it in the DTP
because it does not provide any benefit, but slows the loading of the data.