SAP Data Intelligence / SAP Datasphere - ABAP Integration

Planning to use ABAP integration functionality in SAP Data Intelligence / SAP Datasphere and want to get an overview of the things you should know before you start.

Learning more about ABAP-based SAP data integration functionality in SAP Data Intelligence / SAP Datasphere.

Table of Contents:

1. Functionality Overview and Scenarios 2. Recommended System Landscape 3. Important Considerations 4.?Installation and Upgrade aspects 5. Release information 6.?Corrections 7. Additional Information

1. Functionality Overview and Scenarios

"SAP Data Intelligence - ABAP Integration" and "SAP Datasphere - ABAP Integration" are the generic terms for scenarios involving SAP Data Intelligence (formerly SAP Data Hub) or SAP Datasphere and ABAP-based SAP systems.

For SAP Data Intelligence there are three use cases:

1. Accessing the data from an ABAP-based SAP systems. In general, the supported data sources are:CDS Views - available when using an SAP S/4HANA system as the sourceTables - available when using SAP LT Replication Server (SLT) as the source (Through SLT most of SAP NetWeaver based systems can be used as a data source - for more information about SLT in general see SAP Note 1605140)ODP - available in ABAP-based systems that have the ODP framework (context SAPI and BW)
2. Accessing the metadata of supported objects
3. Using operators based on the ABAP language which will be executed in the connected ABAP-based SAP system. In addition to predefined operators, it is possible to implement your own custom operator.

For SAP Datasphere there are two use cases:

1. Accessing the data from an ABAP-based SAP systems. In general, the supported data sources are:CDS Views - available when using an SAP S/4HANA system as the sourceTables - available when using SAP LT Replication Server (SLT) as the source (Through SLT most of SAP NetWeaver based systems can be used as a data source - for more information about SLT in general see SAP Note 1605140)ODP - available in ABAP-based systems that have the ODP framework?(context SAPI and BW)
2. Accessing the metadata of supported objects

Which of these use cases are available and what kind of features are offered for the specific use case depends on the version of SAP Data Intelligence / SAP Datasphere as well as the version of the connected ABAP-based SAP system. A "recent"?version of the DMIS add-on is an SP level which was released in the last two years is highly recommended. For example if the current release of DMIS 2018 is SP11 - hence the "recent" versions are DMIS 2018 SP08-SP11. When you are starting a new project, you should never use an older version.

Accessing the data from ABAP-based SAP systems

There are three different features for SAP Data Intelligence which allow the access to data from ABAP-based SAP systems. If your ABAP-based SAP system meets the system requirements but you are not able to see / use the feature in SAP Data Intelligence, you may need to upgrade SAP Data Intelligence. These features are:

• Operators for Generation 1 Graphs:CDS Reader OperatorAvailable starting with SAP S/4HANA 1909 (SAP Basis 7.54)Available when using SAP S/4HANA CloudSLT Connector OperatorAvailable when using a recent version of add-on DMIS 2018 in the SLT system or?a recent version of add-on?DMIS 2020 in the SLT systemAvailable when using a recent version of add-on DMIS 2011 in the SLT system with SAP NetWeaver 7.00 or higher (Note that using DMIS 2011 is no longer recommended anymore and DMIS 2018 / DMIS 2020 should be used instead when possible)Available when using SLT as part of S/4HANA 1909 or higher (Attention: using SLT in S/4HANA is not recommended and DMIS 2018 / DMIS 2020 should be used instead whenever possible)ODP Reader OperatorAvailable when using?a recent version of add-on DMIS 2018?or?a recent version of add-on DMIS 2020Available when using a recent version of add-on DMIS 2011 on a system with SAP NetWeaver 7.00 or higher and with SAP_BW component 7.30 or higherTo make use of the search help for the ODP object name, check the requirements for "Metadata of ODP objects" below
• Operator for Generation 2 Graphs:Read Data from SAP System OperatorAccessing CDS view data starting SAP S/4HANA 1909Accessing CDS view data starting SAP S/4HANA Cloud 2202Accessing data stored in tables by using SLT when using an SLT system with DMIS 2018 SP06 / DMIS 2020 SP03 or higherAccessing data exposed via the ODP framework starting SAP S/4HANA 1909 or DMIS 2011 SP23 / DMIS 2018 SP08 / DMIS 2020 SP04 or higher
• Replication Flows (Replication Management Service (RMS)):Accessing CDS view data starting SAP S/4HANA 1909Accessing data stored in tables by using SLT when using an SLT system with DMIS 2018 SP06 / DMIS 2020 SP03 or higherAccessing data exposed via the ODP framework starting SAP S/4HANA 1909 or DMIS 2011 SP23 / DMIS 2018 SP08 / DMIS 2020 SP04 or higherAlso see SAP Note 3223810 - Known Limitations and important considerations for Replication Flows

There are two different features for SAP Datasphere which allow the access to data from ABAP-based SAP systems. These features are:

• Dataflows (Generation 1 Operators):CDS Reader OperatorAvailable starting with SAP S/4HANA 1909 (SAP Basis 7.54)Available when using SAP S/4HANA CloudSLT Connector OperatorAvailable when using a recent version of add-on DMIS 2018 in the SLT system or using a recent version of add-on DMIS 2020 in the SLT systemAvailable when using a recent version of add-on DMIS 2011 in the SLT system with SAP NetWeaver 7.00 or higher (Note that using DMIS 2011 is no longer recommended anymore and DMIS 2018 / DMIS 2020 should be used instead when possible)Available when using SLT as part of S/4HANA 2021 or higher (Attention: using SLT in S/4HANA is not recommended and DMIS 2018 / DMIS 2020 should be used instead whenever possible)
• Replication Flows?(Replication Management Service (RMS)):Accessing CDS view data starting SAP S/4HANA 1909Accessing data stored in tables by using SLT when using an SLT system with DMIS 2018 SP06 / DMIS 2020 SP03 or higherAccessing data exposed via the ODP framework starting SAP S/4HANA 1909 or DMIS 2011 SP23 / DMIS 2018 SP08 / DMIS 2020 SP04 or higherAlso see SAP Note?3297105 - Important considerations for SAP Datasphere Replication Flows

Accessing the Metadata of Supported Objects

Provide metadata of objects in the connected ABAP-based SAP system:

• Metadata of CDS ViewsAvailable starting with SAP S/4HANA 1909 (SAP Basis 7.54)Available when using SAP S/4HANA CloudSupported actionsBrowseFact SheetPreviewProfile (starting SAP S/4HANA 2022)Publish
• Metadata of tablesAvailable when using a recent version of add-on DMIS 2011 on a system?with SAP NetWeaver 7.00 or higher?/ a recent version of add-on DMIS 2018 / a recent version of add-on DMIS 2020Supported actionsBrowseFact SheetPreviewPublish
• Metadata of SLT ConfigurationsAvailable when using?a recent version of add-on?DMIS 2011 on a system?with SAP NetWeaver 7.00 or higher?/?a recent version of add-on?DMIS 2018 /?a recent version of add-on?DMIS 2020 in the SLT systemBrowseFact SheetPreview (starting DMIS 2018 SP08 / DMIS 2020 SP04 on the SLT system)Publish
• Metadata of ODP objectsRequires?SAP_BW component version 7.55?or higherAvailable when using the add-on DMIS 2018 SP06 or higher / DMIS 2020 SP03 or higherSupported ActionsBrowseFact SheetPreviewPublish

Using Operators Based on the ABAP language

When working with Generation 1 graphs in SAP Data Intelligence, there are - besides the operators used for accessing data - additional operators available:

• ABAP Converter Operator - Used to convert the data output into a string
• Custom ABAP Operator - Used to access and execute custom logic implemented by using a BAdI

The operators are available starting with SAP S/4HANA 1909 (SAP Basis 7.54) /?DMIS 2011 SP17 or higher on a system?with SAP NetWeaver 7.00 or higher?/?DMIS 2018 SP02 or higher / DMIS 2020.

Note: The Custom ABAP Operator is not available when using SAP S/4HANA Cloud.

2. Recommended System Landscape

In SAP Data Intelligence, we recommend that you use the different approaches for replicating data in the order listed below. You should only use a subsequent option if you either need to customize the data transfer in a more detailed way, or if the target system is not yet supported by one of the previous approaches.

1. Replication Flows
2. Data Intelligence Graphs using Generation 2 Operators
3. Data Intelligence Graphs using Generation 1 Operators

Based on the different data sources, you may also have different options for replicating the data. The following recommendations apply:

1. If your source system is an SAP S/4HANA system offering the needed data by means of CDS views, directly connecting SAP Data Intelligence to this source system and using the respective CDS views is the best approach.
2. If the source system does not use CDS views, you may have the option to directly access the tables that store the data or there may be options to use objects provided by the ODP framework.For ODP objects, directly connect SAP Data Intelligence to the source system that contains the add-on DMIS 2020, DMIS 2018, or DMIS 2011. In case you are connecting to an SAP S/4HANA system (we recommend using CDS Views instead of ODP when possible) the available functionality for accessing ODP may be limited depending on the S/4HANA version.For tables, the recommended approach is to access the source by using a standalone SAP LT Replication Server system using SAP LT Replication Server for SAP S/4HANA 1.0 (addon DMIS 2020) or SAP LT Replication Server 3.0 (addon DMIS 2018). While technically possible, it is not recommended to use SAP LT Replication Server as part of an SAP S/4HANA system or with the version SAP LT Replication Server 2.0 (addon DMIS 2011). The reasons for this are as follows: - The SAP LT Replication Server version embedded in SAP S/4HANA cannot be upgraded independently of SAP S/4HANA. This means that if new functionality is needed, an upgrade of the whole SAP S/4HANA system is required. Due to different dependencies, it is also not possible to downport new functionality to older SAP S/4HANA versions. - SAP LT Replication Server 2.0 is not recommended as it still uses SAP NetWeaver 7.00 as the basis component. This version lacks important features of later ABAP versions for a stable connection and the full feature support for SAP Data Intelligence. SAP LT Replication Sever 2.0 is therefore no longer updated with new functionality like RMS or support for Generation 2 operators.

In?SAP Datasphere, we recommend that you use the different approaches for replicating data in the order listed below. You should only use a subsequent option if you either need to customize the data transfer in a more detailed way, or if the target system is not yet supported by one of the previous approaches.

1. Replication Flows
2. Dataflows (Generation 1 Operators)

Based on the different data sources, you may also have different options for replicating the data. The following recommendations apply:

1. If your source system is an SAP S/4HANA system offering the needed data by means of CDS views, directly connecting SAP Datasphere to this source system and using the respective CDS views is the best approach.
2. If the source system does not use CDS views, you may have the option to directly access the tables that store the data or there may be options to use objects provided by the ODP framework.For ODP objects, directly connect SAP Datasphere to the source system that contains the add-on DMIS 2020, DMIS 2018, or DMIS 2011. In case you are connecting to an SAP S/4HANA system (we recommend using CDS Views instead of ODP when possible) the available functionality for accessing ODP may be limited depending on the S/4HANA version.For tables, the recommended approach is to access the source by using a?standalone?SAP LT Replication Server system using SAP LT Replication Server for SAP S/4HANA 1.0 (addon DMIS 2020) or SAP LT Replication Server 3.0 (addon DMIS 2018). While technically possible, it is?not recommended?to use SAP LT Replication Server as part of an SAP S/4HANA system or with the version SAP LT Replication Server 2.0 (addon DMIS 2011). The reasons for this are as follows: - The SAP LT Replication Server version embedded in SAP S/4HANA cannot be upgraded independently of SAP S/4HANA. This means that if new functionality is needed, an upgrade of the whole SAP S/4HANA system is required. Due to different dependencies, it is also not possible to downport new functionality to older SAP S/4HANA versions. - SAP LT Replication Server 2.0 is not recommended as it still uses SAP NetWeaver 7.00 as the basis component. This version lacks important features of later ABAP versions for a stable connection and the full feature support for SAP Data Intelligence. SAP LT Replication Sever 2.0 is therefore no longer updated with new functionality like RMS or support for Generation 2 operators.

3. Important Considerations

Performance

• When you are using Generation 1 or Generation 2 operators for accessing the data, it is currently not possible to transfer data of a single source object in parallel.
• When you are using Replication Flows (RMS) for accessing the data, the initial load parallelization is limited to the maximum amount of available sessions on the source system. Additionally, delta data of a single object cannot be transferred in parallel (no parallel replication). Further information on optimizing the performance when using Replication Flows (RMS) can be found in SAP Note "3360905 - SAP Data Intelligence / SAP Datasphere - ABAP Integration - Performance when using RMS / Replication Flows".
• If the connected ABAP-based SAP system uses MaxDB, functionality related to metadata lookup might result in a timeout.

Structural changes in the source

• If the structure of a table or CDS view changes during the load / replication of data, the data extraction process will terminate with an error message.
• It is currently not possible to continue after such an error. The data extraction has to be started from the beginning.

Available Filter Options

• When using Generation 1 operators for accessing data stored in CDS Views or ODP it is not possible to apply filters. Filtering is possible when data is being accessed through SLT by setting the filter in the SLT settings (transaction LTRS).
• When using Generation 2 operators, it is generally possible to specify filters. However, there are the following restrictions:It is not possible to filter on columns of RAW or String data type.When using SLT as the source, it is not be possible to specify a filter value exceeding a length of 28 characters on older versions.
• When using Replication Flows (RMS), it is generally possible to specify filters. However, there are the following restrictions:It is not possible to filter on columns of RAW or String data type.When using SLT as the source, it is not be possible to specify a filter value exceeding a length of 28 characters on older versions.

Not supported CDS Views

• CDS View not having the "dataExtraction" annotation are not supported for data transfer.
• Replication of CDS Views (delta transfer) requires the "dataExtraction" and "changeDataCapture" annotations (see help.sap.com).
• CDS Views having parameters are not supported.
• CDS Views making use of a table function are not supported.
• CDS Views including tables with a client field as key field that are not client dependent (e.g. T000)
• Important:?In case there is no standard CDS view available for the data you would want to access, reach out to the respective S/4HANA product team and fill a feature request.If a standard view is not available / will not be made available for the S/4HANA version you are using, there is the option to build your own CDS Views (custom CDS Views). The implementation of such CDS Views is customer specific (consulting) and will not be supported via standard support.
• CDS Views having more than 2 billion records in the result set:Due to the internal memory constrains in SAP HANA, a result set of a query cannot contain more than 2 billion records. As the access plan calculation needs to execute a respective select to determine the boundaries of an access plan, it will fail when there are more than 2 billion records returned.Workaround -?Create CDS Views which partition the original data into parts which are less than 2 billion each and either replicate into the same target table or join the individual target objects.A future version will include an out-of-the-box solution in case the underlying tables of the CDS View are range partitioned. In all other cases, there will be the option to manually define ranges (instead of creating multiple CDS Views).

Using SAP LT Replication Server to access data

• In case ABAP-based source systems should be accessed through SLT, there are dependencies on the SAP NetWeaver Release of the SLT system.?The reason is that the ABAP data types used by the tables which should be replicated from the source through SLT to Data Intelligence also need to be present in the SLT system.?If your SLT system is not on an SAP NetWeaver release which supports the data types of the table you want to replicate, the replication of this table is not possible. For more information see SAP Note?1605140.
• When you connect to SLT, make sure that all jobs of SLT only run in a single client and that the RFC connection from SAP Data Intelligence only accesses this client.
• When you are accessing the data of tables from an SAP S/4HANA source system via SAP LT Replication Server, tables with redirect views (replacement objects) are not supported (for example MBEW, MARD, COEP).
• Declustering of INDX-like tables is only supported for table STXL. Others (e.g. PCL1, ...) are not supported.

Access to a Non-Unicode system

• When working with RFC connected source systems, the RFC connection will handle the needed conversion of data if the source system only uses a single code page.
• MDMP systems are not supported. If you still try to use SAP Data Intelligence / SAP Datasphere -?ABAP Integration?in an MDMP system, texts may be displayed incorrectly and data may be replicated incorrectly. For general guidance regarding MDMP, see SAP Note?79991.

Using S/4HANA Foundation system as the source

• When using SAP Data Intelligence / SAP Dataspehre - ABAP Integration in a system that has SAP S/4HANA Foundation and the DMIS add-on, access to CDS Views is not available / supported.

Accessing data provided via the ODP framework

• Minimum required SAP_BW version in the system:SAP_BW 7.3 SP 14 (see SAP Note 1931427 - ODP Data Replication API 2.0)BW context: Infocube requires SAP_BW >=7.4 SP16
• Special aspect regarding the transfer with Replication Flows:BW context: Infocube is not supported for SAP_BW >= 7.54 SP 01 and for earlier releases according to the support packages listed in SAP Note 2834352. Reason: No information about key fields is exposed.

4.?Installation and Upgrade aspects

Chapter 1 of this SAP note already lists the minimum releases for each use case / functionality. In summary, the ABAP Integration is available:

• On SAP S/4HANA Cloud
• On SAP S/4HANA On-Premise systems starting with SAP S/4HANA 1909
• On SAP NetWeaver-based systems when using the DMIS add-on starting with DMIS 2011 SP17 / DMIS 2018 SP02 / DMIS 2020

It is recommended to always use the latest available version of SAP S/4HANA and the DMIS add-on where possible. Especially for the DMIS addon the available functionality when using DMIS 2011 is very limited and therefore DMIS 2018 / DMIS 2020 should be used.

In addition, several SAP Notes need to be implemented in the systems. Some functionality is only available after implementing a respective transport-based correction instruction (TCI) in the system. Ensure that you check the release notes listed in chapter 4 of this SAP Note regularly for updates on available corrections or use the Note Analyzer listed in chapter 5.

In order to enable certain ODP functionality, the following SAP Notes may also be required:

• 2775549 - Release-independent ODP interface for SAP Data Hub
• 3069623 - Metadata extraction of ODP BW objects in Data Intelligence
• 3069621 - Metadata extraction of ODP DataSources in Data Intelligence

Note: In case of any issues, first check for missing SAP Notes and implement the missing SAP Notes. If the issue persists, open an incident using component CA-DI-IS-ABA. We recommend that you keep your system up-to-date by regularly checking for new SAP Notes; the system will be more stable and incident processing will be much faster.

Additional Information

• Security: For an overview of security topics related to the ABAP Integration see SAP Note?3100673 (old versions:?2981615?/2831756).
• Connection Setup: To connect to ABAP-based SAP systems, you use the special connection type "ABAP" in SAP Data Intelligence - see SAP Note?2835207.
• Data Retention Settings: 3088317 - SAP Data Intelligence - ABAP Integration – Settings for Retention Period
• General Settings, e.g. configuring the maximum amount of parallel sessions: 3044005 - SAP Data Intelligence - ABAP Integration - General Parameters (DHBAS_CONFIG / LTBAS_CONFIG)
• Job Settings: 3223735 - SAP Data Intelligence - ABAP Integration - Job Settings (Transaction DHCDCSTG / Table DHCDC_JOBSTG)
• RMS:?3223810 - Known Limitations and important considerations for Replication Flows
• Performance Optimization:?3360905 - SAP Data Intelligence / SAP Datasphere - ABAP Integration - Performance when using RMS / Replication Flows