[et_pb_section fb_built=“1″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n
This blog explains how to use the OpenAPI client in an SAP Data Intelligence data flow. Information about a specific location (point of interest), e.g. geo-coordinates, opening hours, websites are to be obtained with the help of GoogleMaps. It is thus shown how data can be enriched with additional information with the help of APIs.<\/p>\n
This blog follows the following structure:<\/p>\n
[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n
SAP Data Intelligence (formerly Data Hub) is offered as a data management and pipeline solution. Data flows for orchestrating and monitoring data can be created flexibly and offer insights into the entire IT landscape through various connection options. In addition, these data flows can be cleansed, filtered, enriched, operationalized and optimized, leaving a solid database that can be leveraged by ML scenarios.<\/p>\n
In Pipelines, there are many different operators that can perform the functions mentioned above. A large part of the operators is self-explanatory and easy to handle. However, there are other operators that are not so self-explanatory. One such case is the standard operator \"OpenAPI Client\". We will describe this in more detail with the GeoEnrichment use case.<\/p>\n
<\/p>\n
[\/et_pb_text][et_pb_image src=“https:\/\/teststage.bix-consulting.com\/wp-content\/uploads\/2022\/02\/sapdi_1.png“ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][\/et_pb_image][et_pb_text module_id=“anker-sapdiblog1″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n
<\/p>\n
For this blog, the Google Places API (Application Programming Interface) is to be used with the OpenAPI client. This can provide detailed information for a Place Id (Google internal unique ID).[1]<\/a><\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text module_id=“anker-sapdiblog2″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n In order to call a Google API, you first need a so-called API Key, with which you identify yourself for each call and are billed via the Google Account after a certain number of calls.[2],[3]<\/a><\/p>\n In addition, a Place ID must be available. This ID represents a specific location. This can be read with the Google Geocoding API from names and addresses or an example is taken from the documentation. In our case we take a Place ID from the official documentation.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n The structure of the graph takes place as follows:<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_image src=“https:\/\/teststage.bix-consulting.com\/wp-content\/uploads\/2022\/02\/sapdi_2.png“ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text module_id=“anker-sapdiblog4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n The first step is to configure the OpenAPI Client Operator. We select \"manual\" as the Connection Type to be able to maintain it directly. In productive operation, the connection parameters should be stored in the Connection Manager.[4]<\/a> In the configuration area of the OpenAPI Client we have to fill Host, Schemes, Base Path, Method and Produces.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_image src=“https:\/\/teststage.bix-consulting.com\/wp-content\/uploads\/2022\/02\/sapdi_3.png“ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n The filling of these fields can be understood with the documentation of the Google Place API already linked above. The figure below reflects an excerpt of the documentation, from which the areas of the URL with their corresponding meaning can be taken.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_image src=“https:\/\/teststage.bix-consulting.com\/wp-content\/uploads\/2022\/02\/sapdi_4.png“ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][\/et_pb_image][et_pb_text _builder_version=“4.16″ _module_preset=“default“ text_font_size=“10px“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n Abbildung 1: Eigene Darstellung in Anlehnung an Google Places Documentation<\/em><\/span><\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n The \"mandatory field\" API Key Value is not filled directly in the OpenAPI client operator in this example. This is filled in the Python operator so that the input can also be scheduled as a parameter when starting the graph. The API Key Name can be freely chosen and \"header\" must be selected as API Key Type.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text module_id=“anker-sapdiblog5″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n This now leads to the Python operator. This operator does not have to be configured or linked to a Dockerfile, but only adapted in the coding.<\/p>\n The OpenAPI client operator processes an input message. This is created in the code of the Python operator with the help of the \"header\" dictionary. The dictionary contains the API key and the place ID that are passed to the OpenAPI client.<\/p>\n The labels...<\/p>\n header[\u201copenapi.query_params.key\u201c] or follow a certain logic, which can be found in the OpenAPI Client Operator documentation.[5]<\/a><\/p>\n Additional header attributes can be used to pass more information for later processing. For example, we passed user and roles. The additional attributes have no influence on the result of the OpenAPI Client Operator and are therefore only passed on, but not processed.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_image src=“https:\/\/teststage.bix-consulting.com\/wp-content\/uploads\/2022\/02\/sapdi_5.png“ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n After realization of the configuration and implementation, the graph can be executed.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n Once the graph has been executed, wiretaps can be used to view the various processing steps. The Python operator passes a message to the OpenAPI client operator, which was created from a dictionary in Python (see below). This message is the input to the OpenAPI Client Operator. The wiretap illustrates which information is passed and how. Also, it shows what can help with potential problems.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_image src=“https:\/\/teststage.bix-consulting.com\/wp-content\/uploads\/2022\/02\/sapdi_6.png“ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n Result: {\u201cRole\u201c:\u201cConsultant\u201c, <\/em>\u201cUser\u201c:\u201cNordhues\u201d, \u201dopenapi.query_params.key\u201d:\u201dmy_api_key\u201d, \u201dopenapi.query_params.place_id\u201d:\u201dChIJgUbEo8cfqokR5lP9_wh_DaM\u201d}<\/em><\/p>\n \u00a0<\/em><\/p>\n The OpenAPI client processes these results and outputs them in the final wiretap. The figure below shows only a small part of the delivered data.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_image src=“https:\/\/teststage.bix-consulting.com\/wp-content\/uploads\/2022\/02\/sapdi_7.png“ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text module_id=“anker-sapdiblog6″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n This example should be used as an example for further API's. The structure, after which API's are called, often does not differ. Accordingly, this example can also be used for other API's, for example What3Words.[6]<\/a> It remains to be said that API guidelines are to be studied in detail, which defaults there are regarding storage and further processing of the API results.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n Further processing of the output string is more complex if we have a scenario where many API calls have to be made. Each OpenAPI client operator processes only single calls at a time, accordingly more operators need to be added in the graph. In addition, error handling functions must be implemented in case the OpenAPI client operator does not find a result.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n In a scenario where a simple graph is to process multiple records, it is necessary to wait until all records are processed before taking further steps. The Python operator initiates an API call for each record (through the OpenAPI Client operator). The step after the OpenAPI Client Operator is thus started for each row individually. Creativity is now required to ensure that all records have been processed and recollected before further steps follow.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n When using GeoEnrichment API's, you end up with a dataset that has been enriched with geo-data and other information about a specific location. This results in many different use cases. For example, the use of the map functionality in the SAC or distance calculations between different stores.<\/p>\n For example, in the figure below, longitude, latitude, and type of location were read for the three locations. This allows data sets to be built and filtered more granularly, so that sales can be reported by type, for example.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_image src=“https:\/\/teststage.bix-consulting.com\/wp-content\/uploads\/2022\/02\/sapdi_8.png“ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n If you have any questions regarding the conception or implementation of SAP DI, please do not hesitate to contact us.<\/p>\n More information:<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ border_width_bottom=“1px“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n Footnotes<\/em><\/p>\n [\/et_pb_text][et_pb_text module_id=“anker-sapdiblog“ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n 1: https:\/\/developers.google.com\/maps\/documentation\/places\/web-service\/details?hl=de<\/a> (go to section<\/a><\/em><\/span>)<\/p>\n 2: Prices can only be found after logging into the Google Cloud Platform. They vary depending on the API and information requested. (go to section<\/a><\/em><\/span>)<\/p>\n 3: https:\/\/developers.google.com\/maps<\/a> (go to section<\/a><\/em><\/span>)<\/p>\n 4: https:\/\/help.sap.com\/viewer\/b13b5722c8ff4bf9bb097251310031d0\/3.0.2\/en-US\/974be4fbb13a48eda16f7b061508eb59.html<\/a> (go to section<\/a><\/em><\/span>)<\/p>\n 5: https:\/\/help.sap.com\/viewer\/97fce0b6d93e490fadec7e7021e9016e\/Cloud\/en-US\/8a70738566e6466eb8d0f7d68be80247.html<\/a> (go to section<\/a><\/em><\/span>)<\/p>\n 6: https:\/\/developer.what3words.com\/public-api<\/a> (go to section<\/a><\/em><\/span>)<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.16″ _module_preset=“default“ locked=“off“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_column type=“4_4″ _builder_version=“4.16″ _module_preset=“default“ global_colors_info=“{}“ theme_builder_area=“post_content“][et_pb_text _builder_version=“4.16″ _module_preset=“default“ text_font=“Roboto|300|||||||“ text_font_size=“30px“ locked=“off“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\n Contact Person<\/span><\/p>\n [\/et_pb_text][et_pb_image src=“https:\/\/teststage.bix-consulting.com\/wp-content\/uploads\/2021\/05\/Julius_RUND.jpg“ title_text=“Julius_RUND“ align=“center“ _builder_version=“4.16″ _module_preset=“default“ transform_scale=“86%|86%“ transform_translate=“-4px|22px“ locked=“off“ global_colors_info=“{}“ theme_builder_area=“post_content“][\/et_pb_image][et_pb_text _builder_version=“4.16″ _module_preset=“default“ text_font=“Roboto|300|||||||“ text_font_size=“16px“ transform_translate=“1px|-33px“ locked=“off“ global_colors_info=“{}“ theme_builder_area=“post_content“]<\/p>\nRequirements<\/h5>\n
Structure of the graph<\/h5>\n
\n
Calling the OpenAPI Client Operator<\/h5>\n
Python Operator<\/h5>\n
header[\u201copenapi.query_params.place_id\u201c]<\/p>\nExecution<\/h5>\n
[2022-01-29 13:24:15,480] <\/em><\/p>\nOutlook<\/h5>\n
Error handling<\/h5>\n
Package processing<\/h5>\n
Examples and scenarios<\/h5>\n
\n