Leveraging a SaaS Data Platform

Abstract

In today’s IT ecosystem, many companies are moving to SaaS enterprise applications due to the many benefits and efficiencies gained by the organizations (not discussed in this paper). SaaS enterprise applications have a wealth of data and information waiting to be unlocked; however, the impact and tradeoffs to an organization’s data ecosystem must still be evaluated. From a data analytics perspective a major tradeoff is the constraints imposed by the SaaS system on data accessibility and the loss of control of how data is ingested and curated. It is possible to successfully unlock and give “The Data Keys” back to the organization to enable payment order optimization for discounts, vendor price forecasting, purchasing mix optimization, and vendor scoring. This paper discusses a situation where “The Data Keys” were handed back to an organization, after a procurement to pay and invoicing application modernization through a SaaS enterprise application (SAP Ariba Platform) and how the business captured significant value.

1 Introduction

The SAP Ariba Platform is used to digitize a company’s procurement, invoicing, and supply chain processes. When implementing the SAP Ariba Platform organizations are now able to leverage an Analytical Database from Ariba that will give a holistic view of how the company is performing from a procurement to pay perspective. It also gives an organization the ability to track suppliers and how well they are performing with regards to procurement, invoicing, and supply chain. The difficulty in leveraging the Ariba data platform are the constraints enforced due to the nature of it being a SaaS enterprise application. In this paper we explain the constraints we ran into while implementing the external reporting for the Ariba Platform and how we overcame those constraints by using the Ariba APIs to create an external Ariba data warehouse for analytical trending, matching, and data science. This allowed the business to create system health checks to ensure proper data flow throughout the P2P network – enabling early issue identification saving IT and business time on ticket research and enabling batch resolution of issues. The external reporting database was created on Amazon Web Services (AWS) using the Ariba Analytical API with the Spend/Procurement Analysis data sets. This external database allows a business to analyze their Spend and Procurement data without any constraints using their visualization tool of choice. The following use cases were implemented with the Qlik Sense enterprise visualization tool that the business had already acquired:

1. Data Quality and Mining*
   • Duplicate POs
   • Missing Receipts
   • Unit of Measure discrepancy
   • PO Attribute Discrepancies
2. Approver Analysis
3. Supplier Invoice Analysis
4. Plant Invoice Analysis

The relational data model that was created in AWS by Aspirent is a replication of the Ariba dimensional model but indexed to support traditional reporting tools and extended to with additional fields.

Section 3 describes the SaaS constraints from the Ariba platform that prompted the need for the external Ariba database. Section 4 is a description of the Ariba analytical data model with a link to the conceptual model from SAP. Section 5 describes the Ariba APIs, constraints we encountered while building out the analytical solution, and how we the API was implemented. Section 6 is a walk-through of our architecture and technologies used to build out the analytical solution. Section 7 is a conclusion of the analytical solution and the natural next steps we believe are needed in the long-term vision.

*Data quality and mining was completed by comparing to an internal Datawarehouse in AWS that holds all ERP data from internal systems

2 Business Problems Solved

Automation of data acquisition and systematic integrated reporting were the major soft benefits to the organization. This saved the analytics team time and effort and allowed them to now combine data to get new insights. This proved to be difficult in the existing Ariba reporting modules as will be discussed below in section 3.

Please see the following examples of specific problems solved:

• Potential reduction in past due invoices due to the ability to trend data and identify data integration errors
• Improved analytic capabilities due to the removal of limitations within the Ariba Analytical Reporting user interface.
• Reporting to automatically identify data quality issues within Ariba and the external ERP systems
• Increased visibility into active approvers through improved approval workflow auditing

3 Ariba Analytical Reporting Constraints

Ariba has multiple reporting modules available throughout the system. The business used the out of the box reporting from Ariba Network and SAP Ariba Analytical Reports. The data available in the Ariba system was very valuable, but due to the SaaS nature of the Ariba Platform users were constrained by the following:

1. Visualization of trends (i.e., year over year, month over month, geo spatial)
2. Data extraction and ingestion
   • Automated Data extraction from a user perspective was limited to automated emails in XLS (97-2003 Excel format)
   • CSV format is only available via manual data extraction
   • Limit on the number of reports that can be scheduled at once
   • Inability to use FTP or send reports to external system without email
3. Volume constraints on reporting file export
   • Ariba Network had a 10,000-record limit and the Analytical reporting had a 1.4 GB limit on exports*

The above constraints limited the user on being able to do trending across subject areas (Approvers, Invoicing, PO, Receipt, Payments) and caused manual work to combine data to produce accurate decisions based on historical data.

*This limit was hit six months into the project for Invoices and 8 months in for other entities

4 Ariba Analytical Data Model

The Ariba Data model is loosely based on a star schema with lookup composite keys. The model is very wide without surrogate keys and has an extensive list of degenerate dimensions* per fact.

Please see the following link to the most extensive explanation of the schema from SAP Help: https://help.sap.com/viewer/

5 Ariba APIs

The Ariba APIs are at the heart of the solution and are the key to unlocking the Ariba data. For trending and looking at historical data across departments, SAP Ariba recommends using the Analytical API. This API is asynchronous, returns data in json format zipped, and uses the open API framework with rest protocol* https://swagger.io/specification/.

The Ariba APIs consist of the following components:

1. Developer API Access
   • Generated and accessed through https://developer.ariba.com/api/
   • The admin of Ariba will be able to grant individuals’ access
2. Create Application
   • For example, we created the following application:
     
3. Access application using above through Postman or favorite API query application.
4. Manually run the above workflow with post man

The most important aspect of the project is this discovery as you are now able to codify the workflow in the programming language or tool of choice. We will dive into the solution architecture and codified workflow in the next section.

We produced the following workflow to implement the Ariba Analytical API using any type of API or Integration software provided (Postman, MuleSoft, Tibco, Informatica, Talend):

1. Nightly Batch
   • We created this process as a nightly batch process that can ran at any interval selected
     • Currently runs every 12 hours
2. Get_Token
   • Due to this being an asynchronous process we retrieve a new token for every call
3. Post_Job
   • Posts specified reporting view needed
   • We posted for 16 different views within the Ariba Analytical Api (ApprovalsFactSystemView, CustomIncrSSPPOLineItemFactView, CustomReceiptFactView, CustomSSPInvoiceLineItemFactView, etc.)
4. Get_Job_Status
   • Loops through every job posted from the Post Job step to get the status of the job based on the JobID returned from Post Job
   • Once job is completed check for Page Token (limit of 50,000 records per post) and if Page Token exists send to Post Job as a new job with Page Token
   • Send job to download files component
5. Get File Downloaded
   • Loop through list of files from Get_Job_Status and download each file to favorite file storage (AWS S3 in our case)

6 Ariba External Reporting Implementation

The final solution was built on top of the Ariba APIs as the foundation and utilized the following tools and technologies:

1. MuleSoft
   • Ingestion from Ariba API and land data into S3 bucket in raw zipped json format
2. Talend
   • Convert JSON to proper flattened format (in our case delimited flat file) and move to curated S3 bucket
   • Orchestrate E2E process and manage dependencies
3. Amazon S3
   • Store data in the raw and curated layer
4. Redshift Spectrum
   • Create a Fact/Dimensional model for reporting purposes
   • Data mining and transformations needed by business stakeholders
5. Redshift Internal Tables
   • Used by Qlik Sense and Tableau for reporting on the data
6. Presentation tool
   • Due to the agnostic Analytic layer, front-end tool of choice can be used for analysis
   • In this case, Tableau and Qlik Sense were leveraged by two different departments

The final solution exposed data using ELT and ETL through the following data flow:

1. Source Layer
   • Ariba API as source and utilized MuleSoft to capture data and land into S3
2. Raw Layer
   • Data was landed from MuleSoft in native json format from Ariba and stored all of history
3. Curated Layer
   • Data was converted/flattened to a delimited flat file and enriched with WHO? fields utilizing Talend
4. Analytic Layer
   • Redshift Spectrum tables were utilized to pull data from Curated
     • Pointed to curated S3 bucket to consume flattened data
   • Redshift Stored Procedures were utilized to integrate data with the internal EDW and create a star schema
     • Create Fact/Dimensional model for reporting purposes
     • Data mining and transformations needed by business stakeholders
   • Redshift Internal Tables
     • Used by Qlik Sense and Tableau for reporting on the data
5. Presentation tool
   • Due to the agnostic Analytic layer, front-end tool of choice can be used for analysis
   • In this case, Tableau and Qlik Sense were used from two different departments

7 Conclusion

Our IT and Data Ecosystems are becoming more external and less internal (SaaS and cloud solutions). One of these trends is for companies to move to SaaS enterprise applications. SaaS enterprise applications have many benefits, but, due to the nature of SaaS applications, there are more constraints in being able to unlock and access that data. In our solution with the SAP Ariba Platform, the Ariba Analytical API is a great tool to be able to ingest data from Ariba and report out holistically across your supply chain and procurement organizations. In many cases the companies key to the unlocking of data in a SaaS enterprise application will be in the API layer. The solution described above has successfully unlocked Ariba data for historical trending, data error identification, and future Analytical capabilities. The natural next steps for this type of solution are to use ML/AI models to predict and understand when invoices will be late by supplier, when discounts will be missed, and how best to pay invoices to gain the top return on investment from suppliers. Finally, we have found that one of the most important advantages of this solution is having control of your own data. Our clients have gained increased flexibility in being able to gather the data and present the data in their own model. It leads to new insights and understanding of the invoicing, procurement, and supply chain processes. It is a journey to implement the solution, but with the hours saved from manual data exports along with the ML/AI opportunities, our clients believe the results are well worth the investment.

Jay Watkins – Data Architect Manager