Data Warehousing, Corporate Portal & e-Business Intelligence Applications
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Mimno, Myers & Holum
COLUMN: TECHNIQUES FOR CUSTOMIZING PACKAGED DATA WAREHOUSING SOLUTIONS
FlashPoint Column - October 28, 2002
COLUMN: TECHNIQUES FOR CUSTOMIZING PACKAGED DATA WAREHOUSING SOLUTIONS
by Pieter Mimno
Independent Consultant
SUMMARY: Packaged data warehousing solutions, such as SAP BW or PeopleSoft EPM, are widely used to integrate data from multiple business processes and deliver business content for B2B, CRM, and SCM applications. Typically, packaged solutions support between 60-80% of the business metrics required for an organization. However, it is difficult and expensive to customize packages to support the 20-40% of requirements not supported by the package. An important alternative to modifying a packaged data warehousing solution is to leverage the packaged application, but build custom applications in parallel with the package, using a general-purpose data warehousing solution. The general-purpose data warehousing application populates custom data marts that operate in parallel with the packaged data warehouse and are integrated with the package at the metadata level.
SAP BW and PEOPLESOFT EPM
SAP BW and PeopleSoft EPM are widely used to access data from multiple business processes, provide business content for B2B, CRM, and SCM, and Increase visibility into key business metrics and indicators. These packaged DW solutions support the requirement of business analysts to access integrated views of data across multiple business processes and generate business metrics and integrated reports.
SAP Business Information Warehouse (SAP BW) is a proprietary data warehousing environment that is appropriate for organizations that have made a major commitment to SAP R/3. It leverages the investment in SAP R/3 by providing a wide range of pre-configured business metrics and reports. New functions supported by SAP BW 3.0 include closed loop analytical applications, support for open, heterogeneous DW environments, and real-time e-business requirements across the business value chain. The SAP Business Information Architecture has four major components:
- Source systems - Source databases include R/3 OLTP application databases, and non-R/3 OLTP applications, i.e., legacy files, relational data sources, other application packages, external data feeds, real-time data feeds, purchased data, etc.
- Production Data Extraction - SAP provides a set of ABAP programs with BW that provide the ability to extract, transform, and load SAP R/3 data into BW. For integration of non-SAP data into BW, SAP provides a set of "Staging Engine" BAPIs that allow third-party ETL tools to extract and integrate non-SAP data sources, transform the data, and load it into the Staging Engine
- Business Information Warehouse Server - Proprietary SAP server that includes a Staging Engine used to store extracted and transformed data into an Operational Data Store. It also incorporates a Data Manager used to build and manage InfoCubes, and a Meta Data Manager used to generate and manage a central Meta Data Repository
- Business Explorer - Proprietary SAP tool used in combination with the OLAP Processor to support end-user requirements for queries, reports, drill-down, and OLAP analysis
A second ETL tool, called the Cube Manager, is used to populate 50 to 60 data marts, which support packaged applications. Browser-based Workbenches provide decision-support functions, which are delivered via end-user query, reporting, and analysis tools.
ADVANTAGES AND DISADVANTAGES OF PACKAGED DW SOLUTIONS
The advantage of packaged data warehousing solutions is that they leverage the investment in ERP packages, such as SAP R/3, PeopleSoft, Oracle Financials, Siebel, etc. They provide an end-to-end solution, including extraction/transformation of source data, population of target databases, generation of pre-packaged metrics, and delivery of reports and business metrics. Typically, they do not require expertise in ERP data sources.
Pre-packaged data warehousing solutions are attractive because they can provide an immediate return on investment through generation of a large number of common business metrics and reports. The products support a major trend toward the use of pre-packaged analytics, rather than building custom solutions. The use of pre-packaged analytics solutions and generation of key metrics leads to faster implementation of decision-support systems and faster speed to market. The packages are scalable and can be used to support a wide range of business requirements. The use of packages avoids the need to develop custom decision support applications.
The primary limitations of pre-packaged data warehousing solutions are that they are proprietary, expensive, and may be very difficult to customize to support functionality that is not available out-of-the-box. Pre-packaged data warehousing solutions, such as SAP BW and PeopleSoft EPM, typically support a range of 60-80% of the business metrics required for a specific organization. The 20-40% of requirements not supported by the packaged application requires additional customization.
Packaged applications are difficult and expensive to modify; requiring knowledge of ETL processes, data models, and BI tools. Extracting additional source data, modifying reports, and generating additional metrics may require substantial modifications to the pre-packaged data warehousing applications. The more the packaged application is modified, the harder it is to upgrade. The organization may obtain immediate benefit from the metrics generated out-of-the box by the packaged application. However, support for metrics that are needed, but not provided by the package, may be very expensive and may require utilization of external consulting resources.
OPTIONS FOR EXTENSION OF FUNCTIONALITY OF PACKAGED DW SOLUTIONS
Several options are available for extension of the functionality of packaged data warehousing solutions, such as SAP BW and PeopleSoft EPM. The first option is to modify the package to incorporate additional data sources, change extraction and transformation rules, enhance data models (such as the ODS and InfoCubes), and modify calculation and reporting modules. The objective of this option is to use the package to provide an integrated view of multiple data sources and enhance the functionality of the package to support additional or changing business requirements. However, many organizations have found that it is difficult and expensive to make substantial changes in the functionality of packages, such as SAP BW or PeopleSoft EPM.
Due to the complexity of customizing packaged data warehousing solutions, it is important to consider alternative approaches to providing an integrated view of information across point solutions and building custom applications. An important alternative to modifying a pre-packaged data warehousing solution is to leverage the packaged application, but build custom applications in parallel with the package, using a general-purpose data warehousing solution. The general-purpose data warehousing application operates in parallel with the package and is integrated with the package at the metadata level.
For example, a general-purpose ETL tool can be used in parallel with SAP BW to extend the functionality of SAP BW. In this example, the third-party, general-purpose ETL tool is used to extract data from SAP R/3 tables and non-R3 data sources and load data into the SAP BW ODS and InfoCubes via the BW InfoSource component. InfoSource is the transport object within BW that supplies BW with source system data from the external ETL tool. The SAP BW Production Data Extraction routines may continue to be used to extract R/3 data, but their role will gradually diminish. Metadata integration between the external ETL tool and BW supports the creation and updating of hierarchies in SAP BW. General-purpose, third-party ETL tools are available that have been certified by SAP for integration with SAP R/3 and SAP BW.
The same third-party ETL tool can be used in parallel with SAP BW to populate custom, external data marts for applications, such as supply-chain management, procurement, and specialized metrics, that may not be well-supported by SAP BW. In addition, the ETL tool can be used to populate packaged analytic applications from the same vendor that support procurement/spend/sourcing and other applications. The ETL tool is used both to populate SAP BW as well as to generate external, custom data marts and packaged, analytic applications. The ETL tool maintains metadata synchronization between BW and the custom data marts and analytic applications via integration with the BW InfoSource component.
The advantage of this approach is that it leverages the investment in SAP BW, supports the development of custom applications not implemented by SAP BW, and avoids modification of SAP BW functionality. Utilization of this alternative approach can provide an organization with a major cost-saving opportunity, relative to modifying the packaged data warehousing solution.
In summary, by using an external ETL tool to populate both packaged DW applications and custom data marts, organizations can continue to leverage the investment made in ERP and CRM source systems, as well as in packaged data warehousing solutions, such as SAP BW and PeopleSoft EPM. In addition, they can avoid the expense and complexity of customizing these packaged DW applications.
For further information about the issues discussed in this report, please contact Pieter Mimno, Independent Consultant, at pmimno@mimno.com, or visit his Web site at www.mimno.com. Mr. Mimno specializes in the selection of system components and support for all phases of development of data warehousing applications.
|
[ TOP ] |
|