ODI 11g in a Services Oriented Architecture? Exposing data services and consuming web services with ODI 11g. Part I.

Get our e-books Discover the Oracle Data Integrator 11g Repository Data Model and Oracle Data Integrator Snippets and Recipes

ODI in a SOA environment

This will be a four part series/tutorial on using ODI in a SOA environment.

One of the most overlooked features in ODI in general and in ODI 11g in particular is the ability to fully integrate ODI into a SOA environment. Typically technical architects associate ODI with high data volume ELT type data loads. However, ODI is very versatile when it comes down to meeting data integration requirements. Apart from the usual high volume, high performance ELT loads it can also be used in a web services centric environment.

In this series I will outline how ODI 11g can be deployed in a SOA environment to make data stores available to clients as data services. I will also show you how you can use ODI to consume data and web services. In the article I will give a step by step guide on how to create, deploy, and test a data service through ODI on Weblogic (without writing a single line of code). For the example I will use the customer table in the OE sample schema (http://download.oracle.com/docs/cd/B28359_01/server.111/b28328.pdf). On top of this, ODI can act as a consumer of web services. It can access both SOAP and RESTful web services. I will walk you through consuming both SOAP and RESTful web services using ODI. For the SOAP example we will connect to the OBIEE 11g web services API. For the RESTful web services we will consume the OpenStreet Map API to augment data in the OE schema customer table by longitude and latitude information.

Before we dive into the step by step guides, however, I first want to give a brief overview on the world of SOA, data services, SOAP and RESTful web services and how they are related to ODI.

What is SOA?

SOA is not a technology. It is rather a set of design principles for enterprise application architecture. In a SOA environment software units are loosely tied together as reusable services. They are platform independent and can be consumed by applications over standard network protocols. In a SOA environment the various web and data services are orchestrated in a workflow to implement a complex business process. The main benefit of SOA is faster application development. Already existing services can be orchestrated to quickly implement a given business process.

What are data services?

Data services expose enterprise data. Typically a data service makes data in one or more database tables available to consumers in the form of a web service. Client applications and end users can retrieve this data across physical and logical boundaries. On the web this could be across different geographies. For example this happens when you query the OpenStreet Map API. Irrespective of your location the data service returns the exact same data set. In the enterprise world this could be across business units within an enterprise. For example both the marketing and sales department can consume centrally held customer data. It could also be across enterprise boundaries, e.g. company 1 consumes product data from company 2.

What obstacles do we encounter in the implementation of data services?

Exposing data as a service without a supporting Master Data Management (MDM) program is like data warehousing without MDM. Difficult at best. Ideally the data that data services expose should be integrated, ought to be cleansed of errors, should be complete, needs to be up to date, and most not violate referential integrity. How would you feel when the longitude and latitude information changes each time you query the Open Street Map API? Good data quality is key for an environment that relies on data services. Any data services program should be accompanied by a data quality and master data management initiative. The so called golden record is a pre-requisite for a successful implementation of data services.

How are data services consumed?

Data services are typically consumed as web services. In the enterprise world you will mainly find SOAP based implementations. In the Web 2.0 world a lot of data services are now implemented as RESTful web services.

Data services and web services in ODI 11g

With ODI you can both create data services and consume them via web services. Once you have integrated, cleansed, and created the golden record of your data using ODI you can also expose this master data via data services. In turn you can consume data services via the ODI web services functionality.
When implementing data services you are not limited to read only access. You can also perform other operations on your ODI data services such as: deleting, updating, filtering.
Data service functionality has been before the latest release of ODI. However, in ODI 10g the data services had to be deployed in an Axis web services stack. This was a tedious and time consuming process. In ODI 11g it is a lot easier. ODI now supports the Java API for XML Web Services (JAX-WS) and you can deploy your data services directly to your Weblogic application server. This all happens automagically with a single click of a button. You don’t have to write a single line of code.
This means that you can use ODI to do the heavy lifting in a data warehouse environment but also in a master data management program to expose your golden record to client applications via data services.
As of ODI version 11.1.1.5.0 you can now also use web service container based authentication. It is now also possible to use fully formed SOAP messages as an input to a web service call.

In part of using Oracle Data Integrator in a Services Oriented Architecture we will create and deploy some ODI data services in Weblogic.