Creating interfaces between applications to exchange data is almost as old as computing itself. As systems evolved from mainframes to open systems, from monolithic ERPs to independent applications, the need for these systems and applications to exchange data became more and more important.

More recent is the emergence of data warehousing and business intelligence, which require collecting vast amounts of data from multiple systems.

In the past few years, both needs – data exchange between application and loading of Data Warehouses – have been systemized with the introduction of EAI (Enterprise Application Integration) suites and complex ETL (Extraction Transformation and Loading) solutions.

Even though these two types of solutions are very distinct and fulfill different needs, there are many common points that make their convergence inevitable. This article examines three major areas where convergence is today a reality.

Data Latency

Conversely, a traditional ETL process runs in batch mode – daily, weekly or even monthly – and usually while production systems are either taken offline, or at least show minimal activity (middle of the night, weekends, etc.)

The recent years have shown an increase of interest for real-time Data Warehousing and Business Activity Monitoring (BAM), where data needs to be analyzed in a timeframe sufficient to be able to make proactive (as opposed to reactive) decisions. At this stage, the Data Warehouse can be viewed as one of many systems that must be updated in a timely fashion with new data, making it very much a part of the Application Integration scheme. However Data Warehouse loading still encompasses several requirements such as complex transformation and data dependencies that must be taken into account.