3971 views since 2012-08-09 Page last modified on 2021-05-31.
  XHTML RSS QR Commons
Valid XHTML 1.0 Transitional RSS Logo and feed page link Hist-Op URL in QR encoded form Creative Commons licensing

“Hist-Op” DataStage® Custom Operator

Historisation Operator Screenshot

While DataStage® is a powerful and extensible ETL Tool, it is oriented towards row processing and despite the myriad of programming methods in the application, the specifics of processing data historisation for 2-dimensional and bi-temporal involve iterative and recursive processing and are thus beyond the built-in capabilities of this tool. Nonetheless, DataStage® does offer an API for adding custom operators and the “Hist-Op” is an example of such a custom built DataStage® operator.
At Arulan Consulting we've been involved in projects at banks and insurance companies where the bespoke data warehouse architecture specified bi-temporal processing; in each case a solution had to be programmed specific to that situation - with a significant investment in time and money and subsequent support. It became clear that as more warehouses implement two time dimensions the need for a robust and easily implemented solution within DataStage® would become actue and this was the birth of the “Hist-Op”. Drawing on years of involvement with these industries and expertise in analysing requirements, the “Hist-Op” was specified and programmed to cover a very wide range of possible data constellations, as can be seen by the significant number of options that can be set to alter the runtime behaviour of the operator.

Easy to install, compact, fast, stable and efficient, the “Hist-Op” is a unique tool for processing historised data using DataStage®