Sense & response service architecture (SARESA): an approach towards a real-time business intelligence solution and its use for a fraud detection application

ABSTRACT

The dynamic business environment of many organizations require massive monitoring of their processes in real-time in order to proactively respond to exceptional situations and to take advantage of time-sensitive business opportunities. The ability to sense and interpret events about a changing business environment requires an event-driven IT infrastructure for pwerforming fast and well-informed decisions and putting them into action. However, traditional Business Intelligence (BI) and Data Warehousing technologies do not directly address time sensitive monitoring and analytical requirements. We introduce an enhanced BI architecture that covers the complete process to sense, interpret, predict, automate and respond to business environments and thereby aims to decrease the reaction time needed for business decisions. We propose an event-driven IT infrastructure to operate BI applications which enable real-time analytics across corporate business processes, notifies the business of actionable recommendations or automatically triggers business operations, and effectively closing the gap between Business Intelligence systems and business processes. A scenario from the area of mobile phone fraud detection was chosen for building a prototype that illustrates the proposed approach by using current available IT technologies.

REFERENCES

• 1: Brian Babcock , Shivnath Babu , Mayur Datar , Rajeev Motwani , Jennifer Widom, Models and issues in data stream systems, Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, June 03-05, 2002, Madison, Wisconsin [doi>10.1145/543613.543615]
• 2: Brian Babcock , Mayur Datar , Rajeev Motwani, Load Shedding for Aggregation Queries over Data Streams, Proceedings of the 20th International Conference on Data Engineering, p.350, March 30-April 02, 2004
• 3: Brian Babcock , Chris Olston, Distributed top-k monitoring, Proceedings of the 2003 ACM SIGMOD international conference on Management of data, June 09-12, 2003, San Diego, California [doi>10.1145/872757.872764]
• 4: Shivnath Babu , Jennifer Widom, Continuous queries over data streams, ACM SIGMOD Record, v.30 n.3, September 2001 [doi>10.1145/603867.603884]
• 5: BROBST, S; BALLINGER, C, Active Data Warehousing, Whitepaper EB-1327, NCR Corporation, 2000.
• 6: Surajit Chaudhuri , Umeshwar Dayal, An overview of data warehousing and OLAP technology, ACM SIGMOD Record, v.26 n.1, p.65-74, March 1997 [doi>10.1145/248603.248616]
• 7: Robert M. Bruckner , Beate List , Josef Schiefer, Striving towards Near Real-Time Data Integration for Data Warehouses, Proceedings of the 4th International Conference on Data Warehousing and Knowledge Discovery, p.317-326, September 04-06, 2002
• 8: Kaushik Chakrabarti , Minos Garofalakis , Rajeev Rastogi , Kyuseok Shim, Approximate query processing using wavelets, The VLDB Journal — The International Journal on Very Large Data Bases, v.10 n.2-3, p.199-223, September 2001
• 9: CHANDRASEKARAN, S.; FRANKLIN, M., Streaming queries over streaming data, Proc. 28th Intl. Conf. on Very Large Data Bases, Aug. 2002.
• 10: A Data-Warehouse/OLAP Framework for Scalable Telecommunication Tandem Traffic Analysis, Proceedings of the 16th International Conference on Data Engineering, p.201, February 28-March 03, 2000
• 11: COMPAQ CORP, Compaq Global Services - Zero Latency Enterprise, http://clac.compaq.com/globalser vices/zle/
• 12: Alin Dobra , Minos Garofalakis , Johannes Gehrke , Rajeev Rastogi, Processing complex aggregate queries over data streams, Proceedings of the 2002 ACM SIGMOD international conference on Management of data, June 03-06, 2002, Madison, Wisconsin [doi>10.1145/564691.564699]
• 13: GARTNER GROUP, Introducing the Zero-Latency Enterprise, Research Note COM-04-3770, June 1998.
• 14: S. Guha , N. Mishra , R. Motwani , L. O'Callaghan, Clustering data streams, Proceedings of the 41st Annual Symposium on Foundations of Computer Science, p.359, November 12-14, 2000
• 15: HAISTEN , M, Real-time Data Warehousing Defined, Library article from BetterManagement.com, 2002
• 16: HAN J. et al, Multi Dimensional Regression Analysis of Time-Series Data Streams, Proc. of the 28th VLDB Conf. Hong Kong, 2002.
• 17: Christian Hidber, Online association rule mining, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.145-156, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 18: HEWLETT-PACKARD, Zero latency enterprise architecture, White paper, June 2002.
• 19: Geoff Hulten , Laurie Spencer , Pedro Domingos, Mining time-changing data streams, Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, p.97-106, August 26-29, 2001, San Francisco, California [doi>10.1145/502512.502529]
• 20: William H. Inmon, Building the Operational Data Store, John Wiley & Sons, Inc., New York, NY, 1999
• 21: LANGSETH, J, Real-time Data Warehousing: Challenges and Solutions, Article published at DSSResources.COM, 02/08/2004.
• 22: MUTHUKRISHNAN, S ;STRAUSS, M, Maintenance of Multidimensional Histograms The 23rd Conference FSTTCS 2003, India, 2003.
• 23: NATIS, Y, Service-Oriented Architecture Scenario, Gartner Research, ID ID Number: AV-19-6751, 16 April 2003
• 24: PALLOS, M, Service-Oriented Architecture: A Primer , eAI Journal , December 2001
• 25: RADEN, N, Exploring the Business Imperative of Real-time Analytics, Teradata white paper, October 2003.
• 26: STREAMBASE, StreamBase Systems (2005) The World's first Stream Processing Engine, <http://www.streambase.com/>
• 27: TERR, S, Real-time Data Warehousing 101, Article published at DataWarehouse.com, March 29,2004
• 28: Thomas Thalhammer , Michael Schrefl, Realizing active data warehouses with off-the-shelf database technology, Software—Practice & Experience, v.32 n.12, p.1193-1222, October 2002 [doi>10.1002/spe.479 ]
• 29: THO, N.; TJOA A., Zero-Latency Data Warehousing: Continuous Data Integration and Assembling Active Rules, 5th Intl. Conf. on Information Integration and Web-based Applications and Services (IIWAS2003)
• 30: TUCKER, P.; MAIER, D.; SHEARD, T., Applying Punctuation Schemes to Queries Over Continuous Data Streams, Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, March 2003.
• 31: WIDOM J. et al, Query processing, approximation, and resource management in a data stream management system, Proc. First Biennial Conf. on Innovative Data Systems Research (CIDR), Jan. 2003.
• 32: WHITE, C, Intelligent Business Strategies: Real-time Data Warehousing Heats Up, DMReview Publication, August 2002
• 33: SCHULTE, W., Application Integration Scenario: How the War is Being Won, in: Gartner Group (Ed.): Application Integration - Making E-Business Work, London, 6-7 September 2000.
• 34: SCHIEFER, J., MCGREGOR, C., Correlating Events for Monitoring Business Processes, International Conference on Enterprise Information Systems, Porto, 2004.
• 35: THO, M.N, SCHIEFER, J., TJOA M., ZELESSA (Zero-Latency Event Sensing and Responding): An Enabler for Real-time Business Intelligence, Technical Report 123/IFS/2005 (submitted to OeNB project proposal).
• 36: THO, M.N, Zero-Latency Data Warehousing: Toward a Zero Latency Event Sensing and Responding Data Warehousing, PhD Thesis, Vienna University of Technology, August 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.7 Database Administration
          Subjects: Data warehouse and repository

General Terms:
Design, Management, Measurement, Reliability

Keywords:
data analysis, event sense & response, real-time business intelligence, real-time data warehousing and OLAP