research-article

Public Access

Robust Contextual Outlier Detection: Where Context Meets Sparsity

Authors:

Jiongqian Liang,

Srinivasan ParthasarathyAuthors Info & Claims

CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

Pages 2167 - 2172

https://doi.org/10.1145/2983323.2983660

Published: 24 October 2016 Publication History

Abstract

Outlier detection is a fundamental data science task with applications ranging from data cleaning to network security. Recently, a new class of outlier detection algorithms has emerged, called contextual outlier detection, and has shown improved performance when studying anomalous behavior in a specific context. However, as we point out in this article, such approaches have limited applicability in situations where the context is sparse (i.e., lacking a suitable frame of reference). Moreover, approaches developed to date do not scale to large datasets. To address these problems, here we propose a novel and robust approach alternative to the state-of-the-art called RObust Contextual Outlier Detection (ROCOD). We utilize a local and global behavioral model based on the relevant contexts, which is then integrated in a natural and robust fashion. We run ROCOD on both synthetic and real-world datasets and demonstrate that it outperforms other competitive baselines on the axes of efficacy and efficiency. We also drill down and perform a fine-grained analysis to shed light on the rationale for the performance gains of ROCOD and reveal its effectiveness when handling objects with sparse contexts.

References

[1]

A. Abishevatextitet al. Who watches (and shares) what on youtube? and when?: using twitter to understand youtube viewership. In WSDM. ACM, 2014.

Digital Library

[2]

M. Bartlett. The square root transformation in analysis of variance. JASA, pages 68--78, 1936.

[3]

M. M. Breunigtextitet al. Lof: identifying density-based local outliers. In SIGMOD. ACM, 2000.

Digital Library

[4]

V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. CSUR'09, 41(3):15, 2009.

Digital Library

[5]

M. Hauskrecht et al. Conditional outlier approach for detection of unusual patient care actions. In AAAI'13.

Digital Library

[6]

D. M. Hawkins. Identification of outliers, volume 11. Springer, 1980.

[7]

J. Liang and S. Parthasarathy. Robust contextual outlier detection: Where context meets sparsity. arXiv preprint arXiv:1607.08329, 2016.

Digital Library

[8]

C.-T. Lu, D. Chen, and Y. Kou. Algorithms for spatial outlier detection. In ICDM'03. IEEE, 2003.

Digital Library

[9]

N. J. Nagelkerke. A note on a general definition of the coefficient of determination. Biometrika, 1991.

[10]

H. V. Nguyentextitet al. Mining outliers with ensemble of heterogeneous detectors on random subspaces. In DSFAA. Springer, 2010.

Digital Library

[11]

G. H. Orairtextitet al. Distance-based outlier detection: consolidation and renewed bearing. VLDB, 2010.

[12]

S. Roberts and L. Tarassenko. A probabilistic resource allocating network for novelty detection. Neural Computation, 6(2):270--284, 1994.

Digital Library

[13]

X. Songtextitet al. Conditional anomaly detection. TKDE, 19(5):631--645, 2007.

Digital Library

[14]

G. Tangtextitet al. Mining multidimensional contextual outliers from categorical relational data. In SSDBM, page 43. ACM, 2013.

Digital Library

[15]

J. Tangtextitet al. Enhancing effectiveness of outlier detections for low density patterns. In PAKDD, pages 535--548. Springer, 2002.

Digital Library

[16]

W. R. Tobler. A computer movie simulating urban growth in the detroit region. Economic geography, 46:234--240, 1970.

[17]

M. Valkotextitet al. Conditional anomaly detection with soft harmonic functions. In ICDM, 2011.

[18]

Y. Wangtextitet al. Locality sensitive outlier detection: A ranking driven approach. In ICDE, 2011.

[19]

A. S. Weigend, M. Mangeas, and A. N. Srivastava. Nonlinear gated experts for time series: Discovering regimes and avoiding overfitting. IJNS, 1995.

[20]

K. Yamanishitextitet al. On-line unsupervised outlier detection using finite mixtures with discounting learning algorithms. DMKD, 2004.

Cited By

Mukhriya AKumar R(2025)Iterative target updation based boosting ensembles for outlier detectionPattern Recognition10.1016/j.patcog.2024.111023158(111023)Online publication date: Feb-2025
https://doi.org/10.1016/j.patcog.2024.111023
Yang ZRadke R(2024)Context-aware Video Anomaly Detection in Long-Term Datasets2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00404(4002-4011)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00404
Cederic LMaximilian BFabian RChristian HAnsgar T(2024)Contextual Anomaly Detection in Hot Forming Production Line using PINN Architecture2024 IEEE International Conference on Advanced Intelligent Mechatronics (AIM)10.1109/AIM55361.2024.10637234(1020-1025)Online publication date: 15-Jul-2024
https://doi.org/10.1109/AIM55361.2024.10637234
Show More Cited By

Index Terms

Robust Contextual Outlier Detection: Where Context Meets Sparsity
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Anomaly detection
2. Information systems
  1. Information systems applications
    1. Data mining

Recommendations

Enhancing Outlier Detection by an Outlier Indicator
Machine Learning and Data Mining in Pattern Recognition
Abstract
Outlier detection is an important task in data mining and has high practical value in numerous applications such as astronomical observation, text detection, fraud detection and so on. At present, a large number of popular outlier detection ...
CoMadOut—a robust outlier detection algorithm based on CoMAD
Abstract
Unsupervised learning methods are well established in the area of anomaly detection and achieve state of the art performances on outlier datasets. Outliers play a significant role, since they bear the potential to distort the predictions of a ...
Outlier detection based on rough sets theory

An outlier in a dataset is a point or a class of points that is considerably dissimilar to or inconsistent with the remainder of the data. Detection of outliers is important for many applications and has always attracted attention among data mining ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

October 2016

2566 pages

ISBN:9781450340731

DOI:10.1145/2983323

General Chairs:
Snehasis Mukhopadhyay
Indiana University Purdue University Indianapolis, USA
,
ChengXiang Zhai
University of Illinois at Urbana-Champaign, USA
,
Program Chairs:
Elisa Bertino
Purdue University
,
Fabio Crestani
University of Lugano
,
Javed Mostafa
University of North Carolina
,
Jie Tang
Tsinghua University
,
Luo Si
Alibaba Group Inc & Purdue University
,
Xiaofang Zhou
University of Queensland
,
Yi Chang
Yahoo Research
,
Yunyao Li
IBM Research - Almaden
,
Parikshit Sondhi
WalmartLabs

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

CIKM'16

Sponsor:

CIKM'16: ACM Conference on Information and Knowledge Management

October 24 - 28, 2016

Indiana, Indianapolis, USA

Acceptance Rates

CIKM '16 Paper Acceptance Rate 160 of 701 submissions, 23%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
752
Total Downloads

Downloads (Last 12 months)126
Downloads (Last 6 weeks)18

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mukhriya AKumar R(2025)Iterative target updation based boosting ensembles for outlier detectionPattern Recognition10.1016/j.patcog.2024.111023158(111023)Online publication date: Feb-2025
https://doi.org/10.1016/j.patcog.2024.111023
Yang ZRadke R(2024)Context-aware Video Anomaly Detection in Long-Term Datasets2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00404(4002-4011)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00404
Cederic LMaximilian BFabian RChristian HAnsgar T(2024)Contextual Anomaly Detection in Hot Forming Production Line using PINN Architecture2024 IEEE International Conference on Advanced Intelligent Mechatronics (AIM)10.1109/AIM55361.2024.10637234(1020-1025)Online publication date: 15-Jul-2024
https://doi.org/10.1109/AIM55361.2024.10637234
Duari GKumar R(2024)Attribute Subspace Partitioning with Neural Regression for Contextual Outlier DetectionProcedia Computer Science10.1016/j.procs.2024.04.180235(1892-1902)Online publication date: 2024
https://doi.org/10.1016/j.procs.2024.04.180
Calikus ENowaczyk SDikmen O(2024)Context discovery for anomaly detectionInternational Journal of Data Science and Analytics10.1007/s41060-024-00586-x19:1(99-113)Online publication date: 18-Jun-2024
https://doi.org/10.1007/s41060-024-00586-x
Mukhriya AKumar R(2023)Combination fairness with scores in outlier detection ensemblesInformation Sciences10.1016/j.ins.2023.119337645(119337)Online publication date: Oct-2023
https://doi.org/10.1016/j.ins.2023.119337
Angiulli FFassetti FSerrao C(2023)Anomaly detection with correlation lawsData & Knowledge Engineering10.1016/j.datak.2023.102181145:COnline publication date: 5-Jun-2023
https://dl.acm.org/doi/10.1016/j.datak.2023.102181
Li Zvan Leeuwen M(2023)Explainable contextual anomaly detection using quantile regression forestsData Mining and Knowledge Discovery10.1007/s10618-023-00967-z37:6(2517-2563)Online publication date: 9-Aug-2023
https://doi.org/10.1007/s10618-023-00967-z
Esmaeili FCassie ENguyen HPlank NUnsworth CWang A(2022)Predicting Analyte Concentrations from Electrochemical Aptasensor Signals Using LSTM Recurrent NetworksBioengineering10.3390/bioengineering91005299:10(529)Online publication date: 6-Oct-2022
https://doi.org/10.3390/bioengineering9100529
Calikus ENowaczyk SBouguelia MDikmen O(2022)Wisdom of the contexts: active ensemble learning for contextual anomaly detectionData Mining and Knowledge Discovery10.1007/s10618-022-00868-736:6(2410-2458)Online publication date: 4-Oct-2022
https://doi.org/10.1007/s10618-022-00868-7
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten