skip to main content
10.1145/1376616.1376686acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

MCDB: a monte carlo approach to managing uncertain data

Published: 09 June 2008 Publication History

Abstract

To deal with data uncertainty, existing probabilistic database systems augment tuples with attribute-level or tuple-level probability values, which are loaded into the database along with the data itself. This approach can severely limit the system's ability to gracefully handle complex or unforeseen types of uncertainty, and does not permit the uncertainty model to be dynamically parameterized according to the current state of the database. We introduce MCDB, a system for managing uncertain data that is based on a Monte Carlo approach. MCDB represents uncertainty via "VG functions," which are used to pseudorandomly generate realized values for uncertain attributes. VG functions can be parameterized on the results of SQL queries over "parameter tables" that are stored in the database, facilitating what-if analyses. By storing parameters, and not probabilities, and by estimating, rather than exactly computing, the probability distribution over possible query answers, MCDB avoids many of the limitations of prior systems. For example, MCDB can easily handle arbitrary joint probability distributions over discrete or continuous attributes, arbitrarily complex SQL queries, and arbitrary functionals of the query-result distribution such as means, variances, and quantiles. To achieve good performance, MCDB uses novel query processing techniques, executing a query plan exactly once, but over "tuple bundles" instead of ordinary tuples. Experiments indicate that our enhanced functionality can be obtained with acceptable overheads relative to traditional systems.

References

[1]
P. Agrawal, O. Benjelloun, A. D. Sarma, C. Hayworth, S. U. Nabar, T. Sugihara, and J. Widom. Trio: A system for data, uncertainty, and lineage. In VLDB, 2006.
[2]
P. Andritsos, A. Fuxman, and R. J. Miller. Clean answers over dirty databases: A probabilistic approach. In ICDE, page 30, 2006.
[3]
L. Antova, C. Koch, and D. Olteanu. 10106 worlds and beyond: Efficient representation and processing of incomplete information. In ICDE, pages 606--615, 2007.
[4]
L. Antova, C. Koch, and D. Olteanu. MayBMS: Managing incomplete information with probabilistic world-set decompositions. In ICDE, pages 1479--1480, 2007.
[5]
B. Biller and B. L. Nelson. Modeling and generating multivariate time-series input processes using a vector autoregressive technique. ACM Trans. Modeling Comput. Simulation, 13(3):211--237, 2003.
[6]
D. Burdick, A. Doan, R. Ramakrishnan, and S. Vaithyanathan. OLAP over imprecise data with domain constraints. In VLDB, pages 39--50, 2007.
[7]
R. Cheng, D. V. Kalashnikov, and S. Prabhakar. Evaluation of probabilistic queries over imprecise data in constantly-evolving environments. Inf. Syst., 32(1):104--130, 2007.
[8]
R. Cheng, S. Singh, and S. Prabhakar. U-DBMS: A database system for managing constantly-evolving data. In VLDB, pages 1271--1274, 2005.
[9]
R. Cheng, S. Singh, S. Prabhakar, R. Shah, J. S. Vitter, and Y. Xia. Efficient join processing over uncertain data. In CIKM, pages 738--747, 2006.
[10]
D. Chu, A. Deshpande, J. M. Hellerstein, and W. Hong. Approximate data collection in sensor networks using probabilistic models. In ICDE, page 48, 2006.
[11]
N. N. Dalvi and D. Suciu. Efficient query evaluation on probabilistic databases. VLDB J., 16(4):523--544, 2007.
[12]
A. Deshpande and S. Madden. MauveDB: supporting model-based user views in database systems. In SIGMOD, pages 73--84, 2006.
[13]
L. Devroye. Non-Uniform Random Variate Generation. Springer, 1986.
[14]
L. Devroye and G. Lugosi. Combinatorial Methods in Density Estimation. Springer, 2001.
[15]
G. Fishman. Monte Carlo: Concepts, Algorithms, and Applications. Springer, 1996.
[16]
N. Fuhr and T. Rölleke. A probabilistic relational algebra for the integration of information retrieval and database systems. ACM Trans. Inf. Syst., 15(1):32--66, 1997.
[17]
J. E. Gentle. Random Number Generation and Monte Carlo Methods. Springer, second edition, 2003.
[18]
L. Getoor and B. Taskar, editors. Introduction to Statistical Relational Learning. MIT Press, 2007.
[19]
R. Gupta and S. Sarawagi. Creating probabilistic databases from information extraction models. In VLDB, pages 965--976, 2006.
[20]
J. M. Hellerstein, P. J. Haas, and H. J. Wang. Online aggregation. In SIGMOD, pages 171--182, 1997.
[21]
S. G. Henderson and B. L. Nelson, editors. Simulation. North-Holland, 2006.
[22]
C. M. Jermaine, S. Arumugam, A. Pol, and A. Dobra. Scalable approximate query processing with the DBO engine. In SIGMOD, pages 725--736, 2007.
[23]
B. Kimelfeld and Y. Sagiv. Matching twigs in probabilistic XML. In VLDB, pages 27--38, 2007.
[24]
B. Kimelfeld and Y. Sagiv. Maximally joining probabilistic data. In PODS, pages 303--312, 2007.
[25]
R. Murthy and J. Widom. Making aggregation work in uncertain and probabilistic databases. In Proc. 1st Int. VLDB Work. Mgmt. Uncertain Data (MUD), pages 76--90, 2007.
[26]
A. Nadas. An extension of a theorem by Chow and Robbins on sequential confidence intervals for the mean. Ann. Math. Statist., 40(2):667--671, 1969.
[27]
R. B. Nelsen. An Introduction to Copulas. Springer, second edition, 2006.
[28]
A. O'Hagan and J. J. Forster. Bayesian Inference. Volume 2B of Kendall's Advanced Theory of Statistics. Arnold, second edition, 2004.
[29]
J. Pei, B. Jiang, X. Lin, and Y. Yuan. Probabilistic skylines on uncertain data. In VLDB, pages 15--26, 2007.
[30]
C. Re, N. N. Dalvi, and D. Suciu. Query evaluation on probabilistic databases. IEEE Data Eng. Bull., 29(1):25--31, 2006.
[31]
C. Re, N. N. Dalvi, and D. Suciu. Efficient top-k query evaluation on probabilistic data. In ICDE, pages 886--895, 2007.
[32]
C. Re and D. Suciu. Materialized views in probabilistic databases for information exchange and query optimization. In VLDB, pages 51--62, 2007.
[33]
P. Sen and A. Deshpande. Representing and querying correlated tuples in probabilistic databases. In ICDE, pages 596--605, 2007.
[34]
R. J. Serfling. Approximation Theorems of Mathematical Statistics. Wiley, 1980.
[35]
S. Singh, C. Mayfield, S. Prabhakar, R. Shah, and S. E. Hambrusch. Indexing uncertain categorical data. In ICDE, pages 616--625, 2007.
[36]
J. Xie, J. Yang, Y. Chen, H. Wang, and P. Yu. A sampling-based approach to information recovery. In ICDE, 2008. To appear.

Cited By

View all
  • (2024)A Data-Driven Approach to Set-Theoretic Model Predictive Control for Nonlinear SystemsInformation10.3390/info1507036915:7(369)Online publication date: 23-Jun-2024
  • (2023)Optimizing Tensor Computations: From Applications to Compilation and Runtime TechniquesCompanion of the 2023 International Conference on Management of Data10.1145/3555041.3589407(53-59)Online publication date: 4-Jun-2023
  • (2023)Query Processing Over RelationalCross Model in Uncertain and Probabilistic Databases2023 Third International Conference on Artificial Intelligence and Smart Energy (ICAIS)10.1109/ICAIS56108.2023.10073827(763-769)Online publication date: 2-Feb-2023
  • Show More Cited By

Index Terms

  1. MCDB: a monte carlo approach to managing uncertain data

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGMOD '08: Proceedings of the 2008 ACM SIGMOD international conference on Management of data
    June 2008
    1396 pages
    ISBN:9781605581026
    DOI:10.1145/1376616
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 09 June 2008

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. mcdb
    2. monte carlo
    3. uncertainty

    Qualifiers

    • Research-article

    Conference

    SIGMOD/PODS '08
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 785 of 4,003 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)34
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 16 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)A Data-Driven Approach to Set-Theoretic Model Predictive Control for Nonlinear SystemsInformation10.3390/info1507036915:7(369)Online publication date: 23-Jun-2024
    • (2023)Optimizing Tensor Computations: From Applications to Compilation and Runtime TechniquesCompanion of the 2023 International Conference on Management of Data10.1145/3555041.3589407(53-59)Online publication date: 4-Jun-2023
    • (2023)Query Processing Over RelationalCross Model in Uncertain and Probabilistic Databases2023 Third International Conference on Artificial Intelligence and Smart Energy (ICAIS)10.1109/ICAIS56108.2023.10073827(763-769)Online publication date: 2-Feb-2023
    • (2023)Set-theoretic receding horizon control for nonlinear systems: a data-driven approachIEEE EUROCON 2023 - 20th International Conference on Smart Technologies10.1109/EUROCON56442.2023.10198968(579-584)Online publication date: 6-Jul-2023
    • (2023)Distributed probabilistic top-k dominating queries over uncertain databasesKnowledge and Information Systems10.1007/s10115-023-01917-365:11(4939-4965)Online publication date: 1-Jul-2023
    • (2022)A Probabilistic Data Fusion Modeling Approach for Extracting True Values from Uncertain and Conflicting AttributesBig Data and Cognitive Computing10.3390/bdcc60401146:4(114)Online publication date: 13-Oct-2022
    • (2022)HypeR: Hypothetical Reasoning With What-If and How-To Queries Using a Probabilistic Causal ApproachProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3526149(1598-1611)Online publication date: 10-Jun-2022
    • (2022)Interactive Mining of Probabilistic Frequent Patterns in Uncertain DatabasesInternational Journal of Uncertainty, Fuzziness and Knowledge-Based Systems10.1142/S021848852250011830:02(263-283)Online publication date: 18-Apr-2022
    • (2022)Diversifying repairs of Denial constraint violationsInformation Systems10.1016/j.is.2022.102041108:COnline publication date: 3-Jun-2022
    • (2022)Approximate weighted model integration on DNF structuresArtificial Intelligence10.1016/j.artint.2022.103753311(103753)Online publication date: Oct-2022
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media