skip to main content
10.5555/1402298.1402329acmconferencesArticle/Chapter ViewAbstractPublication PagesaamasConference Proceedingsconference-collections
research-article

Sequential decision making with untrustworthy service providers

Published: 12 May 2008 Publication History

Abstract

In this paper, we deal with the sequential decision making problem of agents operating in computational economies, where there is uncertainty regarding the trustworthiness of service providers populating the environment. Specifically, we propose a generic Bayesian trust model, and formulate the optimal Bayesian solution to the exploration-exploitation problem facing the agents when repeatedly interacting with others in such environments. We then present a computationally tractable Bayesian reinforcement learning algorithm to approximate that solution by taking into account the expected value of perfect information of an agent's actions. Our algorithm is shown to dramatically outperform all previous finalists of the international Agent Reputation and Trust (ART) competition, including the winner from both years the competition has been run.

References

[1]
R. Ashri, S. D. Ramchurn, J. Sabater, M. Luck, and N. R. Jennings. Trust evaluation through relationship analysis. In Proc. of AAMAS'05, 2005.
[2]
R. Bellman. Dynamic Programming. Princeton University Press, 1957.
[3]
R. Bellman. Adaptive Control Processes: A guided tour. Princeton University Press, 1961.
[4]
G. Chalkiadakis and C. Boutilier. Coordination in Multiagent Reinforcement Learning: A Bayesian Approach. In Proc. of AAMAS'03, 2003.
[5]
G. Chalkiadakis and C. Boutilier. Bayesian Reinforcement Learning for Coalition Formation Under Uncertainty. In Proc. of AAMAS'04, 2004.
[6]
G. Chalkiadakis and C. Boutilier. Sequential decision making in repeated coalition formation under uncertainty. In Proc. of AAMAS'08, 2008.
[7]
R. Dearden, N. Friedman, and D. Andre. Model based Bayesian Exploration. In Proc. of UAI'99, 1999.
[8]
R. Dearden, N. Friedman, and S. Russell. Bayesian Q-Learning. In Proc. of AAAI-98, 1998.
[9]
M. DeGroot and M. Schervish. Probability & Statistics. 2002.
[10]
K. Fullam, T. Klos, G. Muller, J. Sabater, A. Schlosser, Z. Topol, K. Barber, J. Rosenschein, L. Vercouter, and M. Voss. A specification of the agent reputation and trust (art) testbed: experimentation and competition for trust in agent societies. In Proc. of AAMAS-05. ACM Press, 2005.
[11]
S. Ramchurn, T. D. Huynh, and N. R. Jennings. Trust in multiagent systems. The Knowledge Engineering Review, 19(1):1--25, March 2004.
[12]
S. Ramchurn, C. Sierra, L. Godo, and N. R. Jennings. A computational trust model for multi-agent interactions based on confidence and reputation. In Proceedings of the 6th International Workshop of Deception, Fraud and Trust in Agent Societies, pages 69--75, Melbourne, Australia, July 2003. ACM Press.
[13]
K. Regan, P. Poupart, and R. Cohen. Bayesian Reputation Modeling in E-Marketplaces Sensitive to Subjectivity, Deception and Change. In Proc. of AAAI-06, 2006.
[14]
A. Rogers, E. David, and N. R. Jennings. Self-Organized Routing For Wireless Micro-Sensor Networks. IEEE Transactions on Systems, Man, and Cybernetics - Part A, 35(3):349--359, 2005.
[15]
J. Sabater and C. Sierra. Social regret, a reputation model based on social relations. SIGecom Exchanges, 3(1):44--56, 2002.
[16]
W. T. L. Teacy, T. D. Huynh, R. K. Dash, N. R. Jennings, M. Luck, and J. Patel. The art of iam: The winning strategy for the 2006 competition. In Proc. of the 10th International Workshop on Trust in Agent Societies, 2007.
[17]
W. T. L. Teacy, J. Patel, N. R. Jennings, and M. Luck. TRAVOS: Trust and reputation in the context of inaccurate information sources. JAAMAS, 12(2):183--198, 2006.

Cited By

View all
  • (2017)Improving the Efficiency of an Online Marketplace by Incorporating Forgiveness MechanismACM Transactions on Internet Technology10.1145/299618917:1(1-20)Online publication date: 20-Jan-2017
  • (2017)How to trust a few among manyAutonomous Agents and Multi-Agent Systems10.1007/s10458-016-9337-531:3(531-560)Online publication date: 1-May-2017
  • (2015)Trust and Reputation Models for Multiagent SystemsACM Computing Surveys10.1145/281682648:2(1-42)Online publication date: 12-Oct-2015
  • Show More Cited By

Index Terms

  1. Sequential decision making with untrustworthy service providers

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    AAMAS '08: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 2
    May 2008
    673 pages
    ISBN:9780981738116

    Sponsors

    In-Cooperation

    Publisher

    International Foundation for Autonomous Agents and Multiagent Systems

    Richland, SC

    Publication History

    Published: 12 May 2008

    Check for updates

    Author Tags

    1. reinforcement learning
    2. reputation
    3. trust
    4. uncertainty

    Qualifiers

    • Research-article

    Conference

    AAMAS08
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)1
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 09 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2017)Improving the Efficiency of an Online Marketplace by Incorporating Forgiveness MechanismACM Transactions on Internet Technology10.1145/299618917:1(1-20)Online publication date: 20-Jan-2017
    • (2017)How to trust a few among manyAutonomous Agents and Multi-Agent Systems10.1007/s10458-016-9337-531:3(531-560)Online publication date: 1-May-2017
    • (2015)Trust and Reputation Models for Multiagent SystemsACM Computing Surveys10.1145/281682648:2(1-42)Online publication date: 12-Oct-2015
    • (2014)Choosing a Candidate Using Efficient Allocation of Biased InformationACM Transactions on Intelligent Systems and Technology10.1145/25583275:4(1-30)Online publication date: 29-Dec-2014
    • (2011)Argumentation-based reasoning in agents with varying degrees of trustThe 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 210.5555/2031678.2031743(879-886)Online publication date: 2-May-2011
    • (2011)Using argumentation to reason with and about trustProceedings of the 8th international conference on Argumentation in Multi-Agent Systems10.1007/978-3-642-33152-7_12(194-212)Online publication date: 3-May-2011
    • (2010)A probabilistic model for trust and reputationProceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 110.5555/1838206.1838238(225-232)Online publication date: 10-May-2010
    • (2009)Why should i trust in a virtual community member?Proceedings of the 15th international conference on Groupware: design, implementation, and use10.5555/1813962.1813975(126-133)Online publication date: 13-Sep-2009
    • (2009)Strategies for exploiting trust models in competitive multi-agent systemsProceedings of the 7th German conference on Multiagent system technologies10.5555/1791994.1792004(79-90)Online publication date: 9-Sep-2009
    • (2009)Dynamic information source selection for intrusion detection systemsProceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 210.5555/1558109.1558153(1009-1016)Online publication date: 10-May-2009
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media