skip to main content
research-article

Efficient and Robust Emergence of Norms through Heuristic Collective Learning

Published: 27 October 2017 Publication History

Abstract

In multiagent systems, social norms serves as an important technique in regulating agents’ behaviors to ensure effective coordination among agents without a centralized controlling mechanism. In such a distributed environment, it is important to investigate how a desirable social norm can be synthesized in a bottom-up manner among agents through repeated local interactions and learning techniques. In this article, we propose two novel learning strategies under the collective learning framework, collective learning EV-l and collective learning EV-g, to efficiently facilitate the emergence of social norms. Extensive simulations results show that both learning strategies can support the emergence of desirable social norms more efficiently and be applicable in a wider range of multiagent interaction scenarios compared with previous work. The influence of different topologies is investigated, which shows that the performance of all strategies is robust across different network topologies. The influences of a number of key factors (neighborhood size, actions space, population size, fixed agents and isolated subpopulations) on norm emergence performance are investigated as well.

References

[1]
T. Ågotnes and M. Wooldridge. 2010. Optimal social laws. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, Volume 1. 667--674.
[2]
A. Alexander, K. Lloyd, J. Pitt, and M. Sergot. 2005. A protocol for resource sharing in norm-governed ad hoc networks. In Declarative Agent Languages and Technologies II. 221--238.
[3]
A. Alexander, S. Marek, and P. Jeremy. 2009. Specifying norm-governed computational societies. ACM Trans. Comput. Logic 10, 1 (2009), 1.
[4]
A. L. Barabási, R. Albert, and H. Jeong. 2000. Scale-free characteristics of random networks: The topology of the world-wide web. Physica A 281, 1 (2000), 69--77.
[5]
C. K. Chan, J. Y. Hao, and H. F. Leung. 2015. Reciprocal social strategy in social repeated games. In Proceedings of International Conference on Tools with Artificial Intelligence. 966--973.
[6]
N. Criado, E. Argente, A. Garrido, J. A. Gimeno, F. Igual, V. Botti, P. Noriega, and A. Giret. 2011. Norm enforceability in electronic institutions? In Coordination, Organizations, Institutions, and Norms in Agent Systems VI. 250--267.
[7]
H. Franks, N. Griffiths, and A. Jhumka. 2013. Manipulating convention emergence using influencer agents. Auton. Agents Multi-Agent Syst. 26, 3 (2013), 315--353.
[8]
K. Genter, S. Zhang, and P. Stone. 2015. Determining placements of influencing agents in a flock. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems. 247--255.
[9]
N. Griffiths. 2008. Tags and image scoring for robust cooperation. In Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, Volume 2. 575--582.
[10]
N. Griffiths and S. S. Anand. 2012. The impact of social placement of non-learning agents on convention emergence. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems. 1367--1368.
[11]
N. Griffiths and M. Luck. 2010a. Changing neighbours: Improving tag-based cooperation. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems. 249--256.
[12]
N. Griffiths and M. Luck. 2010b. Norm emergence in tag-based cooperation. In Proceedings of the 8th European Workshop on Multi-Agent Systems.
[13]
J. Y. Hao and H. F. Leung. 2013. Achieving social optimality with influencer agents. In Complex Sciences. Springer, 140--151.
[14]
M. James, N. Griffiths, and M. Leeke. 2015. Convention emergence and influence in dynamic topologies. In Proceedings of International Conference on Autonomous Agents and Multiagent Systems. 1785--1786.
[15]
S. Kapetanakis and D. Kudenko. 2002. Reinforcement learning of coordination in cooperative multiagent systems. In Proceedings of the 18th National Conference on Artificial Intelligence. 326--331.
[16]
R. Makar, S. Mahadevan, and M. Ghavamzadeh. 2001. Hierarchical multi-agent reinforcement learning. In Proceedings of the 5th International Conference on Autonomous Agents. 246--253.
[17]
J. Marchant, N. Griffiths, and M. Leeke. 2015. Manipulating conventions in a particle-based topology. In Proceedings of the Coordination, Organizations, Institutions and Norms in Agent Systems Workshop.
[18]
M. Matlock and S. Sen. 2007. Effective tag mechanisms for evolving coordination. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems. ACM, 251.
[19]
M. Matlock and S. Sen. 2009. Effective tag mechanisms for evolving cooperation. In Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems. 489--496.
[20]
M. Mihaylov, K. Tuyls, and A. Nowé. 2014. A decentralized approach for convention emergence in multi-agent systems. Autonomous Agents and Multi-Agent Systems 28, 5 (2014), 749–778.
[21]
J. Morales, M. Lopez-Sanchez, J. A. Rodriguez-Aguilar, M. Wooldridge, and W. Vasconcelos. 2013. Automated synthesis of normative systems. In Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems. 483--490.
[22]
R. Polikar. 2006. Ensemble based systems in decision making. Circ. Syst. Mag. 6, 3 (2006), 21--45.
[23]
S. Redner. 1998. How popular is your paper an empirical study of the citation distribution. Eur. Phys. J. B 4, 2 (1998), 132--134.
[24]
A. Réka and A. L. Barabási. 2002. Statistical mechanics of complex networks. Rev. Mod. Phys. 74, 1 (2002), 47--97.
[25]
O. Sen and S. Sen. 2010. Effects of social network topology and options on norm emergence. In Coordination, Organizations, Institutions and Norms in Agent Systems V. 211--222.
[26]
S. Sen and S. Airiau. 2007. Emergence of norms through social learning. In Proceedings of the 16th International Joint Conference on Artificial Intelligence. 1507--1512.
[27]
Y. Shoham and M. Tennenholtz. 1997. On the emergence of social conventions: Modeling, analysis and simulations. Artif. Intell. 94(1-2) (1997), 139--166.
[28]
D. Villatoro, J. Sabater-Mir, and S. Sen. 2011. Social instruments for robust convention emergence. In Proceedings of the 20th International Joint Conference on Artificial Intelligence. 420--425.
[29]
D. Villatoro, S. Sen, and J. Sabater-Mir. 2009. Topology and memory effect on convention emergence. In Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, Volume 02. 233--240.
[30]
X. F. Wang and G. R. Chen. 2003. Complex networks: Small-world, scale-free and beyond. IEEE Circ. Syst. Mag. 3, 1 (2003), 6--20.
[31]
T. P. Yang, Z. P. Meng, J. Y. Hao, S. Sen, and C. Yu. 2016. Accelerating norm emergence through hierarchical heuristic learning. In Proceedings of the 22th European Conference on Artificial Intelligence (ECAI’16). 1344--1352.
[32]
H. P. Young. 1996. The economics of convention. J. Econ. Perspect. 10, 2 (1996), 105--122.
[33]
C. Yu, H. T. Lv, F. H. Ren, H. L. Bao, and J. Y. Hao. 2015. Hierarchical learning for emergence of social norms in networked multiagent systems. In AI 2015: Advances in Artificial Intelligence. Springer, Berlin, 630--643.
[34]
C. Yu, G. Z. Tan, H. T. Lv, Z. Wang, J. Meng, J. Y. Hao, and F. H. Ren. 2016. Modelling adaptive learning behaviours for consensus formation in human societies. Sci. Rep. 6 (2016).
[35]
C. Yu, M. J. Zhang, and F. H. Ren. 2014. Collective learning for the emergence of social norms in networked multiagent systems. IEEE Trans. Cybernet. 44, 12 (2014), 2342--2355.
[36]
C. Yu, M. J. Zhang, F. H. Ren, and X. D. Luo. 2013. Emergence of social norms through collective learning in networked agent societies. In Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems. 475--482.

Cited By

View all
  • (2024)Norm Enforcement with a Soft Touch: Faster Emergence, Happier AgentsProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663046(1837-1846)Online publication date: 6-May-2024
  • (2024)Norm Emergence through Conflict-Blocking Interactions in Industrial Internet of Things EnvironmentsSensors10.3390/s2418604724:18(6047)Online publication date: 19-Sep-2024
  • (2023)A perspective on the enabling technologies of explainable AI-based industrial packetized energy managementiScience10.1016/j.isci.2023.10841526:12(108415)Online publication date: Dec-2023
  • Show More Cited By

Index Terms

  1. Efficient and Robust Emergence of Norms through Heuristic Collective Learning

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Autonomous and Adaptive Systems
    ACM Transactions on Autonomous and Adaptive Systems  Volume 12, Issue 4
    December 2017
    224 pages
    ISSN:1556-4665
    EISSN:1556-4703
    DOI:10.1145/3155314
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 October 2017
    Accepted: 01 July 2017
    Revised: 01 January 2017
    Received: 01 November 2015
    Published in TAAS Volume 12, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Norm emergence
    2. multiagent collective learning

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    • Fundamental Research Funds for the Central Universities of China
    • National Natural Science Foundation of China
    • Tianjin Research Program of Application Foundation and Advanced Technology

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)16
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 18 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Norm Enforcement with a Soft Touch: Faster Emergence, Happier AgentsProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663046(1837-1846)Online publication date: 6-May-2024
    • (2024)Norm Emergence through Conflict-Blocking Interactions in Industrial Internet of Things EnvironmentsSensors10.3390/s2418604724:18(6047)Online publication date: 19-Sep-2024
    • (2023)A perspective on the enabling technologies of explainable AI-based industrial packetized energy managementiScience10.1016/j.isci.2023.10841526:12(108415)Online publication date: Dec-2023
    • (2022)Prosocial Norm Emergence in Multi-agent SystemsACM Transactions on Autonomous and Adaptive Systems10.1145/354020217:1-2(1-24)Online publication date: 7-Sep-2022
    • (2022)Emergence of norms in interactions with complex rewardsAutonomous Agents and Multi-Agent Systems10.1007/s10458-022-09585-337:1Online publication date: 26-Oct-2022
    • (2020)Collective Learning: A 10-Year Odyssey to Human-centered Distributed Intelligence2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)10.1109/ACSOS49614.2020.00043(205-214)Online publication date: Aug-2020
    • (2020)A novel subnetwork representation learning method for uncovering disease-disease relationshipsMethods10.1016/j.ymeth.2020.09.002Online publication date: Sep-2020
    • (2019)On learning in collective self-adaptive systemsProceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems10.1109/SEAMS.2019.00012(13-24)Online publication date: 25-May-2019
    • (2019)Norm emergence in multiagent systems: a viewpoint paperAutonomous Agents and Multi-Agent Systems10.1007/s10458-019-09422-0Online publication date: 30-Sep-2019
    • (2018)An Adaptive Learning Based Network Selection Approach for 5G Dynamic EnvironmentsEntropy10.3390/e2004023620:4(236)Online publication date: 29-Mar-2018
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media