Self-managed decentralised systems using K-components and collaborative reinforcement learning

ABSTRACT

Components in a decentralised system are faced with uncertainty as how to best adapt to a changing environment to maintain or optimise system performance. How can individual components learn to adapt to recover from faults in an uncertain environment? How can a decentralised system coordinate the adaptive behaviour of its components to realise system optimisation goals given problems establishing consensus in dynamic environments? This paper introduces a self-adaptive component model, called K-Components, that enables individual components adapt to a changing environment and a decentralised coordination model, called collaborative reinforcement learning, that enables groups of components to learn to collectively adapt their behaviour to establish and maintain system-wide properties in a changing environment.

REFERENCES

• 1: N. H. Minsky, "On conditions for self-healing in distributed software systems," Proceedngs of the Autonomic Computing Workshop, AMS '03, 2003.
• 2: A. Montresor, H. Meling, and O. Babaoglu, "Towards self-organizing, self-repairing and resilient distributed systems," Future Directions in Distributed Computing, vol. LNCS 2584, 2003.
• 3: Mark Garland Hayden, The ensemble system, 1998
• 4: David Garlan , Bradley Schmerl, Model-based adaptation for self-healing systems, Proceedings of the first workshop on Self-healing systems, November 18-19, 2002, Charleston, South Carolina [doi>10.1145/582128.582134]
• 5: Rohit Khare , Richard N. Taylor, Extending the Representational State Transfer (REST) Architectural Style for Decentralized Systems, Proceedings of the 26th International Conference on Software Engineering, p.428-437, May 23-28, 2004
• 6: Ian Clarke , Scott G. Miller , Theodore W. Hong , Oskar Sandberg , Brandon Wiley, Protecting Free Expression Online with Freenet, IEEE Internet Computing, v.6 n.1, p.40-49, January 2002 [doi>10.1109/4236.978368 ]
• 7: E. Curran and J. Dowling, "Sample: An on-demand probabilistic routing protocol for ad-hoc networks," Technical Report: Department of Computer Science, Trinity College Dublin, 2004.
• 8: James Kennedy , Russell C. Eberhart, Swarm intelligence, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2001
• 9: Eric Bonabeau , Marco Dorigo , Guy Theraulaz, Swarm intelligence: from natural to artificial systems, Oxford University Press, Inc., New York, NY, 1999
• 10: Jim Dowling , Vinny Cahill, The K-Component Architecture Meta-model for Self-Adaptive Software, Proceedings of the Third International Conference on Metalevel Architectures and Separation of Crosscutting Concerns, p.81-88, September 25-28, 2001
• 11: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 12: Jim Dowling , Raymond Cunningham , Eoin Curran , Vinny Cahill, Collaborative Reinforcement Learning of Autonomic Behaviour, Proceedings of the Database and Expert Systems Applications, 15th International Workshop on (DEXA'04), p.700-704, August 30-September 03, 2004 [doi>10.1109/DEXA.2004.47]
• 13: Scott Camazine , Nigel R. Franks , James Sneyd , Eric Bonabeau , Jean-Louis Deneubourg , Guy Theraula, Self-Organization in Biological Systems, Princeton University Press, Princeton, NJ, 2001
• 14: M. Jelasity, A. Montresor, and O. Babaoglu, "A modular paradigm for building self-organizing peer-to-peer applications," Proceedings of ESOP03: International Workshop on Engineering Self-Organising Applications, 2003.

INDEX TERMS

Keywords:
architectural reflection, collaborative reinforcement learning, decentralised self-adaptive systems