Towards a formal framework for multi-objective multiagent planning

ABSTRACT

Multi-Objective Multiagent Planning (MOMAP) addresses the problem of resolving conflicts between individual agent interests and the group interests. In this paper, we address this problem by presenting a formal framework to represent objective relationships, a decision model using a Vector-Valued Decentralized Markov Decision Process (2V-DEC-MDP) and an algorithm to solve the resulting 2V-DEC-MDP. The formal framework of a Vector-Valued MDP considered uses the value function which returns a vector representing the individual and the group interests. An optimal policy in such contexts is not clear but in this approach we develop a regret-based technique to find a good tradeoff between the group and individual interests. To do that, the approach we present uses Egalitarian Social Welfare orderings that allow an agent to consider during its local optimization the satisfaction of all criteria and reducing their differences. The obtained result is a good balance between individual and group satisfactions where the local policies can lead to more global satisfying behaviors in some settings. This result is illustrated in many examples and compared to alternate local policies.

References

1. A. Beynier and A. Mouaddib. A polynomial algorithm to solve decentralized mdp with temporal constraints. In AAMAS, 2005.
2. C. Boutilier, F. Bacchus, and R. Brafman. Ucp-networks: A directed graphical representation of conditional utilities. In International Conference on Uncertainty in Artificial Intelligence (UAI), 2001.
3. S. Jiaying, Z. Xiaoqin, and V. Lesser. Degree of local coordiantion and its implication on global utility. In Proceedings of AAMAS, pages 546--553, 2004.
4. R. Keeney and H. Raiffa. Decision with Multiple Objectives: Preferences and Value Tradeoffs. John Wiley and Sons, Inc., 1976.
5. A. Mouaddib. Anytime coordination for progressive planning agents. In AAAI-99, pages 564--569, 1999.
6. P. Perny, O. Spanjaard, and P. Weng. Algebraic markov decision processes. In IJCAI05, 2005.
7. R. Sabbadin. Possibilistic markov decision process. In ECAI, pages 586--590, 2000.
8. W. Stirling, M. Goodrich, and P. Packard. Satisficing equilibria: A non-classical approach to games and decisions. Journal of Autonomous Agents and Multiagent Systems, 5:305--328, 2002.
9. J. Von-Neumann and O. Morgenstern. The theory of games and economic behavior. In Princeton Univ. Press, 1947.
10. K. Wakuta and K. Togawa. Solution procedures for multi-objective markov decison processes. Optimization, 43:29--46, 1998.
11. D. Wolpert and K. Tumer. Introduction to collective intelligence. Handbook of Agent Technology, AAAI Press/MIT Press, 2000.
12. G. Zoltan, Z. Kalmar, and C. Szepesvari. Multi-criteria reinforcement learning. In International Conference on machine Learning (ICML), pages 197--205, 1998.

• Computing methodologies

  • Artificial intelligence

    • Distributed artificial intelligence