Article

Stochastic modeling and optimization for robust power management in a partially observable system

Authors:

Qinru Qiu,

Ying Tan,

Qing WuAuthors Info & Claims

DATE '07: Proceedings of the conference on Design, automation and test in Europe

Pages 779 - 784

Published: 16 April 2007 Publication History

Get Access

Abstract

As the hardware and software complexity grows, it is unlikely for the power management hardware/software to have a full observation of the entire system status. In this paper, we propose a new modeling and optimization technique based on partially observable Markov decision process (POMDP) for robust power management, which can achieve near-optimal power savings, even when only partial system information is available. Three scenarios of partial observations that may occur in an embedded system are discussed and their modeling techniques are presented. The experimental results show that, compared with power management policy derived from traditional Markov decision process model that assumes the system is fully observable, the new power management technique gives significantly better performance and energy tradeoff.

References

[1]

L. Benini, A. Bogliolo and G. De Micheli, "A survey of design techniques for system-level dynamic power management," IEEE Transactions on Very Large Scale Integrated Systems, Vol. 8, Issue 3, pp. 299--316, 2000.

Digital Library

Google Scholar

[2]

L. Benini, G. Paleologo, A. Bogliolo, and G. De Micheli, "Policy optimization for dynamic power management," IEEE Transactions on Computer-Aided Design, Vol. 18, pp. 813--33, June 1999.

Digital Library

Google Scholar

[3]

Q. Qiu, Q Wu and M. Pedram, "Stochastic modeling of a power-managed system-construction and optimization," IEEE Transactions on Computer-Aided Design, Vol. 20, pp. 1200--1217, October 2001.

Digital Library

Google Scholar

[4]

T Simunic, L Benini, P Glynn and G. D. Micheli, "Event-driven power management," IEEE Transactions on Computer-Aided Design, Vol. 20, pp. 840--857, Jul. 2001.

Digital Library

Google Scholar

[5]

Q. Qiu, Q. Wu, M. Pedram, "Dynamic Power Management of Complex Systems Using Generalized Stochastic Petri Nets," Proceedings of the Design Automation Conference, June 2000.

Digital Library

Google Scholar

[6]

P. Rong and M. Pedram, "Determining the optimal timeout values for a power-managed system based on the theory of Markovian processes: Offline and online algorithms," Proc. of Design Automation and Test in Europe, Mar. 2006.

Digital Library

Google Scholar

[7]

P. Rong and M. Pedram, "Hierarchical dynamic power management with application scheduling," Proc. of Symp. on Low Power Electronics and Design, Aug. 2005.

Digital Library

Google Scholar

[8]

T. Simunic, S. P. Boyd, P. Glynn, "Managing power consumption in networks on chips," IEEE Transactions on Very Large Scale Integration Systems, Vol 12, Issue 1, pp. 96--107, January 2004.

Digital Library

Google Scholar

[9]

W. Zhang, "Algorithms for Partially Observable Markov Decision Processes," Ph.D thesis, 2001.

Digital Library

Google Scholar

[10]

A. R. Cassandra, "Exact and Approximate Algorithms for Partially Observable Markov Decision Processes," Ph.D thesis, 1994.

Digital Library

Google Scholar

[11]

E. Y. Chung, L. Benini, A. Bogliolo, Y. H. Lu; G. De Micheli, "Dynamic power management for nonstationary service requests," IEEE Transactions on Computers, Vol. 51, Issue 11, pp. 1345 -- 1361, November 2002.

Digital Library

Google Scholar

[12]

Z. Ren, B. H. Krogh, R. Marculescu, "Hierarchical adaptive dynamic power management," IEEE Transactions on Computers, Vol. 54, Issue 4, pp. 409--420, April 2005.

Digital Library

Google Scholar

[13]

D. Braziunas, "POMDP Solution Methods," technical report, Department of Computer Science, University of Toronto, 2003.

Google Scholar

[14]

http://pomdp.org/pomdp/code/index.shtml

Google Scholar

Cited By

View all

Pan GJou JLai B(2014)Scalable Power Management Using Multilevel Reinforcement Learning for MultiprocessorsACM Transactions on Design Automation of Electronic Systems10.1145/262948619:4(1-23)Online publication date: 29-Aug-2014
https://dl.acm.org/doi/10.1145/2629486
Shen HTan YLu JWu QQiu Q(2013)Achieving autonomous power management using reinforcement learningACM Transactions on Design Automation of Electronic Systems10.1145/2442087.244209518:2(1-32)Online publication date: 11-Apr-2013
https://dl.acm.org/doi/10.1145/2442087.2442095
Wang YXie QAmmari APedram MStok LDutt NHassoun S(2011)Deriving a near-optimal power management policy using model-free reinforcement learning and Bayesian classificationProceedings of the 48th Design Automation Conference10.1145/2024724.2024735(41-46)Online publication date: 5-Jun-2011
https://dl.acm.org/doi/10.1145/2024724.2024735
Show More Cited By

Stochastic modeling and optimization for robust power management in a partially observable system
1. Mathematics of computing
  1. Probability and statistics
    1. Probabilistic representations
    2. Stochastic processes

Recommendations

Robust partially observable Markov decision process
ICML'15: Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37

We seek to find the robust policy that maximizes the expected cumulative reward for the worst case when a partially observable Markov decision process (POMDP) has uncertain parameters whose values are only known to be in a given region. We prove that ...
Planning and acting in partially observable stochastic domains

In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains. We begin by introducing the theory of Markov decision processes (mdps) and partially observable ...
Limiting Discounted-Cost Control of Partially Observable Stochastic Systems

This paper presents two main results on partially observable (PO) stochastic systems. In the first one, we consider a general PO system $$ x_{t+1}= F (x_t, a_t, xi_t), , y_t= G(x_t, eta_t) (t=0,1,ldots) hspace{1in} (*) $$ on Borel spaces, with possibly ...

Comments

Information & Contributors

Information

Published In

DATE '07: Proceedings of the conference on Design, automation and test in Europe

April 2007

1741 pages

ISBN:9783981080124

General Chair:
Rudy Lauwereins
IMEC and KULeuven, Belgium
,
Program Chair:
Jan Madsen
DTU, Denmark

Publisher

EDA Consortium

San Jose, CA, United States

Publication History

Published: 16 April 2007

Check for updates

Qualifiers

Article

Conference

DATE07

Sponsor:

EDAA
SIGDA
The Russian Academy of Sciences

DATE07: Design, Automation and Test in Europe

April 16 - 20, 2007

Nice, France

Acceptance Rates

Overall Acceptance Rate 518 of 1,794 submissions, 29%

Upcoming Conference

DATE '25

Sponsor:
sigda

Design, Automation and Test in Europe

March 31 - April 2, 2025

Lyon , France

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
192
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pan GJou JLai B(2014)Scalable Power Management Using Multilevel Reinforcement Learning for MultiprocessorsACM Transactions on Design Automation of Electronic Systems10.1145/262948619:4(1-23)Online publication date: 29-Aug-2014
https://dl.acm.org/doi/10.1145/2629486
Shen HTan YLu JWu QQiu Q(2013)Achieving autonomous power management using reinforcement learningACM Transactions on Design Automation of Electronic Systems10.1145/2442087.244209518:2(1-32)Online publication date: 11-Apr-2013
https://dl.acm.org/doi/10.1145/2442087.2442095
Wang YXie QAmmari APedram MStok LDutt NHassoun S(2011)Deriving a near-optimal power management policy using model-free reinforcement learning and Bayesian classificationProceedings of the 48th Design Automation Conference10.1145/2024724.2024735(41-46)Online publication date: 5-Jun-2011
https://dl.acm.org/doi/10.1145/2024724.2024735
Liu WTan YQiu QDe Micheli GAl-Hashimi BMueller WMacii E(2010)Enhanced Q-learning algorithm for dynamic power management with performance constraintProceedings of the Conference on Design, Automation and Test in Europe10.5555/1870926.1871068(602-605)Online publication date: 8-Mar-2010
https://dl.acm.org/doi/10.5555/1870926.1871068
Tan YLiu WQiu QRoychowdhury J(2009)Adaptive power management using reinforcement learningProceedings of the 2009 International Conference on Computer-Aided Design10.1145/1687399.1687486(461-467)Online publication date: 2-Nov-2009
https://dl.acm.org/doi/10.1145/1687399.1687486
Tan YQiu QSciuto D(2008)A framework of stochastic power management using hidden Markov modelProceedings of the conference on Design, automation and test in Europe10.1145/1403375.1403402(92-97)Online publication date: 10-Mar-2008
https://dl.acm.org/doi/10.1145/1403375.1403402

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

Robust partially observable Markov decision process

Planning and acting in partially observable stochastic domains

Limiting Discounted-Cost Control of Partially Observable Stochastic Systems

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations