research-article

A Constraint Programming Based Energy Aware Resource Management Middleware for Clouds Processing MapReduce Jobs with Deadlines

Authors:

Shikharesh MajumdarAuthors Info & Claims

ICPE '16 Companion: Companion Publication for ACM/SPEC on International Conference on Performance Engineering

Pages 15 - 20

https://doi.org/10.1145/2859889.2859892

Published: 12 March 2016 Publication History

Abstract

This paper concerns guarantees on system performance through Service Level Agreement (SLA) compliance and focuses on devising energy aware resource management techniques based on Dynamic Voltage and Frequency Scaling (DVFS) used by resource management middleware in clouds that handle MapReduce jobs. This research formulates the resource management problem as an optimization problem using Constraint Programming (CP). Experimental results presented in the paper demonstrate the effectiveness of the technique.

References

[1]

Cardosa, M., Singh, A., Pucha, H., Chandra, A., "Exploiting Spatio-temporal Tradeoffs for Energy-Aware MapReduce in the Cloud," Cloud Computing (CLOUD), 2011 IEEE International Conference on, vol., no., pp.251,258.

Digital Library

[2]

Dean, J. and Ghemawat, S. 2004. MapReduce: Simplified data processing on large clusters. International Symposium on Operating System Design and Implementation (December 2004). 137--150.

Digital Library

[3]

Íñigo Goiri, Josep Ll. Berral, J. Oriol Fitó, Ferran Julií, Ramon Nou, Jordi Guitart, Ricard Gavaldí, and Jordi Torres. 2012. Energy-efficient and multifaceted resource management for profit-driven virtualized data centers. Future Gener. Comput. Syst. 28, 5 (May 2012), 718--731

Digital Library

[4]

Hamilton, J. "Cooperative expendable micro-slice servers(cems): low cost, low power servers for internet-scale services," in Proc. of the Conf. on Innovative Data Systems Research, 2009.

[5]

Hewlett-Packard Corporation, Intel Corporation, Microsoft Corporation, Phoenix Technologies Ltd., and Toshiba Corporation, Advanced Configuration and Power Interface Specification, Revision 5.0a, December 6, 2011.

[6]

IBM. IBM ILOG CPLEX Optimization Studio. Available: http://www-03.ibm.com/software/products/us/en/ibmilogcpleoptistud

[7]

Kyong Hoon Kim, Buyya, R., Jong Kim, "Power Aware Scheduling of Bag-of-Tasks Applications with Deadline Constraints on DVS-enabled Clusters," in Cluster Computing and the Grid, 2007. CCGRID 2007. vol., no., pp.541--548, 14--17 May 2007

Digital Library

[8]

Koomey, J. 2011. Growth in data center electricity use 2005 to 2010. Oakland, CA: Analytics Press. August, vol . 1.

[9]

Lim, N, Majumdar, S, and Ashwood-Smith, P. 2014. "Engineering Resource Management Middleware for Optimizing the Performance of Clouds Processing MapReduce Jobs with Deadlines", in Proc. 5Th ACM/SPEC ICPE Dublin, March 2014.

Digital Library

[10]

Rossi, F, van Beek, P, and Walsh, T. 2004. "Chapter 4: Constraint Programming," in Handbook of Knowledge Representation, San Diego, CA: Elsevier Science. 181--211.

[11]

Tan, L and Chen, Z. 2015. Slow down or halt: Saving the optimal energy for scalable HPC systems. in Proc. ICPE, 2015, pp. 241--244.

Digital Library

[12]

Verma, A.; Cherkasova, L.; Kumar, V.S.; Campbell, R.H., "Deadline-based workload management for MapReduce environments: Pieces of the performance puzzle," Network Operations and Management Symposium (NOMS), 2012 IEEE, vol., no., pp.900,905, 16--20 April 2012

[13]

Wang, L, von Laszewski, G, Dayal, J, and Wang, F. 2010. "Towards energy aware scheduling for precedence constrained parallel tasks in a cluster with dvfs," in Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, ser. CCGRID '10. Washington, DC, USA: IEEE Computer Society, 2010, pp. 368--377.

Digital Library

[14]

Wirtz, T and Ge, R. 2011. Improving MapReduce energy efficiency for computation intensive workloads. Green Computing Conference and Workshops (IGCC), 2011 International, vol., no,pp.1,8, 25--28.

Digital Library

[15]

Yanfei Li, Ying Wang, Bo Yin, Lu Guan, "An energy efficient resource management method in virtualized cloud environment," Network Operations and Management Symposium (APNOMS), 2012 14th Asia-Pacific, vol., no., pp.1,8, 25--27 Sept. 2012

Cited By

Stavrinides GKaratza H(2022)The Effect of Laxity of Real-Time Workflow Applications on the Performance of Elastic Cloud Resources2022 9th International Conference on Future Internet of Things and Cloud (FiCloud)10.1109/FiCloud57274.2022.00008(1-8)Online publication date: Aug-2022
https://doi.org/10.1109/FiCloud57274.2022.00008
Zeng XGarg SBarika MZomaya AWang LVillari MChen DRanjan R(2020)SLA Management for Big Data Analytical Applications in CloudsACM Computing Surveys10.1145/338346453:3(1-40)Online publication date: 12-Jun-2020
https://dl.acm.org/doi/10.1145/3383464
Alamro SXu MLan TSubramaniam S(2020)Shed+: Optimal Dynamic Speculation to Meet Application Deadlines in CloudIEEE Transactions on Network and Service Management10.1109/TNSM.2020.298647717:3(1515-1526)Online publication date: Sep-2020
https://doi.org/10.1109/TNSM.2020.2986477
Show More Cited By

Index Terms

A Constraint Programming Based Energy Aware Resource Management Middleware for Clouds Processing MapReduce Jobs with Deadlines

Recommendations

A Constraint Programming Based Hadoop Scheduler for Handling MapReduce Jobs with Deadlines on Clouds
ICPE '15: Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering

A novel MapReduce constraint programming based matchmaking and scheduling algorithm (MRCP) that can handle MapReduce jobs with deadlines and achieve high system performance is devised. The MRCP algorithm is incorporated into Hadoop, which is a widely ...
A Constraint Programming-Based Resource Management Technique for Processing MapReduce Jobs with SLAs on Clouds
BRACIS '14: Proceedings of the 2014 Brazilian Conference on Intelligent Systems

Clouds that are rapidly gaining in popularity require an effective resource manager that can harness the power of the underlying resource pool, and provide resources on demand to its users. This paper focuses on resource management on clouds for ...
Engineering resource management middleware for optimizing the performance of clouds processing mapreduce jobs with deadlines
ICPE '14: Proceedings of the 5th ACM/SPEC international conference on Performance engineering

This paper focuses on devising efficient resource management techniques used by the resource management middleware in clouds that handle MapReduce jobs with end-to-end service level agreements (SLAs) comprising an earliest start time, execution time, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPE '16 Companion: Companion Publication for ACM/SPEC on International Conference on Performance Engineering

March 2016

86 pages

ISBN:9781450341479

DOI:10.1145/2859889

General Chairs:
Alberto Avritzer
Sonatype, Inc., USA
,
Alexandru Iosup
Delft University of Technology, the Netherlands
,
Program Chairs:
Xiaoyun Zhu
Futurewei Technologies, USA
,
Steffen Becker
Chemnitz University of Technology, Germany

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 March 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICPE'16

Sponsor:

ICPE'16: ACM/SPEC International Conference on Performance Engineering

March 12 - 16, 2016

Delft, The Netherlands

Acceptance Rates

ICPE '16 Companion Paper Acceptance Rate 19 of 57 submissions, 33%;

Overall Acceptance Rate 252 of 851 submissions, 30%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
94
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Stavrinides GKaratza H(2022)The Effect of Laxity of Real-Time Workflow Applications on the Performance of Elastic Cloud Resources2022 9th International Conference on Future Internet of Things and Cloud (FiCloud)10.1109/FiCloud57274.2022.00008(1-8)Online publication date: Aug-2022
https://doi.org/10.1109/FiCloud57274.2022.00008
Zeng XGarg SBarika MZomaya AWang LVillari MChen DRanjan R(2020)SLA Management for Big Data Analytical Applications in CloudsACM Computing Surveys10.1145/338346453:3(1-40)Online publication date: 12-Jun-2020
https://dl.acm.org/doi/10.1145/3383464
Alamro SXu MLan TSubramaniam S(2020)Shed+: Optimal Dynamic Speculation to Meet Application Deadlines in CloudIEEE Transactions on Network and Service Management10.1109/TNSM.2020.298647717:3(1515-1526)Online publication date: Sep-2020
https://doi.org/10.1109/TNSM.2020.2986477
Xu MAlamro SLan TSubramaniam S(2018)Chronos: A Unifying Optimization Framework for Speculative Execution of Deadline-Critical MapReduce Jobs2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS.2018.00075(718-729)Online publication date: Jul-2018
https://doi.org/10.1109/ICDCS.2018.00075
Gregory AMajumdar S(2016)A Configurable Energy Aware Resource Management Technique for Optimization of Performance and Energy Consumption on Clouds2016 IEEE International Conference on Cloud Computing Technology and Science (CloudCom)10.1109/CloudCom.2016.0041(184-192)Online publication date: Dec-2016
https://doi.org/10.1109/CloudCom.2016.0041
Gregory AMajumdar S(2016)Energy Aware Resource Management for MapReduce Jobs with Service Level Agreements in Cloud Data Centers2016 IEEE International Conference on Computer and Information Technology (CIT)10.1109/CIT.2016.42(568-577)Online publication date: Dec-2016
https://doi.org/10.1109/CIT.2016.42

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten