extended-abstract

Energy efficiency of large scale graph processing platforms

Authors:

Kashif Nizam Khan,

Mohammad Ashraful Hoque,

Jukka K. NurminenAuthors Info & Claims

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

Pages 1287 - 1294

https://doi.org/10.1145/2968219.2968296

Published: 12 September 2016 Publication History

Abstract

A number of graph processing platforms have emerged recently as a result of the growing demand on graph data analytics with complex and large-scale graph structured datasets. These platforms have been tailored for iterative graph computations and can offer an order of magnitude performance gain over generic data-flow frameworks like Apache Hadoop and Spark. Nevertheless, the increasing availability of such platforms and their functionality overlap necessitates a comparative study on various aspects of the platforms, including applications, performance and energy efficiency. In this work, we focus on the energy efficiency aspect of some large scale graph processing platforms. Specifically, we select two representatives, e.g., Apache Giraph and Spark GraphX, for the comparative study. We compare and analyze the energy consumption of these two platforms with PageRank, Strongly Connected Component and Single Source Shortest Path algorithms over five different realistic graphs. Our experimental results demonstrate that GraphX outperforms Giraph in terms of energy consumption. Specifically, Giraph consumes 1.71 times more energy than GraphX on average for the mentioned algorithms.

References

[1]

Tom Bostoen, Sape Mullender, and Yolande Berbers. 2013. Power-reduction Techniques for Data-center Storage Systems. ACM Comput. Surv. 45, 3, Article 33 (July 2013), 38 pages.

Digital Library

[2]

Mihai Capotă, Tim Hegeman, Alexandru Iosup, Arnau Prat-Pérez, Orri Erling, and Peter Boncz. 2015. Graphalytics: A Big Data Benchmark for Graph-Processing Platforms. In Proceedings of the GRADES'15 (GRADES'15). ACM, New York, NY, USA, Article 7, 6 pages.

Digital Library

[3]

Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: Simplified Data Processing on Large Clusters. Commun. ACM 51, 1 (Jan. 2008), 107--113.

Digital Library

[4]

Andy Feng. 2015. Rise of Scalable Machine Learning at Yahoo. (2015). http://www.slideshare.net/Hadoop_Summit/surge-rise-of-scalable-machine-learning-at-yahoo.

[5]

The Apache Software Foundation. 2016a. Apache Giraph. (2016). Retrieved 14th June, 2016 from http://giraph.apache.org/

[6]

The Apache Software Foundation. 2016b. Apache Hadoop. (2016). Retrieved 14th June, 2016 from http://hadoop.apache.org/

[7]

Yun Gao, Wei Zhou, Jizhong Han, Dan Meng, Zhang Zhang, and Zhiyong Xu. 2015. An Evaluation and Analysis of Graph Processing Frameworks on Five Key Issues. In Proceedings of the 12th ACM CF (CF '15). ACM, New York, NY, USA, Article 11, 8 pages.

Digital Library

[8]

Ionel Gog, Jana Giceva, Malte Schwarzkopf, Kapil Vaswani, Dimitrios Vytiniotis, Ganesan Ramalingam, Manuel Costa, Derek G. Murray, Steven Hand, and Michael Isard. 2015. Broom: Sweeping Out Garbage Collection from Big Data Systems. In 15th Workshop on HotOS. USENIX Association, Kartause Ittingen, Switzerland.

Digital Library

[9]

Joseph E. Gonzalez, Reynold S. Xin, Ankur Dave, Daniel Crankshaw, Michael J. Franklin, and Ion Stoica. 2014. GraphX: Graph Processing in a Distributed Dataflow Framework. In Proceedings of the 11th USENIX OSDI (OSDI'14). USENIX Association, Berkeley, CA, USA, 599--613.

Digital Library

[10]

Lei Gu and Huan Li. 2013. Memory or Time: Performance Evaluation for Iterative Operation on Hadoop and Spark. In IEEE 10th International Conference on HPCC. 721--727.

[11]

Minyang Han, Khuzaima Daudjee, Khaled Ammar, M. Tamer Özsu, Xingfang Wang, and Tianqi Jin. 2014. An Experimental Comparison of Pregel-like Graph Processing Systems. Proc. VLDB Endow. 7, 12 (Aug. 2014), 1047--1058.

Digital Library

[12]

Intel. 2014. Intel 64 and IA-32 Architectures Software Developer's Manual Volume 3 (3A, 3B & 3C): System Programming Guide. http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-system-programming-manual-325384.pdf. Accessed 2016-06-14.

[13]

Rini T. Kaushik and Milind Bhandarkar. 2010. GreenHDFS: Towards an Energy-conserving, Storage-efficient, Hybrid Hadoop Compute Cluster. In Proceedings of the 2010 HotPower (HotPower'10). USENIX Association, Berkeley, CA, USA, 1--9.

Digital Library

[14]

Jure Leskovec. 2016. Stanford Network Analysis Project. (2016). Retrieved 14th June, 2016 from https://snap.stanford.edu/

[15]

Jure Leskovec, Daniel Huttenlocher, and Jon Kleinberg. 2010. Signed Networks in Social Media. In Proceedings of the SIGCHI Conference (CHI '10). ACM, New York, NY, USA, 1361--1370.

Digital Library

[16]

Jure Leskovec, Jon Kleinberg, and Christos Faloutsos. 2005. Graphs over Time: Densification Laws, Shrinking Diameters and Possible Explanations. In Proceedings of the Eleventh ACM SIGKDD (KDD '05). ACM, New York, NY, USA, 177--187.

Digital Library

[17]

Jure Leskovec, Kevin J. Lang, Anirban Dasgupta, and Michael W. Mahoney. 2008. Community Structure in Large Networks: Natural Cluster Sizes and the Absence of Large Well-Defined Clusters. CoRR abs/0810.1355 (2008).

[18]

Yucheng Low, Danny Bickson, Joseph Gonzalez, Carlos Guestrin, Aapo Kyrola, and Joseph M. Hellerstein. 2012. Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud. Proc. VLDB Endow. 5, 8 (April 2012), 716--727.

Digital Library

[19]

Grzegorz Malewicz, Matthew H. Austern, Aart J.C Bik, James C. Dehnert, Ilan Horn, Naty Leiser, and Grzegorz Czajkowski. 2010. Pregel: A System for Large-scale Graph Processing. In Proceedings of the 2010 ACM SIGMOD (SIGMOD '10). ACM, New York, NY, USA, 135--146.

Digital Library

[20]

Leslie G. Valiant. 1990. A Bridging Model for Parallel Computation. Commun. ACM 33, 8 (Aug. 1990), 103--111.

Digital Library

[21]

Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauley, Michael J. Franklin, Scott Shenker, and Ion Stoica. 2012. Resilient Distributed Datasets: A Fault-tolerant Abstraction for In-memory Cluster Computing. In Proceedings of the 9th USENIX NSDI (NSDI'12). USENIX Association, Berkeley, CA, USA, 2--2.

Digital Library

Cited By

Veiga JEnes JExpósito RTouriño J(2018)BDEv 3.0: Energy efficiency and microarchitectural characterization of Big Data processing frameworksFuture Generation Computer Systems10.1016/j.future.2018.04.03086(565-581)Online publication date: Sep-2018
https://doi.org/10.1016/j.future.2018.04.030
Chen SRodero IAnjum ASill AZhao XFarid MPallickara SCao J(2017)Understanding Behavior Trends of Big Data Frameworks in Ongoing Software-Defined Cyber-InfrastructureProceedings of the Fourth IEEE/ACM International Conference on Big Data Computing, Applications and Technologies10.1145/3148055.3148079(199-208)Online publication date: 5-Dec-2017
https://dl.acm.org/doi/10.1145/3148055.3148079

Index Terms

Energy efficiency of large scale graph processing platforms

Recommendations

A Spark-Based Big Data Platform for Massive Remote Sensing Data Processing
ICDS 2015: Proceedings of the Second International Conference on Data Science - Volume 9208

With the fast development of remote sensing techniques, the volume of acquired data grows exponentially. This brings a big challenge to process massive remote sensing data. In the paper, an in-memory computing framework is proposed to address this ...
Big Graph Processing Systems: State-of-the-Art and Open Challenges
BIGDATASERVICE '15: Proceedings of the 2015 IEEE First International Conference on Big Data Computing Service and Applications

Graph is a fundamental data structure that captures relationships between different data entities. In practice, graphs are widely used for modeling complicated data in different application domains such as social networks, protein networks, ...
Research on Big Data Parallel Processing Platform Based on Postal Industry
ICDSP '20: Proceedings of the 2020 4th International Conference on Digital Signal Processing

With the development of cloud computing, big data, and the Internet of Things, for the data collection and daily drama of the postal express industry, in the face of such a large-scale data set, the traditional storage and calculation related theories ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

September 2016

1807 pages

ISBN:9781450344623

DOI:10.1145/2968219

General Chairs:
Paul Lukowicz
DFKI and University of Kaiserslautern
,
Antonio Krüger
DFKI and Saarland University
,
Program Chairs:
Andreas Bulling
Max Planck Institute for Informatics
,
Youn-Kyung Lim
KAIST
,
Shwetak N. Patel
University of Washington

Copyright © 2016 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2016

Check for updates

Author Tags

Qualifiers

Extended-abstract

Conference

UbiComp '16

Sponsor:

UbiComp '16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 12 - 16, 2016

Germany, Heidelberg

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
224
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)2

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Veiga JEnes JExpósito RTouriño J(2018)BDEv 3.0: Energy efficiency and microarchitectural characterization of Big Data processing frameworksFuture Generation Computer Systems10.1016/j.future.2018.04.03086(565-581)Online publication date: Sep-2018
https://doi.org/10.1016/j.future.2018.04.030
Chen SRodero IAnjum ASill AZhao XFarid MPallickara SCao J(2017)Understanding Behavior Trends of Big Data Frameworks in Ongoing Software-Defined Cyber-InfrastructureProceedings of the Fourth IEEE/ACM International Conference on Big Data Computing, Applications and Technologies10.1145/3148055.3148079(199-208)Online publication date: 5-Dec-2017
https://dl.acm.org/doi/10.1145/3148055.3148079

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten