skip to main content
research-article

A provider-side view of web search response time

Published: 27 August 2013 Publication History

Abstract

Using a large Web search service as a case study, we highlight the challenges that modern Web services face in understanding and diagnosing the response time experienced by users. We show that search response time (SRT) varies widely over time and also exhibits counter-intuitive behavior. It is actually higher during off-peak hours, when the query load is lower, than during peak hours. To resolve this paradox and explain SRT variations in general, we develop an analysis framework that separates systemic variations due to periodic changes in service usage and anomalous variations due to unanticipated events such as failures and denial-of-service attacks. We find that systemic SRT variations are primarily caused by systemic changes in aggregate network characteristics, nature of user queries, and browser types. For instance, one reason for higher SRTs during off-peak hours is that during those hours a greater fraction of queries come from slower, mainly-residential networks. We also develop a technique that, by factoring out the impact of such variations, robustly detects and diagnoses performance anomalies in SRT. Deployment experience shows that our technique detects three times more true (operator-verified) anomalies than existing techniques.

References

[1]
Central limit theorem. http://en.wikipedia.org/wiki/Central_limit_theorem.
[2]
Measure page load time. https://developers.google.com/speed/docs/pss/LatencyMeasure.
[3]
Pagespeed tools family. https://developers.google.com/speed/pagespeed/.
[4]
WebPageTest - website performance and optimization test. http://www.webpagetest.org/.
[5]
Yslow. http://yslow.org/.
[6]
T. Abdelzaher, K. Shin, and N. Bhatti. Performance guarantees for Web server end-systems: A control-theoretical approach. IEEE Transactions on Parallel and Distributed Systems, 2002.
[7]
E. Agichtein, E. Brill, S. Dumais, and R. Ragno. Learning user interaction models for predicting Web search result preferences. In ACM SIGIR, 2006.
[8]
M. Al-Fares, K. Elmeleegy, B. Reed, and I. Gashinsky. Overclocking the Yahoo! CDN for faster Web page loads. In IMC, 2011.
[9]
P. Bahl, R. Chandra, A. Greenberg, S. Kandula, D. Maltz, and M. Zhang. Towards highly reliable enterprise network services via inference of multi-level dependencies. In ACM SIGCOMM, 2007.
[10]
L. Barroso, J. Dean, and U. Holzle. Web search for a planet: The Google cluster architecture. IEEE Micro, 2003.
[11]
M. Butkiewicz, H. Madhyastha, and V. Sekar. Understanding website complexity: Measurements, metrics, and implications. In IMC, 2011.
[12]
Y. Chen, S. Jain, V. Adhikari, and Z. Zhang. Characterizing roles of front-end servers in end-to-end performance of dynamic content distribution. In IMC, 2011.
[13]
J. Dean. Achieving rapid response times in large online services. In Berkeley AMPLab Cloud Seminar, 2012.
[14]
M. Freedman, E. Freudenthal, and D. Mazieres. Democratizing content publication with coral. In USENIX NSDI, 2004.
[15]
A. Gelman. Analysis of variance, 2005. http://www.stat.columbia.edu/ gelman/research/unpublished/econanova.pdf%.
[16]
C. Gkantsidis and P. Rodriguez. Network coding for large scale content distribution. In IEEE INFOCOM, 2005.
[17]
C. Huang, A. Wang, J. Li, and K. Ross. Measuring and evaluating large-scale cdns. In IMC, 2008.
[18]
H. Jiang, J. Li, Z. Li, and X. Bai. Performance evaluation of content distribution in hybrid CDN-P2P network. In IEEE Future Generation Communication and Networking, 2008.
[19]
K. Johnson, J. Carr, M. Day, and M. Kaashoek. The measured performance of content distribution networks. Elsevier, 2001.
[20]
S. Kandula, R. Mahajan, P. Verkaik, S. Agarwal, J. Padhye, and P. Bahl. Detailed diagnosis in enterprise networks. In ACM SIGCOMM, 2009.
[21]
J. Kangasharju, K. Ross, and J. Roberts. Performance evaluation of redirection schemes in content distribution networks. Elsevier, 2001.
[22]
R. Kohavi, R. M. Henne, and D. Sommerfield. Practical guide to controlled experiments on the Web: Listen to your customers not to the HiPPO. In SIGKDD, 2007.
[23]
R. Krishnan, H. Madhyastha, S. Srinivasan, S. Jain, A. Krishnamurthy, T. Anderson, and J. Gao. Moving beyond end-to-end path information to optimize CDN performance. In IMC, 2009.
[24]
A. Pinto. How to measure page load time with google analytics. http://www.yottaa.com/blog/bid/215491/How-to-Measure-Page-Load-Time-Wit%h-Google-Analytics.
[25]
D. G. Rees. Foundations of statistics, volume 214. Chapman & Hall, 1987.
[26]
M. Roughan, A. Greenberg, C. Kalmanek, M. Rumsewicz, J. Yates, and Y. Zhang. Experience in measuring backbone traffic variability: Models, metrics, measurements and meaning. In ACM IMW, 2002.
[27]
A. Singhal and M. Cutts. Official Google webmaster central blog: Using site speed in Web search ranking, 2009. http://googlewebmastercentral.blogspot.com/2010/04/using-site-speed-in-%web-search-ranking.html.
[28]
Y. Song, V. Ramasubramanian, and E. Sirer. Optimal resource utilization in content distribution networks. Technical Report TR2005--2004, Cornell University, 2005.
[29]
S. Stefanov. Progressive rendering via multiple flushes. http://www.phpied.com/progressive-rendering-via-multiple-flushes/.
[30]
W. Taylor. Change-point analysis: a powerful new tool for detecting changes, 2000. http://www.variation.com/cpa/tech/changepoint.html.
[31]
S. Triukose, Z. Wen, and M. Rabinovich. Measuring a commercial content delivery network. In ACM WWW, 2011.
[32]
X. S. Wang, A. Balasubramanian, A. Krishnamurthy, and D. Wetherall. Demystify page load performance with WProf. In USENIX NSDI, 2013.
[33]
R. White and S. Dumais. Characterizing and predicting search engine switching behavior. In ACM conference on Information and knowledge management, 2009.
[34]
S. Zander, L. Andrew, G. Armitage, G. Huston, and G. Michaelson. Mitigating sampling error when measuring Internet client IPv6 capabilities. In IMC, 2012.

Cited By

View all
  • (2024)Joint Subsequence Anomaly Detection and Fault Classification for Multivariate Time Series2024 9th International Conference on Computer and Communication Systems (ICCCS)10.1109/ICCCS61882.2024.10603208(143-148)Online publication date: 19-Apr-2024
  • (2024)An in-depth and insightful exploration of failure detection in distributed systemsComputer Networks10.1016/j.comnet.2024.110432247(110432)Online publication date: Jun-2024
  • (2023)Interactive Learning for Network Anomaly Monitoring and Detection with Human Guidance in the LoopSensors10.3390/s2318780323:18(7803)Online publication date: 11-Sep-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGCOMM Computer Communication Review
ACM SIGCOMM Computer Communication Review  Volume 43, Issue 4
October 2013
595 pages
ISSN:0146-4833
DOI:10.1145/2534169
Issue’s Table of Contents
  • cover image ACM Conferences
    SIGCOMM '13: Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM
    August 2013
    580 pages
    ISBN:9781450320566
    DOI:10.1145/2486001
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 August 2013
Published in SIGCOMM-CCR Volume 43, Issue 4

Check for updates

Author Tags

  1. anomaly detection and diagnosis
  2. performance monitoring
  3. search response time
  4. web services

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)164
  • Downloads (Last 6 weeks)25
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Joint Subsequence Anomaly Detection and Fault Classification for Multivariate Time Series2024 9th International Conference on Computer and Communication Systems (ICCCS)10.1109/ICCCS61882.2024.10603208(143-148)Online publication date: 19-Apr-2024
  • (2024)An in-depth and insightful exploration of failure detection in distributed systemsComputer Networks10.1016/j.comnet.2024.110432247(110432)Online publication date: Jun-2024
  • (2023)Interactive Learning for Network Anomaly Monitoring and Detection with Human Guidance in the LoopSensors10.3390/s2318780323:18(7803)Online publication date: 11-Sep-2023
  • (2023)Adversarial Density Ratio Estimation for Change Point DetectionProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615248(4254-4258)Online publication date: 21-Oct-2023
  • (2023)Root Cause Location Based on Prophet and Kernel Density EstimationIEEE Transactions on Network and Service Management10.1109/TNSM.2023.326841620:2(904-917)Online publication date: 1-Jun-2023
  • (2023)A Multivariate KPIs Anomaly Detection Framework With Dynamic Balancing Loss TrainingIEEE Transactions on Network and Service Management10.1109/TNSM.2022.322480320:2(1418-1429)Online publication date: 1-Jun-2023
  • (2023)A Multivariate Time Series Anomaly Detection Model Based on Spatio-Temporal Dual Features2023 International Conference on Networking and Network Applications (NaNA)10.1109/NaNA60121.2023.00075(416-421)Online publication date: Aug-2023
  • (2023)Accurate Anomaly Interval Recognition and Fault Classification by Pattern Mining and ClusteringIEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)10.1109/INFOCOMWKSHPS57453.2023.10226079(1-6)Online publication date: 20-May-2023
  • (2023)C-LLDP-monitoring: Latency Monitoring across Large-scale Software Defined Networks2023 5th International Conference on Frontiers Technology of Information and Computer (ICFTIC)10.1109/ICFTIC59930.2023.10455805(792-799)Online publication date: 17-Nov-2023
  • (2023)Trusted Log Tracing Framework Based on Fabric2023 9th International Conference on Computer and Communications (ICCC)10.1109/ICCC59590.2023.10507359(2644-2648)Online publication date: 8-Dec-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media