research-article

Player retention in league of legends: a study using survival analysis

Authors:

Simon Demediuk,

Alexandra Murrin,

Michael Hitchens,

Anders Drachen,

William L. Raffe,

Marco TamassiaAuthors Info & Claims

ACSW '18: Proceedings of the Australasian Computer Science Week Multiconference

Article No.: 43, Pages 1 - 9

https://doi.org/10.1145/3167918.3167937

Published: 29 January 2018 Publication History

Abstract

Multi-player online esports games are designed for extended durations of play, requiring substantial experience to master. Furthermore, esports game revenues are increasingly driven by in-game purchases. For esports companies, the trends in players leaving their games therefore not only provide information about potential problems in the user experience, but also impacts revenue. Being able to predict when players are about to leave the game - churn prediction - is therefore an important solution for companies in the rapidly growing esports sector, as this allows them to take action to remedy churn problems.

The objective of the work presented here is to understand the impact of specific behavioral characteristics on the likelihood of a player continuing to play the esports title League of Legends. Here, a solution to the problem is presented based on the application of survival analysis, using Mixed Effects Cox Regression, to predict player churn. Survival Analysis forms a useful approach for the churn prediction problem as it provides rates as well as an assessment of the characteristics of players who are at risk of leaving the game. Hazard rates are also presented for the leading indicators, with results showing that duration between matches played is a strong indicator of potential churn.

References

[1]

Zoheb Borbora, Jaideep Srivastava, Kuo-Wei Hsu, and Dmitri Williams. 2011. Churn prediction in mmorpgs using player motivation theories and an ensemble approach. In Privacy, Security, Risk and Trust (PASSAT) and 2011 IEEE Third International Conference on Social Computing (SocialCom), 2011 IEEE Third International Conference on. IEEE, 157--164.

[2]

Ionut Brandusoiu and Gavril Toderean. 2013. Churn prediction modeling in mobile telecommunications industry using decision trees. Journal of Computer Science and Control Systems 6, 1 (2013), 14.

[3]

James Brightman. 2016. League of Legends generates 150m a month - SuperData. http://www.gamesindustry.biz/articles/2016-06-09-league-of-legends-generates-usd150m-a-month-superdata. (2016).

[4]

Chris Chambers, Wu-chang Feng, Sambit Sahu, and Debanjan Saha. 2005. Measurement-based characterization of a collection of on-line games. In Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement. USENIX Association, 1--1.

Digital Library

[5]

Kristof Coussement and Koen W De Bock. 2013. Customer churn prediction in the online gambling industry: The beneficial effect of ensemble learning. Journal of Business Research 66, 9 (2013), 1629--1636.

[6]

David R Cox. 1992. Regression models and life-tables. In Breakthroughs in statistics. Springer, 527--541.

[7]

Wu-chang Feng, David Brandt, and Debanjan Saha. 2007. A long-term study of a popular MMORPG. In Proceedings of the 6th ACM SIGCOMM Workshop on Network and System Support for Games. ACM, 19--24.

Digital Library

[8]

John Hadden, Ashutosh Tiwari, Rajkumar Roy, and Dymitr Ruta. 2007. Computer assisted customer churn management: State-of-the-art and future trends. Computers & Operations Research 34, 10 (2007), 2902--2917.

Digital Library

[9]

Fabian Hadiji, Rafet Sifa, Anders Drachen, Christian Thurau, Kristian Kersting, and Christian Bauckhage. 2014. Predicting player churn in the wild. In Computational intelligence and games (CIG), 2014 IEEE conference on. IEEE, 1--8.

[10]

David W Hosmer Jr and Stanley Lemeshow. 1999. Applied survival analysis: regression modelling of time to event data (1999). Eur Orthodontic Soc (1999), 561--2.

Digital Library

[11]

Shin-Yuan Hung, David C Yen, and Hsiu-Yu Wang. 2006. Applying data mining to telecom churn management. Expert Systems with Applications 31, 3 (2006), 515--524.

[12]

Deborah Kaminski and Cheryl Geisler. 2012. Survival analysis of faculty retention in science and engineering by gender. Science 335, 6070 (2012), 864--866.

[13]

Edward L Kaplan and Paul Meier. 1958. Nonparametric estimation from incomplete observations. Journal of the American statistical association 53, 282 (1958), 457--481.

[14]

Jaya Kawale, Aditya Pal, and Jaideep Srivastava. 2009. Churn prediction in MMORPGs: A social influence based approach. In Computational Science and Engineering, 2009. CSE'09. International Conference on, Vol. 4. IEEE, 423--428.

Digital Library

[15]

P Kollar. 2017. The Past, Present and Future of League of Legends Studio Riot Games, Polygon Platform, 2016. http://www.polygon.com/2016/9/13/12891656/the-past-present-and-future-of-league-of-legends-studio-riot-games. (2017).

[16]

Junxiang Lu. 2002. Predicting customer churn in the telecommunications industry---An application of survival analysis modeling using SAS. SAS User Group International (SUGI27) Online Proceedings (2002), 114--27.

[17]

M. Milosevic, N. Zivic, and I. Andjelkovic. 2017. Early churn prediction with personalized targeting in mobile social games. Expert Systems with Applications (2017).

[18]

Guangli Nie, Wei Rowe, Lingling Zhang, Yingjie Tian, and Yong Shi. 2011. Credit card churn forecasting by logistic regression and decision tree. Expert Systems with Applications 38, 12 (2011), 15273--15285.

Digital Library

[19]

Noppon Prakannoppakun and Sukree Sinthupinyo. 2016. Skill rating method in multiplayer online battle arena. In Electronics, Computers and Artificial Intelligence (ECAI), 2016 8th International Conference on. IEEE, 1--6.

[20]

Fred Reichheld. 2001. Prescription for cutting costs. Bain & Company. Boston: Harvard Business School Publishing (2001). http://www.bain.com/IMages/BB_Prescription_cutting_costs.pdf

[21]

François Rioult, Jean-Philippe Métivier, Boris Helleu, Nicolas Scelles, and Christophe Durand. 2014. Mining tracks of competitive video games. AASRI Procedia 8 (2014), 82--87.

[22]

Samuli Ripatti and Juni Palmgren. 2000. Estimation of multivariate frailty models using penalized partial likelihood. Biometrics 56, 4 (2000), 1016--1022.

[23]

Julian Runge, Peng Gao, Florent Garcin, and Boi Faltings. 2014. Churn prediction for high-value players in casual social games. In Computational Intelligence and Games (CIG), 2014 IEEE Conference on. IEEE, 1--8.

[24]

David Schoenfeld. 1982. Partial residuals for the proportional hazards regression model. Biometrika 69, 1 (1982), 239--241.

[25]

Matthias Schubert, Anders Drachen, and Tobias Mahlmann. 2016. Esports analytics through encounter detection. In Proceedings of the MIT Sloan Sports Analytics Conference.

[26]

Rafet Sifa, Christian Bauckhage, and Anders Drachen. 2014. The Playtime Principle: Large-scale cross-games interest modeling. In Computational Intelligence and Games (CIG), 2014 IEEE Conference on. IEEE, 1--8.

[27]

R. Sifa, F. Hadiji, J. Runge, A. Drachen, K. Kersting, and C. Bauckhage. 2015. Predicting Purchase Decisions in Mobile Free-to-Play Games. In Proc. of AAAI AIIDE.

[28]

Adam Summerville, Michael Cook, and Ben Steenhuisen. 2016. Draft-Analysis of the Ancients: Predicting Draft Picks in DotA 2 using Machine Learning. In Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference.

[29]

Superdata. 2016. 2016 MMO and MOBA Games Market. (2016). https://www.superdataresearch.com/market-data/mmo-market/

[30]

Superdata. 2016. Year in Review, December 2016. (2016). https://www.superdataresearch.com/market-data/market-brief-year-in-review/

[31]

Superdata Research. 2017. European Esports Conference Brief (http://strivesponsorship.com/wp-content/uploads/2017/04/Superdata-2017-esports-market-brief.pdf). (2017). http://strivesponsorship.com/wp-content/uploads/2017/04/Superdata-2017-esports-market-brief.pdf

[32]

Pin-Yun Tarng, Kuan-Ta Chen, and Polly Huang. 2009. On prophesying online gamer departure. In Proceedings of the 8th Annual Workshop on Network and Systems Support for Games. IEEE Press, 16.

Digital Library

[33]

Thanasis Vafeiadis, Konstantinos I Diamantaras, George Sarigiannidis, and K Ch Chatzisavvas. 2015. A comparison of machine learning techniques for customer churn prediction. Simulation Modelling Practice and Theory 55 (2015), 1--9.

[34]

Markus Viljanen, Antti Airola, Jukka Heikkonen, and Tapio Pahikkala. 2017. Playtime Measurement with Survival Analysis. arXiv preprint arXiv:1701.02359 (2017).

[35]

Markus Viljanen, Antti Airola, Anne-Maarit Majanoja, Jukka Heikkonen, and Tapio Pahikkala. 2017. Measuring Player Retention and Monetization using the Mean Cumulative Function. arXiv preprint arXiv:1709.06737 (2017).

[36]

Markus Viljanen, Antti Airola, Tapio Pahikkala, and Jukka Heikkonen. 2016. Modelling user retention in mobile games. In Computational Intelligence and Games (CIG), 2016 IEEE Conference on. IEEE, 1--8.

[37]

Huiwen Wang, Bang Xia, and Zhe Chen. 2015. Cultural Difference on Team Performance Between Chinese and Americans in Multiplayer Online Battle Arena Games. Springer International Publishing, Cham, 374--383.

[38]

M. Wu, S. Xiong, and H. Iida. 2016. Fairness mechanism in multiplayer online battle arena games. In 2016 3rd International Conference on Systems and Informatics (ICSAI). 387--392.

[39]

Hanting Xie, Sam Devlin, Daniel Kudenko, and Peter Cowling. 2015. Predicting Player Disengagement and First Purchase with Event-frequency Based Data Representation. In Proc. of CIG.

Digital Library

[40]

Hong Zhang. 2008. Customer retention in the financial industry: An application of survival analysis. Ph.D. Dissertation. Purdue University.

Cited By

Park JLee GJeong JShin H(2024)Formation of a Pre-Experience System Using the Priming Effect at the Entry Stage of Mobile RPGsJournal of Digital Contents Society10.9728/dcs.2024.25.2.35525:2(355-364)Online publication date: 28-Feb-2024
https://doi.org/10.9728/dcs.2024.25.2.355
Kordyaka BLaato SWeber SNiehaves B(2024)Are stress and engagement in toxicity associated with sleep quality? A study with League of Legends playersProceedings of the ACM on Human-Computer Interaction10.1145/36771018:CHI PLAY(1-17)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677101
Sagtani HJhawar MGupta AMehrotra RChen HDuh WHuang HKato MMothe JPoblete B(2023)Quantifying and Leveraging User Fatigue for Interventions in Recommender SystemsProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592044(2293-2297)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3592044
Show More Cited By

Index Terms

Player retention in league of legends: a study using survival analysis

Recommendations

Using Machine Learning to Predict Game Outcomes Based on Player-Champion Experience in League of Legends
FDG '21: Proceedings of the 16th International Conference on the Foundations of Digital Games

League of Legends (LoL) is the most widely played multiplayer online battle arena (MOBA) game in the world. An important aspect of LoL is competitive ranked play, which utilizes a skill-based matchmaking system to form fair teams. However, players’ ...
Testing men's hormone responses to playing League of Legends

Esports, or competitive video gaming, has rapidly increased in online play and viewing. The popularity of esports such as League of Legends may derive in part because it features skills-based coalitional competition. Whereas a sizable literature focuses ...
Ranking Practices and Distinction in League of Legends
CHI PLAY '16: Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play

Player ranking is a common feature of competitive online games, but little research work has closely examined the ways it mediates player practices within this game genre. In this paper, we present a qualitative study of player practices around ranking ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ACSW '18: Proceedings of the Australasian Computer Science Week Multiconference

January 2018

404 pages

ISBN:9781450354363

DOI:10.1145/3167918

Conference Chair:
David Abramson

Copyright © 2018 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

CORE: Computing Research and Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 January 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

EPSRC
InnovateUK
AHRC

Conference

ACSW 2018

Sponsor:

CORE

ACSW 2018: Australasian Computer Science Week 2018

January 29 - February 2, 2018

Queensland, Brisband, Australia

Acceptance Rates

ACSW '18 Paper Acceptance Rate 49 of 96 submissions, 51%;

Overall Acceptance Rate 204 of 424 submissions, 48%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
1,192
Total Downloads

Downloads (Last 12 months)197
Downloads (Last 6 weeks)15

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Park JLee GJeong JShin H(2024)Formation of a Pre-Experience System Using the Priming Effect at the Entry Stage of Mobile RPGsJournal of Digital Contents Society10.9728/dcs.2024.25.2.35525:2(355-364)Online publication date: 28-Feb-2024
https://doi.org/10.9728/dcs.2024.25.2.355
Kordyaka BLaato SWeber SNiehaves B(2024)Are stress and engagement in toxicity associated with sleep quality? A study with League of Legends playersProceedings of the ACM on Human-Computer Interaction10.1145/36771018:CHI PLAY(1-17)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677101
Sagtani HJhawar MGupta AMehrotra RChen HDuh WHuang HKato MMothe JPoblete B(2023)Quantifying and Leveraging User Fatigue for Interventions in Recommender SystemsProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592044(2293-2297)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3592044
Formosa JO'Donnell NHorton ETürkay SMandryk RHawks MJohnson D(2022)Definitions of Esports: A Systematic Review and Thematic AnalysisProceedings of the ACM on Human-Computer Interaction10.1145/35494906:CHI PLAY(1-45)Online publication date: 31-Oct-2022
https://dl.acm.org/doi/10.1145/3549490
Fernández del Río AGuitart APeriánẽz Á(2021)A time series approach to player churn and conversion in videogamesIntelligent Data Analysis10.3233/IDA-19494025:1(177-203)Online publication date: 26-Jan-2021
https://dl.acm.org/doi/10.3233/IDA-194940
Sifa RDrachen ABlock FMoon SDubhashi AXiao HLi ZKlabjan DDemediuk S(2021)Archetypal Analysis Based Anomaly Detection for Improved Storytelling in Multiplayer Online Battle Arena GamesProceedings of the 2021 Australasian Computer Science Week Multiconference10.1145/3437378.3442690(1-8)Online publication date: 1-Feb-2021
https://dl.acm.org/doi/10.1145/3437378.3442690
Alexandrovsky DFriehs MGrittner JPutze SBirk MMalaka RMandryk RKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Serious Snacking: A Survival Analysis of how Snacking Mechanics Affect Attrition in a Mobile Serious GameProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445689(1-18)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445689
Flunger RMladenow AStrauss C(2021)Game Analytics—Business Impact, Methods and ToolsDevelopments in Information & Knowledge Management for Business Applications10.1007/978-3-030-77916-0_19(601-617)Online publication date: 16-Aug-2021
https://doi.org/10.1007/978-3-030-77916-0_19
Loria EMarconi A(2020)Reading Between the Lines – Towards an Algorithm Exploiting In-game Behaviors to Learn Preferences in Gameful SystemsProceedings of the 15th International Conference on the Foundations of Digital Games10.1145/3402942.3403016(1-12)Online publication date: 15-Sep-2020
https://dl.acm.org/doi/10.1145/3402942.3403016
Vítek A(2020)Cross-Game Modeling of Player's Behaviour in Free-To-Play GamesProceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3340631.3398677(384-387)Online publication date: 7-Jul-2020
https://dl.acm.org/doi/10.1145/3340631.3398677
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten