Roman Zykov
ABSTRACT
Retail Rocket helps web shoppers make better shopping decisions by providing personalized real-time recommendations through multiple channels with over 100MM unique monthly users and 1000+ retail partners. The rapid improvement of the product is important to win on the high-concurrency market of real-time personalization platforms. The necessity of introducing constant innovations and improvements of algorithms for recommendation systems requires correct tools and a process of rapid testing of hypotheses. It's not a secret that 9 out of 10 hypotheses actually do not improve the performance at least. We had the task stated as follows: How to detect and eliminate the idea that doesn't improve as early as possible, to spend a minimum of resources on that process. In the report we will talk about: How we make our process of hypotheses testing faster. One programming language for R&D. Enmity and friendship of offline and online metrics. Why it is difficult to predict the impact of changing diversity of algorithms. What is the benefit of AA/BB online tests. Bayesian statistics for the evaluation of online tests.
Roman Zykov is the Chief Data Scientist at the Retail Rocket. In Retail Rocket is responsible for algorithms of personalized and non-personalized recommendations. Previous to Retail Rocket, Roman was the Head of analytics at the biggest e-commerce companies for almost ten years. He received Ms.Sc. in applied mathematics and physics from the MIPhT in 2004.
Index Terms
- Hypothesis Testing: How to Eliminate Ideas as Soon as Possible
Recommendations
Improving recommendation lists through topic diversification
WWW '05: Proceedings of the 14th international conference on World Wide WebIn this work we present topic diversification, a novel method designed to balance and diversify personalized recommendation lists in order to reflect the user's complete spectrum of interests. Though being detrimental to average accuracy, we show that ...
Auralist: introducing serendipity into music recommendation
WSDM '12: Proceedings of the fifth ACM international conference on Web search and data miningRecommendation systems exist to help users discover content in a large body of items. An ideal recommendation system should mimic the actions of a trusted friend or expert, producing a personalised collection of recommendations that balance between the ...
Are All Rejected Recommendations Equally Bad?: Towards Analysing Rejected Recommendations
UMAP '19: Proceedings of the 27th ACM Conference on User Modeling, Adaptation and PersonalizationWhen evaluating algorithms that recommend a list of relevant items to a user, it is common to use metrics such as precision to measure the system accuracy. When computing precision, one computes the number of items that were selected by the user among ...
Comments