QI Leap Analytics Inc REX A Cloud-Based Personalization Engine Ali Nadaf May 2017
Research shows that online shoppers lose interest after 60 to 90 seconds of searching[ 1] Users either find something interesting or they abandon the website To be competitive, businesses must display relevant items that grab a customer s attention Artificial intelligence engines that continuously learn and offer real-time personalization are the future of online interactions Personalization Drives Conversion Personalization enables customer engagement platforms to seamlessly provide compelling content to each individual user Historical data, trend and interactions are examples of data sources that enable machine learning technology to personalize each customer s experience on a 1:1 basis The addition of Artificial Intelligence (AI) enables the system to continually learn and therefore drive increasingly optimal experiences Customer engagement platforms including web sites, emails, and mobile devices that use personalization with machine learning report increased conversion rate on average of about 29%[ 7] Another feature important to consider while implementing personalization across different platforms is product recommendation Product recommendation systems are machine learning techniques which model users preferences from their past behavior and interactions[ 2] Recommendations as part of personalization systems explicitly suggest compelling content to a customer in real-time New generation of recommendation systems leveraging advanced machine learning methods can do this irrespective of whether the user is a new or a returning visitor Related to items you've viewed, Other products you may like, Inspired by your browsing history, Popular items, Trending items Customers Who Bought This Item Also Bought ; We have all seen these recommendations while browsing the web, be it on Netflix or Amazon or some other platforms This technology is now being democratized and made available to all businesses regardless of their size Personalization is rapidly becoming a core technology that all merchants with growing online presence will have to deploy to remain competitive Personalization technology enables businesses to speak directly to each customer by leveraging their customer knowledge not only from the company s proprietary data but from other sources including social media as well Therefore, personalization engines equipped with these features can convert regular visitors to loyal shoppers more effectively and increase numbers of sales as well as the average basket value Other studies have shown users prefer buying products from a list of relevant recommendations[3] REX 1
Background Personalization engines generally offer two main types of recommendation systems[4] : 1 Content-based Filtering: The system predicts what users like based on what they have liked in the past This mechanism uses the attributes of users and items, and identifies the connection and relevance between them 2 Collaborative Filtering: The system predicts what users like or may be interested in from what other similar users have liked in the past It relies on similar interactions by other users and recommends relevant items to the target user Content-Based Filtering Collaborative Filtering + = Hybrid System Combinations of two algorithms (hybrid systems) create much more effective recommendations These algorithms are popularized by big companies, like Amazon, Facebook and LinkedIn For instance, the idea of a scalable item-item collaborative filtering method developed in 2003 by Amazon and used since after as one of the methods in their recommender systems[5] This method understands if a user purchased item x, then he/she will probably purchase item y Although methods based on Collaborative Filtering have been successful, new applications of it reveal some of potential challenges[ 6] : Sparsity : In practice, in a dataset with a large item set, even active users may have interacted with 1% or 2% of the items Accordingly, a recommender system trained on this data is unable to make a precise recommendation to users Scalability : With a high number of users and items, a typical recommender system which may involve complex mathematical operations often suffers from serious scalability problems Cold-start : The situation in which there is usually insufficient information to produce reasonable product recommendations to users Items or users are new in this use case REX 2
Note that improper recommendations do more harm than help Users might be unhappy receiving irrelevant recommendations so that they initially ignore the recommendations and eventually, they may lose interest in the online merchant s platform Improper recommendations are hard to detect and prevent manually in general They are often domain specific and should be filtered out based on the userdependent context Therefore, it is important to have a sophisticated recommendation system optimized for the specific domain of products/customers a merchant is offering/selling to QI Leap Personalization Engine: REX To address these kinds of common issues, data scientists and developers in QI Leap Analytics have designed REX an effective personalization engine using state-of-the-art AI technologies and an innovative design of a hybrid of several collaborative-based and content-based filtering algorithms REX product recommendation engine can significantly improve user engagement in online and offline shopping The engine leverages all available internal and external data sources and monitors each user s transactions and tries to grab their attentions for the user specific context, interests and needs and optimizes itself over time through A/B testing REX 3
REX Performance: The experimental results for numerous industry-standard datasets show that REX outperforms other popular traditional approaches including Collaborative Filtering and Popular Items The Popular Items method is a common nonpersonalized recommendation approach which recommends the most popular products purchased or viewed by other users The following figures represent the performance of REX and traditional approaches on two benchmarks, Movielens and Online Retail Store[ 10] datasets Movielens dataset is a stable public benchmark against which researchers can invent and test personalized recommendations[ 9] The dataset contains 20 million transactions with 27,000 movies and 138,000 users REX utilizes user and item content information in addition to the transaction data to recommend the most relevant items to users The results of the experiment of comparing REX versus the other benchmark algorithms show that REX performs better in both cold start and non-cold start scenarios by 20% to over 40% respectively The second industry-standard data set used was from online retail transactions Online Retail is a transactional data-set of a UK-based online retail store which contains all the transactions occurring between 01/12/2010 and 09/12/2011 [11] The data has 541,909 transactions with 3,684 items and 4,372 users For this dataset, REX uses only the transaction data (not user and item content information) to understand user shopping behaviour A set of comprehensive experiments on this setting reveals that REX also outperforms other popular methods by 30% and 40% in cold start and non-cold start cases, respectively Similar results have been achieved for other datasets REX 4
Note, the results are highly sensitive to the quality of the given data However, the key finding is that REX can yield substantially superior personalization recommendations over industry standard techniques even with minimum information like limited transaction data REX Features REX contains following features: Solving cold start problems Complete cold start are cases in which no transaction information about users or items is available Incomplete cold start represents situations where few transactional information about users and items is available For complete cold start cases, REX uses sophisticated content-based techniques including Deep Learning and Semantic NLP to uncover deep knowledge about items and users QI Leap scientists have invented a state-of-the-art algorithm[ 8] that performs with only implicit data This technology distinguishes REX from the other engines and enables businesses to convert cold-start users to loyal shoppers Scalable Cloud Solutions REX can handle a high-volume of real-time data and provides real-time monitoring services including KPI, A/B and multivariate testing and recommendation insights (trending, diversity, Serendipity, etc) through its performance dashboard REX 5
Selects the optimum hybrid algorithm REX actively explores different combinations of several existing algorithms for optimal recommendations and automatically selects the combination that maximizes the metrics associated with company's objectives and improves itself over time Captures multiple modes of interest REX provides different types of recommendations like trending items, popular items, similar item to the target item, etc depending on the user s interest Makes your data ready for recommendation All data structures and categories can be imported into REX However, they might be not ready to be incorporated into personalization Our data scientists understand the hidden factors and features of your data and make it ready for personalization Personalized promotion optimization REX determines the optimum promotion price of a product based on users interests and past shopping behavior Conclusion In today s fast-paced market, customers increasingly demand rapid access to realtime personalized and customized services that meets their individual needs REX effectively understands the customer, popular trends and business requirements to make customer engagement platforms more compelling This results in reduced customer bounce rates, increased visit duration, additional page views, higher sales conversions and larger average cart sizes REX enables businesses to better understand their customers and make decisions that dramatically improve the customer experience and in turn their customer lifetime value (CLTV) Future extension of REX includes developing novel machine learning techniques to uncover relationships between the appearances of pairs of items and modeling the human notions of the visual relationships In addition, QI Leap scientists are going to further develop an adaptive recommendation platform which predicts and deals better with changing customer preferences and rapid domain drifts REX 6
About The Author Ali Nadaf, Lead Data Scientist at QI Leap Ali received his PhD in Mathematics at Simon Fraser University He has joined QI Leap Team in 2016 as a Lead Data Scientist from Complex System Modeling Group (CSMG) where he worked as a researcher He has also worked in the United Nations (UN) for six years focusing on modeling human trafficking Ali has demonstrated a record of scholarly accomplishments with patent and scientific papers in Machine Learning and Mathematics About QI Leap Qi-Leap Analytics is founded with the vision to constantly and continually translate the latest technological developments in big data analytics into impactful solutions To discover how QI Leap can help your company achieve its business objectives of better demand prediction by employing a mix of machine learning technology and big data analytics, please contact: Poya Haghnegahdar (CEO), admin@qileapcom For more information, visit: wwwqileapcom REX 7
References 1 Carlos A Gomez-Uribe and Neil Hunt 2015 The Netflix Recommender System: Algorithms, Business Value, and Innovation ACM Trans Manage Inf Syst 6, 4, Article 13 (December 2015), 19 pages DOI: https://doiorg/101145/2843948 2 Dietmar Jannach, Markus Zanker, Alexander Felfernig, and Gerhard Friedrich 2010 Recommender Systems: An Introduction (1st ed) Cambridge University Press, New York, NY, USA 3 The Power of Personalized Product Recommendations Retrieved May 10, 2017, from http:// wwwintelliversecom/blog/the-power-of-personalized-product-recommendations/ 4 Anand Rajaraman and Jeffrey David Ullman 2011 Mining of Massive Datasets Cambridge University Press, New York, NY, USA 5 Greg Linden, Brent Smith, and Jeremy York 2003 Amazoncom Recommendations: Item-to- Item Collaborative Filtering IEEE Internet Computing 7, 1 (January 2003), 76-80 6 FO Isinkaye, YO Folajimi, BA Ojokoh, Recommendation systems: Principles, methods and evaluation, Egyptian Informatics Journal, Volume 16, Issue 3, November 2015, Pages 261-273, ISSN 1110-8665, https://doiorg/101016/jeij201506005 7 Fortunecom Retrieved May 12, 2017, from http://fortunecom/2012/07/30/amazonsrecommendation-secret/ 8 Yubo Zhou, Ali Nadaf (2017), Embedded Collaborative Filtering for Cold Start Prediction, https://arxivorg/abs/170402552 9 Grouplensorg (2013) GroupLens Retrieved May 16, 2017, from https://grouplensorg/ datasets/movielens/ 10 Uciedu Retrieved May 16, 2017, from http://archiveicsuciedu/ml/datasets/online retail 11 Daqing Chen, Sai Laing Sain, and Kun Guo Data mining for the online retail industry: A case study of RFM model-based customer segmentation using data mining In: Journal of Database Marketing & Customer Strategy Management 193 (2012), pp 197 208 REX 8