Analysis of A Multi-Domain Recommender System

Tiffany Y. Tang1, Pinata Winoto1, and Robert Ziqin Ye2 Dept. of Computer Engineering, Konkuk University, Chungju-Si, Korea, {tiffany, pinata}@kku.ac.kr 2 Institute of Electronic and Information Engineering, Central South Forestry Univ. of Technology, China, yezqin@gmail.com
1

Abstract- Recommending items in a multi-domain environment are more challenging than that in a single-domain one. In this paper, we report our findings on uncovering the association between users interests of items across domains that are related to each other to a certain degree using two datasets collected from users with different cultural background. Results lead us to believe that due to cultural influences, user preferences over Western and Eastern items (both entertaining item and entertainer) are significantly different. In addition, results suggest the value of the cross-domain RS and reveal that crossdomain recommendation is sensitive to genre among certain domains (Book, Game and Movie), which is valuable for the algorithmic design and implementation of the RS. The results are valuable for the algorithmic design and implementation of the cross-domain RS targeting at users from different countries. Another most significant potential of cross-domain RS is its ability to exploit users versatile interest on items in different domains to make serendipity and novel recommendations especially in large-scale commercial systems.

strengthening customer loyalty, one of the goals in Relational Marketing [6, 7]. The same benefits can also be realized if cross-domain recommendation is employed; it is especially valuable if such cross-domain recommendation could be employed in large-scale commercial systems such as Amazon.com. The following example looks at the differences of making recommendations in a traditional and a crossdomain RS respectively. Example. Michelle Fong is in her early thirties, and enjoys watching the Fox TV series Sex and the City very much: she owns the DVDs of all seasons. A traditional RS would recommend Ugly Betty to her; while a cross-domain RS can recommend a wide range of items, from the Movie of the same title, the movie The Devil Wears Prada, to the books The Devil Wears Prada and the Lipstick Jungle. This example highlights the differences of the recommendation made by the two different mechanisms: in the traditional RS, recommendation is made on items of the same domain; while in the cross-domain RS, recommendation is made on items from different domains. As such, a crossdomain RS is capable of making use of the most of the items in the system, thus, demonstrating a much greater potential in systems which consist of items from a wide RS and conducted a series of study to uncover two different yet related issues that have been largely ignored: Whether or not, at the group level, cross-domain interest exist: a correlation analysis. Whether, at the individual level, a users interest in items in certain categories can be used to predict the likeness of related items in other categories: a computational analysis. However, the data in our previous study was obtained from a set of homogeneous users, that is, second-year college students from a university in Hong Kong [5]. Hence, it is appealing to do further empirical study to substantiate our understanding on the cross-domain RS through similar process but different dataset. Also, since most studies in RSs overlook cross-culture issue, in this subsequent study we repeat our experiment based on dataset collected from non-Hong Kong students; providing another study on cross-culture issues in cross-domain recommendation. The rest of the paper is organized as follows. In the next section, we will give a brief critical review on four major RS including our proposed cross-domain RS. Then, related work to motivate the necessity and value of the cross-domain RS will be presented. Further, we will focus on a follow-up study in which we intend to substantiate our previous study in [5]

I. INTRODUCTION When there is an information overflow, there is a need for a personalized recommendation which can provide a customized information space for users, and build a community of customers around specific goals or Web services. A successful personalization system would follow the steps of its users, automatically learning from groups of similar users and make recommendations accordingly. Recommender Systems (RSs) have been known to be able to do this based on either on consumed items by a user or implicit observations of the users likes and dislikes. For example, if a user explicitly indicates that he/she favors action movies starring Matt Damon, then he/she could be recommended movies like The Bourne Supremacy. In this case, the system is able to obtain users indicative preferences. Due to its great potential in easing users decision-makings, RS has been studied extensively both in research and industry [1-5]. In major RSs, the candidate items upon which various recommendation computations are made come from one domain only, a mechanism referred to as a single domain RS. That is, take an example of making recommendation in MovieLens consisting of millions of ratings on movies: a user will be receiving movies that are closely related to what he/she likes before, even if related books or TV series could also be recommended based on the user model. But since the domain consists only of movies, such multi-domain recommendation is not possible. However, the idea of making multi-domain recommendation looks similar to the cross-sell in marketing, which has many benefits including increasing the profits,

- 280 -

and compare their differences. Then, we conclude this paper by a discussion of outstanding issues and our future work. II.
RECOMMENDER SYSTEMS: A BRIEF REVIEW

A. Major Recommender Systems Generally, there are four major types of RSs: content-based, collaborative filtering approach, hybrid approach and knowledge-based approach. Content-based approaches make recommendations based on the contents of the items a user has experienced before [1,8]. As user profiles in the content-based approach are built through an association with the contents of the documents, the approach tends to be quite narrowly focused and with a bias towards highly scored documents. Thus, a user might be restricted to those documents that are very similar to the ones he/she has read before. In contrast, collaborative filtering approach (CF) largely relies on the numerical ratings users provide to make recommendation, thus, its simplicity and application-independency makes it the most commonly adopted technique in commercial RSs and the most studied in academic community, e.g. [3, 5, 9, 10, 11]. However, CFs reliance on user ratings leads to two key drawbacks to the overall performance of the system. Particularly, if an item has not received enough ratings from users, or if many users have not rated each item, correlation computations cannot be well performed. These two problems, also known as the first-rater and the data sparsity respectively, prevent users from seeking recommendation information on new items. Hybrid recommendation mechanism attempts to smooth out the drawbacks of the two approaches by learning and constructing a unified user profile through the combinations of them. Knowledge-based approach builds the user profile gradually by many forms of knowledge structure [12]. The Find-Me system is one example of such systems [12]. The restaurant recommender Entre allows users to provide incremental and refined critiques (such as show me more like Reco but less expensive) on the systems suggestions through rounds of interactions until an acceptable option is reached. Although the critique-like feedback is an inherently fuzzy form of feedback with limited abilities to guide the recommendation process and therefore solicit user preferences, in certain complex domains where the dimensions of product include many compound features, it can be of help [12,13]. B. Cross-Domain Recommendations In fact, in systems such as Amazon.com, the recommendation a user will receive will narrowly be in the same category (or domain) as he/she has searched/browsed (i.e. a cross-selling recommendation). For instance, a user will be recommended a list of books relevant, to a certain degree, to the book the user is currently viewing or has bought. This type of the recommendation is referred to as the typical recommendation mechanism or a single domain RS. Overtime, some users might have had experiences of receiving items from different categories. But the service can only be enjoyed after the users have had enough record with the system, especially having bought items from different categories, such

as books, toys, CDs, etc. In the cross-domain recommendations, ratings on items in one or more domains can be used to infer users interest in items in other related categories. A cross-domain recommendation is computed based on candidate items from not just one domain as in the traditional RSs. To illustrate the cross-domain RS, Fig. 1 captures the core behind this type of RS and highlights the difference between it and a typical one.

Fig. 1. An illustration of cross-domain RS and its difference with a typical RS

In our study, we intend to investigate the presence of users cross-domain interests through various analysis including statistical analysis as well as the computational analysis based on the typical CF approach on a large data set collected. C. Related Work Diversifying Returned Items in a Single-Domain RS Realizing the over-specialization of recommendations, [14] attempts to diversify the returned items by first assess the intra-list similarity metric to assess the topical diversity among the recommended items before returning them to end users. Despite it, the topic diversification can only, to some degree, alleviate the problem, since it works on the generated recommendation items which might already be limited to one domain. [4] proposed a hybrid CF to tackle the learner interest diversity issue. For instance, a user may show interest in items in Football and English, but the user may have only provided ratings on items on Football. In this case, the traditional CF cannot be used to deal with this issue, as it relies on co-rated items to make recommendation. Notice here that there is a major difference between this study and ours: the items under scrutiny are not so closely related, while in our study, we are more interested in items that are related. Cross-Sell in Marketing and Customer Behaviors Cross-selling is one of the marketing strategies aiming to promote the sale of additional products or services to a customer. Less frequently it is used to describe the sale of services to additional business units at an account or to different geographic units of a customer [15]. It has been extensively studied in the marking and consumer behavior community, to name a few [6, 7, 16]. Cross selling can be

- 281 -

exercised on products or services belonging to the same domain or different domains with an aim to build tighter customer relationships. Ratneshwar et al. [17] found out that, from the customer psychologys perspective, across-domain consideration is goal-driven. Particularly, when a single product category could not meet all salient goals, crossdomain buying patterns manifest the most. That said, one of the driving forces behind the cross-domain recommendation is to make both conscious and unconscious recommendations to users where the former could be the more obvious and direct ones, while the latter consists of items that lead to surprising yet satisfactory user feedbacks. III. EXPERIMENTS, RESULTS AND DISCUSSIONS We have performed a set of preliminary studies to investigate the inter-domain relationships in user-rating matrix which grounded the cross-domain RS [5]. The dataset is based on ratings given by 144 college students in Hong Kong over 522 items, consisting of 50065 valid ratings. Results suggest that recommendation accuracy is most influenced by the closeness between the crossed domains (from the source domain(s) to identify neighbors for target users to the target domain(s) where items are to be recommended): the closer, the better. Considering more domains in CF does not necessarily increase the accuracy of the recommendation. However, it has several benefits. For example, it can be used to tackle the cold-start problem, that is, when we lack user ratings on the target domain. It can also overcome the overspecialization problem that traditional RS suffers by introducing items from a wider range of domains. Since the data of our previous study are from homogeneous users, the practical implications are limited, which motivates this study. There are two main goals of this study: 1. To verify our previous results using different dataset, that is, to obtain the degree of correlations between users likes/dislikes across different item categories (a.k.a. domains). 2. To answer whether or not users demographical information should be considered when making recommendations, especially in a multi-domain setting. A. Participants In total, there are 333 users participated in our study from two different groups: 133 Hong Kong students (hereafter HKs) and 200 Mainland Chinese students (hereafter MCs). We select MCs in this comparative study because they share common items with HKs, that is all of them are Chinese, watching similar movies, listening to similar songs, etc., yet with different background, for examples, speak different dialects, have some different live-style or educational system, and been exposed to different information. Among 200 MCs, half are men, and the other half are women. Not all of them have received recommendation from such commercial systems as Amazon.com; the majority of them are not familiar with the term RS itself. Due to the limited space, in this paper we will report the analysis of MCs only. Reader may refer to [5] for the analysis of HKs.

B.

Analysis I. Group-Level Data Analysis and Results

Are there any associations between items in different domains? Note that since the ratings are ordered numerical rating, therefore, Spearman correlation is calculated and reported in Fig. 2.

Note.

***

p<.0.0001 ;

**

p<0.001; *p<0.05, significant level D = 0.05 . Gender makes

no difference in our analysis.

Fig. 2. Correlations among items in different domains

Consistent to our results in HKs [5], mixed results were obtained in MCs. In particular, MCs having positive feelings on a person as an actor/actress also exhibit strong interest of the same person as a singer (r =0.782), though much enthusiastic about the TV series or movies the actor/actress stars (r=0.488, r =0.407, respectively, p<0.0001). The three counterpart correlations in HKs are 0.844, 0.415, and 0.601 respectively. Moreover, those users who like a singer does not necessarily tend to enjoy a CD made by the same singer as shown by a moderate correlation (r =0.506, p<0.0001, consistent to r = 0.504 in HKs); though users who favors a singer tend to favor the songs by the singer (r =0.784, p<0.0001, which is much higher than r =0.440 in HKs). So are the strong preference association between a director and the movie he/she directs (r =0.81, p<0.0001). The results revealed consistent pattern between MCs and HKs. Some discrepancies can be explained as the results of different items being rated by each group. Next, we take a broader review on items of the same genre that appeared in different categories, and study their correlations. This approach will enable us to push up the data so as to further our understandings on the correlations between user preferences. Are there any associations between items of the same genre in different domains? Similar to our previous study [5], in the first step, we need to categorize TV series and movies based on established classification of major genres in IMDB, Amazon, and Yahoo Listings. In order to study the correlation between items across different domains, we make aggregation based on genres in all domains. For instance, for those movies in the comedy, drama and romance domains, we sum the ratings up as one aggregated domain known as {comedy, drama, romance}, the result is then used to correlate against books in this genre. The correlations among items in the aggregated theme of Action, Thriller and Suspense from Game, TV Series and Movie are computed and shown in Fig. 3. Those MCs who show positive feedbacks over TV Series, also show similar feelings on Books: the correlation between the ratings on items

- 282 -

in these two categories is r = 0.9 (compared to 0.975 in HKs), though the correlation is not significant. However, there exhibit very strong correlation between Game and Movie, Game and Book, and Movie and Book in this genre, with r =1.000, p<0.05, D= 0.05, compared to 0.967, 0.817, and 0.924 in HKs respectively. The results lead us to believe that crossdomain recommendation is sensitive to genre among certain domains (Book, Game and Movie) in both MCs and HKs. The chi-square test also confirms that there is a correlation between the rows and the columns of the table, F2=38.005, p < 0.001, df = 8, D = 0.05. The observed F2value is higher than critical value.

t=3.367, df = 16, p < 0.005 respectively, both at significant level D = 0.05. Not surprisingly, user preferences over Western and Eastern items (both entertaining item and entertainer) are very significant different, t = 0, df = 36, which is consistent with our hypothesis that due to the cultural background, although in our two studies, subjects are Chinese, but in our first study, subjects were largely and profoundly influenced by Western cultures; while subjects in the second study were raised without the same degree of Western cultural influences. In fact, in our second study, only 10% of the items are Western items, while in the first study, this percentage is much high over 50%. Although this finding is direct and natural, to our knowledge, no research has been done aiming at an understanding of whether cultural difference plays a key role in user preferences. From software usability engineerings perspective, the social issue should be considered when designing a RS. C. Analysis II. The Role of Demographical Information Our second major goal to answer whether or not cultural issues should be considered for the designer or owner of the commercial system making use of RS mechanisms, especially in a multi-domain setting. Many studies in marketing have shown that culture is an important factor that affect human consumption pattern [18, 19]. In our case, although the demographical distance between the two groups of users are close, their cultural differences cannot be ignored. Intuitively, a more homogeneous a group of users is, a more likely that the recommendation made within the group is accurate. Evaluation Protocol First, we only consider items that appear in both data sets (HKs and MCs). Overall, we have 11 books, 20 CDs, 9 Games, 36 Movies, 14 Songs, and 18 TV Series. Then, we will compare the correlation among co-rated items by users within/across group, categorized into 4 treatments in the pair wise of (Target Users, Neighbor Users), that is {(HKs, HKs), (HKs, MCs), (MCs, HKs), (MCs, MCs)}. To do that, we randomly select 100 target users from each group. Since approximately 20% of HKs are female, then we set the composition of our target users in MCs are 80 male and 20 female too. Then for each target user we seek the highest correlation between him/her with others in the same group or the other group. A comparison can then be made to see whether the target user can benefit from seeking recommendation from different group or not. In order to evaluate the performance of multi-domain CF, a cross-validation technique suggested as appropriate for smallsample experiments was used [5, 20]. The evaluation protocol is widely adopted in the RS community including [3, 5, 21] etc. In order to calculate the error, we consider items that are rated with 1 or 2 as bad items, and those rated with 4 or 5 as good items. Then, for each target user, we seek a closest neighbor, and calculate the consistency of good/bad items between them, and described in a confusion matrix. The confusion matrix will consist of predictive accuracy metrics to

Fig. 3. Correlations among items (theme: Action, Thriller, Suspense) in Game, TV

As for Movies, TV series, and Book on Romance, Drama and Comedy, a strong correlation is obtained between Movie and TV Series. The results are generally align with our belief that user interest across domains exist, which can be of important to diversify the recommendations if it can be incorporated into the recommendation process to inform the system. Do users prefer Eastern or Western items? We believe the culture difference play a key role in RSs. We are interested in whether or not there is a trend to favor Western or Eastern items by MCs users. To proceed, we group items into Western and Eastern ones, then we further split them into Entertainers and Entertaining Items. The result indicates that there is a very strong correlation between user preferences between Western Entertainer and Western Entertaining Item, r =1.000, p<0.05 at significant level D = 0.05. Take an example, those who like Will Smith also like iRobot. The result is consistent with that in HKs, where r = 0.972, p<0.005. Meanwhile, although users who enjoy Eastern Entertainer also show positive feelings towards Eastern Entertaining Item, r =.9, this correlation is not significant and weaker when compared with that regarding Western elements. However, it is stronger than the one we obtained from HKs, r = 0.766, yet inconclusive due to insignificant results. The chisquare test also confirms that there is a correlation between the rows and the columns of the table, F2=30.581, p < 0.0005, df = 8, D = 0.05. In order to further study whether there are some cultural effects in our problem domain, we performed t-test on a number of items. At the group level, we did not find many differences regarding user feedbacks on Action, Thriller, Suspense-themed Game, TV Series and Movie, t = 1.294, df = 36, p<0.5 (two tailed), D = 0.05. However, users differ significantly in their preferences over: 1), Romance, Drama and Comedy-themed TV Series, Movie and Book; and 2), Book and Game, where t = 2.153, df = 26, p < 0.05 (two tailed),

- 283 -

examine how close the RSs predicted ratings given by a neighbor are to the true user ratings. Results I: Correlation Analysis Fig. 4 shows the results of correlation analysis when we consider all items, Movies/TV Series only, or Movies/TV Series/Songs/CDs together. The results of cross-culture correlation are highlighted in bold font. As can be seen here, the results suggest the disadvantage of employing crossculture recommendation for HKs, which means that it is better for them to find neighbors in the same group, but not the reverse, which means more target user in MCs may find a better neighbor (higher correlation) in HKs. Also, from Fig. 4 we observe that the correlation increases as the number of domains used to find neighbors decreases (from the top table to the bottom table). However, since the above results are for Pearson correlation per se, they do not give any practical implication; hence, we need to analyze the accuracy of recommendation through CF method.

positive reduce from 66.8% in (HKs, HKs) to 57.9% in (HKs, MCs), while their false negative do not change significantly (55.0% and 56.1%). But the results are not symmetric, which means target users in MCs are indifferent for recommendation from HKs or MCs, as the true positive is 54.8% in (MCs, MCs) that insignificantly lower than 55.5% in (MCs, MCs). One may argue that the true negative of (MCs, HKs) is lower than that in (MCs, MCs). However, this result is not important in recommendation systems since they will not recommend bad item to the target users. We offer one explanation regarding the asymmetric property in cross-cultural recommendation: MCs have more diverse preferences as the average correlation among them is lower than that in HKs (cf. Fig. 5). If this is the reason, then we may conclude that crossculture recommendation is fine for online communities with diverse background, but not for those in homogeneous cultural background. However, more rigorous studies are needed before we can arrive at such a conclusion. Nonetheless, our study has shown that it may not always bad of getting recommendation from other group of users with different cultural background, especially in cross-domain settings.

Fig. 4. The average maximum correlation within and across groups based on ratings on all items (top), movies/TV series/songs/CDs (middle), and movies/TV series (bottom).

Results II: Recommendation Analysis Fig. 5 shows the results of CF based on a single closest neighbor for each target user in the form of confusion matrices. For each treatment, we report a confusion matrix of the number of correct/wrong predictions and its corresponding percentage. The matrix consists of four situations, that is, when a neighbor recommend a predicted good item and indeed rated as a good one by the target user (true positive), when the item is predicted as good but rated badly by the target user (false positive), when it is predicted as bad but turns out be favorable by the target user (false negative), and when it is predicted as a bad one and indeed rated as a bad one (true negative). Fig.5 also shows the total number of co-ratings used in this experiment, with the lowest number in treatment (MCs, HKs) equals to 2610, and the highest in (MCs, MCs) equals to 4046. The results of cross-culture CF are on the bottom-left (HKs, MCs) and upper-right part (MCs, HKs). The results suggest the disadvantage of employing crossculture recommendation for HKs, which means it is better for them to find neighbors in the same group, because the true

Fig. 5. The confusion matrices after CF.

IV. CONCLUDING REMARKS With an aim to cadet users full spectrum of interests and an influence of the cross-sell in marketing, we proposed the multi-domain recommendation and performed a series of studies. Our previous evaluation study revealed that the slight decrease of recommendation accuracy can be complemented by the diversity of returned items which is, to some degree, more essential to end users in terms of the utility of a RS. Of the types of cross-domain recommendation approaches, the results suggest that recommendation accuracy is most influenced by the closeness between the crossed domains (from the source domain(s) to identify neighbors for target users to the target domain(s) where items are to be recommended): the closer, the better. In current study, we have justified our conclusions through another dataset obtained from users with different cultural background. Overall, similar results are obtained and suggest the value of

- 284 -

the cross-domain RS. Our results also reveal that cross-domain recommendation is sensitive to genre among certain domains (Book, Game and Movie), which is valuable for the algorithmic design and implementation of the RS. In addition to the justification to our previous study, we have also performed cross-cultural study with respect to recommendation systems. The results show asymmetric property with respect to the recommendation accuracy on cross-cultural recommendation. Currently, we are planning to conduct a similar study to other group of users including senior citizens, middle-aged working citizens, to further our understanding on the multi-domain and cross-culture recommendation and its characteristics.

REFERENCES
[1] T. Joachims, D. Freitag, and T. Mitchell. WebWatcher: a tour guide for the world wide web. In Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI97), pp.770-775. 1997. J. Herlocker, J. Konstan, A. Borchers, and J. Riedl. An algorithmic framework for performing collaborative filtering. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR99), Berkeley, USA, pp.230-237. 1999. T.Tang, P. Winoto, and K.C.C. Chan. Scaling down candidate sets based on the temporal feature of items for improved hybrid recommendations. In B. Mobasher and S. S. Anand (Eds.) Intelligent Techniques in Web Personalization, LNAI 3169, pp.169-185, Springer, 2005. Y. Li, L. Liu and X. Li. A hybrid collaborative filtering method for multiple-interests and multiple-content recommendation in ECommerce. Expert Systems with Applications 28, pp.6777. 2005. P. Winoto, and T. Tang. If You Like the Devil Wears Prada the Book, Will You also Enjoy the Devil Wears Prada the Movie? A Study of Cross-Domain Recommendations. New Generation Computing, 26(3), pp. 209-225. 2008. A. Beckett, P. Hewer, and B. Howcroft. An exposition of consumer behavior in the financial services industry. International Journal of Bank Marketing. 18(1), pp.15-26. 2000. S. Worthington, and S. Horne. A new relationship marketing model and its application in the affinity credit card market. International Journal of Bank Marketing, 16(1), pp.39-44. 1998. D. Billsus, and M. Pazzani. A hybrid user model for news story classification. In Proceedings of the 7th International Conference on User Modeling, pp.99-108. 1999. P. Resnick, N. Iacouvou, M. Suchak, P. Bergstrom, and J. Riedl. GroupLens: an open architecture for collaborative filtering of Netnews. In Proceedings of ACM Computer-Supported Collaborative Work (CSCW94), Chapel Hill, NC. Addison-Wesley, pp.175-186. 1994. U. Shardanand, and P. Maes. Social information filtering: algorithms for automating word of mouth . In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, (ACM CHI1995), Denver, pp.210-217. 1995. J. Herlocker, J. Konstan, L.Terveen, and J. Riedl. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22 (1): 5-53. 2004. R. Burke. Hybrid recommender systems: survey and experiments. User Modeling and User Adaptive Interaction. 12(4): 331-370. 2002. B. Smyth, L. McGinty, J. Reilly, and K. McCarthy. Compound critiques for conversational recommender systems. In Proceedings of 2004 IEEE/WIC/ACM International Conference on Web Intelligence (WI 2004), pp.141-151. 2004. C-N. Ziegler, S. McNee, J. Konstan, and G. Lausen.Improving recommendation lists through topic diversification. In Proc. of the 14th International World Wide Web Conference (WWW '05), pp. 22-32. 2005. Wikipedia on Cross-selling. Wikipedia at

http://en.wikipedia.org/wiki/Cross-selling [16] B. Lariviere, and D. Van den Poel. Investigating the role of product features in preventing customer churn, by using survival analysis and choice modeling: the case of financial services. Expert Systems with Applications, 27, pp. 277-285. 2004. [17] S. Ratneshwar, C. Pechmann, and A.D. Shocker. Goal-Derived Categories and the Antecedents of Across-Category Consideration. The Journal of Consumer Research, vol. 23, no. 3, pp. 240-250. 1996. [18] C. Park, and J.K. Jun. A cross-cultural comparison of internet buying behaviour: Effects of internet usage, perceived risks, and innovativeness. International Marketing Review, 20 (5), pp.534553.2003. [19] M. Hermeking. Culture and Internet consumption: Contributions from cross-cultural marketing and advertising research. Journal of Computer-Mediated Communication, 11(1), article 10. 2005. [20] G. C. Cawley and N.L.C. Talbot. Efficient leave-one-out crossvalidation of kernel fisher discriminant classifiers. Pattern Recognition 36 (1), 25852592. 2003. [21] P. Melville, R. Mooney, and R. Nagarajan, R. Content-boosted collaborative filtering for improved recommendations. In Proceedings of the 18th National Conference on Artificial Intelligence (AAAI-2002), Edmonton, Canada, pp.187-192.2002.

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12] [13]

[14]

[15]

- 285 -