<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/hybrid-content-based-and-collaborative-filtering-recommendations-with-ordinal-logistic-regression-1-feature-engineering/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.exactness.net/post/159576751640"> The Exactness of Mind</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<figure data-orig-height="600" data-orig-width="450"><img alt="image" src="https://i1.wp.com/68.media.tumblr.com/f76912cd74e149c247bcc48f5bcec571/tumblr_inline_oof2kmYa0F1qa0hyw_540.jpg?resize=450%2C600" data-orig-height="600" data-recalc-dims="1" data-orig-width="450"></figure>
<p>I will use <a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/packages/ordinal/index.html">{ordinal}</a> clm() (and other cool R packages such as <a target="_blank" rel="nofollow" href="https://github.com/dselivanov/text2vec">{text2vec}</a><br>
as well) here to develop a hybrid content-based, collaborative<br>
filtering, and (obivously) model-based approach to solve the<br>
recommendation problem on the MovieLens 100K dataset in R. All R code<br>
used in this project <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders">can be obtained from the respective GitHub repository</a>; the chunks of code present in the body of the post illustrate the essential steps only. The MovieLens 100K dataset <a target="_blank" rel="nofollow" href="https://grouplens.org/datasets/movielens/100k/">can be obtained from the GroupLens research laboratory</a><br>
of the Department of Computer Science and Engineering at the University<br>
of Minnesota. The first part of the study introduces the new approach<br>
and refers to the feature engineering steps that are performed by the <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_1.R">OrdinalRecommenders_1.R</a> script (<a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders">found on GitHub</a>). The second part, to be published soon, relies on the R code in  <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_3.R">OrdinalRecommenders_3.R</a> and presents the model training, cross-validation, and analyses steps. The  <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_2.R">OrdinalRecommenders_2.R</a><br>
script encompasses some tireless for-looping in R (a bad habbit indeed)<br>
across the dataset only in order to place the information from the<br>
dataset in the format needed for the modeling phase. The study aims at<br>
(a) the demonstration of the improvement in predicted ratings for<br>
recommending on a well-known dataset, and (b) attempts to shedd light on<br>
the importance of various types of information in the work of<br>
recommendation engines. Consequently, the code is not suited for use in<br>
production; additional optimizations are straightforward, simple, and<br>
necessary as well.</p>
<p><b>Introduction</b></p>
<p>Avoiding the formal exposition of the problem<br>
altogether, the recommendation problem in the context of contemporary<br>
online markets can expressed in the following way: given (A) the<br>
information on the user’s past ratings of various items (i.e. the  user-item ratings;<br>
for example, the ratings on a 5-point Likert type scale of the movies<br>
or TV shows that she or he has watched in the past, the ratings on<br>
similar scales of the items purchased, and similar), (B) the information<br>
on the user-item ratings of other users, © the attributes of users<br>
and/or items (c.f. the user’s gender, geographical location, age,<br>
occupation, and similar; the genre of the movie, product type, and<br>
similar for items), provide a prediction of the user’s rating of a novel, previously unassessed item.<br>
The predictions are then used to select the novel items (or lists of<br>
items) that a particular user would – by assumption – enjoy to assess,<br>
and eventualy purchase, in the future. The cognitive systems used for<br>
making such predictions are known as recommendation engines, or recommender systems, and are widely used nowadays across the Internet business.</p>
<p>It is generally recognized that recommendation engines can be grouped in two broad categories: (1) content-based systems,  or (2) collaborative filtering systems, with the later additionally described as (1.1) neighborhood or (1.2) model-based methods [1]. Content based<br>
systems (CF) rely on a typical description of items over feature<br>
vectors, and then recommend novel items to users by computing some<br>
similarity metric between them and the items that the user has already<br>
rated. Collaborative filtering systems, on the<br>
other hand, rely on the assumption that the covariations between the<br>
ratings (a) from different users over the same set of items, or (b) to<br>
different items from the same set of users, implicitly carry the<br>
information on item and user attributes. The information about the true<br>
user and item attributes are thus latent in CF. In the neighborhood (or memory) variant of CF, these approaches use the user-item ratings directly and proceed by aggregating the ratings of similar users (user-based CF) and/or similar items (item-based CF)<br>
in order to provide an estimate of the user’s rating of a novel item,<br>
typically weighting the known ratings of similar users/items by the<br>
respective proximity/similarity measures. In the model-based<br>
variant of CF, the systems tries to discover the structure of the<br>
latent factors from the observed behavioral measures (ie. known<br>
user-item ratings), and then a myriad of machine learning algortihms and<br>
statistical approaches can be used to train a predictive model for the<br>
novel user-item ratings of novel items.</p>
<p>I will here introduce a new hybrid approach to<br>
recommender systems with the following characteristics. First, the<br>
system is both content-based and CF. The content-based component of the<br>
system encompasses two matrices: the user-user and the item-item<br>
proximity matrices, both obtained from applying the relevant distance<br>
metric over a set of features that characterize users and items,<br>
respectively. The CF component of the system, relies on the typical<br>
user-user and item-item similarity matrices computed from the known,<br>
past user-item ratings, providing for a memory component of the<br>
recommender. Second, the system is model-based in the sense that it<br>
combines the information from these four matrices in an ordinal logistic<br>
regression model to predict the ratings of novel items on a 5-point<br>
Likert scale. The underlying logic of this approach is to try to express<br>
the recommendation problem as a discrete choice problem, where the<br>
alternatives are provided by ordinal information from a 5-point Likert<br>
scale, while the decision makers and the choice context are described by<br>
a set of content-based and memory-based features, all referring to some<br>
degree of closeness between the unknown user/item combination with the<br>
known user/item in two spaces: the attribute (feature) space and the<br>
neighborhood (memory) space. Because the CF approaches rely on<br>
users-user (or item-item) proximity, the question of how many users (or<br>
items) that are similar to the context of the present, novel user-item<br>
rating should be used in prediction arises naturally. By expressing the<br>
recommender problem as a regression problem essentially, we become able<br>
to judge the significance of additional user-user and/or item-item<br>
similarity information from the change in regression coefficients<br>
obtained by progressively adding more and more information to nested<br>
ordinal logistic models and training them. Thus the new approach also<br>
provides and excellent opportunity for a comparison between<br>
content-based information and memory-based information (the later as<br>
being typically used in CF): which one is more important? Given the<br>
assumption that the former is implicitly present in the later, i.e. that<br>
it presents latent information in the CF approach, does the usage of<br>
manifest memory-based information on user-item ratings completelly<br>
eliminates the need for a content-based description? If not, how the two<br>
should be combined? I will demonstrate how the present approach<br>
resolves this dilemma while resulting in highly improved recommendations<br>
on a well-known and widely used MovieLens dataset, at the same time<br>
reducing drastically the number of features that are needed to build the<br>
relevant prediction model.</p>
<p><b>Motivation</b></p>
<p>Consider the following user-item ratings matrix, encompassing 10 items (columns) and three users (rows):</p>
<figure data-orig-height="88" data-orig-width="450"><img alt="image" src="https://i2.wp.com/68.media.tumblr.com/7e304a138a7973a55756bea229b58339/tumblr_inline_oof2s6ju2X1qa0hyw_540.png?resize=450%2C88" data-orig-height="88" data-recalc-dims="1" data-orig-width="450"></figure>
<p>In this rating matrix, the ratings of user1 and user2<br>
have a maximal Pearson correlation of 1; however, the ratings of user2<br>
and user3 have a linear correlation of .085, sharing around .007 % of<br>
variance only. Upon a closer look, user1 and user2 have only three<br>
common ratings (for items i1, i4, and i5), while user2 and user3 have<br>
rated exactly the same items (i1, i3, i4, i5, i8, i9, and i10). Imagine<br>
that the items rated by user2 and user 3 are all Sci-Fi movies; we can<br>
imagine these two users chating and challenging each other’s views on<br>
the contemporary Sci-Fi production on online forums or fan conferences,<br>
right? They do not have to have perfectly similar tastes, right, but in<br>
fact  they are similar: simply because they both enjoy Sci-Fi. The <a target="_blank" rel="nofollow" href="https://en.wikipedia.org/wiki/Jaccard_index">Jaccard distance</a><br>
measures the proximity of two vectors A and B, both given over binary<br>
features (e.g. present vs. not present), by computing the ratio of (a)<br>
the difference between the union of A and B and the intersection of A<br>
and B to the (b) union of A and B; it ranges from zero to one and can be<br>
viewed as a proportion of common features that the two vectors share in<br>
the total number of features present in both. The Jaccard similarity<br>
coefficient is given by one minus the Jaccard distance; we will use the<br>
{proxy} dist() function to compute these from our ratings matrix:</p>
<pre>### --- Illustration<br>user1 &lt;- c(5, NA, NA, 4, 3, NA, 1, NA, NA, NA)<br>user2 &lt;- c(4, NA, 3, 3, 2, NA, NA, 1, 2, 4)<br># -- User1 and User2 have a high Pearson Correlation Coefficient:<br>cor(user1, user2, use = &quot;pairwise.complete.obs&quot;)<br>user3 &lt;- c(3, NA, 1, 5, 4, NA, NA, 2, 5, 4)<br># -- User2 and User3 have a low Pearson Correlation Coefficient:<br>cor(user2, user3, use = &quot;pairwise.complete.obs&quot;)<br># -- User2 and User3 have a high Jaccard similarity<br># -- i.e. 1 minus the Jaccard distance between them:<br>library(proxy)<br>as.numeric(1 - dist(t(user2), t(user3), method = &quot;jaccard&quot;))<br>as.numeric(1 - dist(t(user1), t(user2), method = &quot;jaccard&quot;))<br>as.numeric(1 - dist(t(user1), t(user3), method = &quot;jaccard&quot;))<br><br></pre>
<p>User2 and User3 who have provided the ratings for the<br>
same items exactly have a Jaccard similarity index of 1; they both have<br>
the Jaccard similarity index of .375 with User1. Then who is the user<br>
similar to User2 – the one whose ratings could be used to approximate<br>
the experiences of User2 with previously unseen items: (a) User1, with<br>
whom he has a memory-based Pearson correlation of 1 on three shared<br>
items, or (b) User3, with whom he has a Pearson correlation of only<br>
.085, but with whom he had shared the same experiences and thus achieved<br>
a maximum level of Jaccard similarity?</p>
<p>The value of the linear correlation coefficient (or<br>
cosine similarity, or any other similar metric) will not preserve the<br>
information on the number of shared items; that information is imlicitly<br>
present in the Type I Error test (for example), not in the extent of<br>
covariance itself. The Jaccard distance, on the other hand, neglects the<br>
covariances from the rating scales altogether, preserving only the<br>
information about the extent of the shared ratings. If we think about<br>
the items rated by a particular user, neglecting the particular rating<br>
values, we understand that we can describe some aspects of the user’s<br>
taste by binarizing his or her ratings; we can know, for example, the<br>
frequency distribution of the item types that he or she has decided to<br>
assess at all. If we additionally compute a distance or similarity<br>
matrix over that information for all users, we express the content-based<br>
approach in terms of the attributes that describe a particular use by<br>
its distance or similarity from other users in terms of the very same<br>
attributes – in a content-based description space. We can add the<br>
avaialable demographic information to the binary vectors before<br>
computing the similarities too and thus enrich the representation. And<br>
we can do the same for items, of course. As we already have the<br>
similarity matrices from linear correlations between the paired users’<br>
ratings at our disposal, we can combine the information from these two<br>
approaches – the content-based and the memory, or neighborhood-based –<br>
and try to predict the unobserved user-item ratings from a unified<br>
model.</p>
<p>The details of feature engineering along these lines in R are provided in the respective section below.</p>
<p><b>Dataset</b></p>
<p>The MovieLens 100K dataset [2] comprises 100,000<br>
ratings (Likert type 1 to 5 scale) from 943 users on 1682 movies;<br>
additionally, there are at least 20 movies rated per user. Some<br>
demographic information for the users is present – age, gender,<br>
occupation, zip – as well as the genre and the release dates for movies.<br>
The dataset <a target="_blank" rel="nofollow" href="https://grouplens.org/datasets/movielens/100k/">can be downloaded from the GroupLens research laboratory</a><br>
of the Department of Computer Science and Engineering at the University<br>
of Minnesota. Historically, it is among the most popular datasets used<br>
to test the new approaches to the development of recommendation engines.<br>
MovieLens 100K can be also obtained from <a target="_blank" rel="nofollow" href="https://www.kaggle.com/prajitdatta/movielens-100k-dataset">Kaggle</a> and <a target="_blank" rel="nofollow" href="https://datahub.io/dataset/movielens/resource/e2117a93-4fd4-41c3-b0e8-6a8ff8b1ad09">Datahub</a>. Our  <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_1.R">OrdinalRecommenders_1.R</a><br>
script reads in the components of this dataset and computes the desired<br>
user-user and item-item distance and similarity matrices following some<br>
data reshaping.</p>
<p><b>Feature engineering</b></p>
<p>Part 1.A of the  <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_1.R">OrdinalRecommenders_1.R</a><br>
script is used merely to read the original MovieLens 100K dataset<br>
files, rename the columns present there, and save as .csv files; you<br>
will need to run it too in order to use the code from part 1.B and the<br>
remaining scripts. In Part 1.B we begin to operate over the following<br>
three data.frames: (1) the ratingsData, which comprises the ratings on<br>
5-point scales for all user-item pairs from the dataset, (2) usersData,<br>
which comprises demographic information on users (Gender, Age,<br>
Occupation, Zip-code), and (3) moviesData, which comprises all item<br>
information (Title, Release Date, and Genre). Part 1.B of this R script<br>
performs the essentialy feature engineering operations in order to<br>
produce the following four matrices:</p>
<p>Matrix P1 – the Item-Item Proximity Matrix:<br>
the Jaccard distances between the items, computed from the binarized<br>
features encompassing relase decade (derived from the original release<br>
year information, provided in the movie Title column) and movie genre.<br>
This matrix plays the role of the content-based representation of items. The dist2() function from {text2vec} is used on a sparse Matrix class to compute the Jaccard distances here:</p>
<pre>### --- Compute moviesDistance w. Jaccard {text2vec} from movie genres<br>moviesData &lt;- moviesData %>% separate(col = Date, <br>                                      into = c('Day', 'Month','Year'), <br>                                      sep = &quot;-&quot;)<br>moviesData$Day &lt;- NULL<br>moviesData$Month &lt;- NULL<br>moviesData$Year &lt;- as.numeric(moviesData$Year)<br>range(moviesData$Year)<br># - that would be: [1] 1922 1998<br># - Introduce Movie Decade in place of Year:<br>decadeBoundary &lt;- seq(1920, 2000, by = 10)<br>moviesData$Year &lt;- sapply(moviesData$Year, function(x) {<br>  wL &lt;- x &lt; decadeBoundary<br>  wU &lt;- x >= decadeBoundary<br>  if (sum(wL) == length(decadeBoundary))  {<br>    return(1)<br>  } else if (sum(wU) == length(decadeBoundary)) {<br>    decadeBoundary[length(decadeBoundary)] <br>  } else {<br>    decadeBoundary[max(which(wL-wU == -1))]<br>  }<br>}) <br># - Match moviesData$Year with ratingsData:<br>mD &lt;- moviesData %>% <br>  select(MovieID, Year)<br>ratingsData &lt;- merge(ratingsData, mD,<br>                     by = 'MovieID')<br># - Movie Year (now Decade) as binary: <br>moviesData &lt;- moviesData %>%<br>  spread(key = Year,<br>         value = Year,<br>         fill = 0,<br>         sep = &quot;_&quot;)<br># - compute moviesDistance:<br>moviesDistance &lt;- moviesData[, 3:ncol(moviesData)]<br>w &lt;- which(moviesDistance > 0, arr.ind = T)<br>moviesDistance[w] &lt;- 1<br>moviesDistance &lt;- dist2(Matrix(as.matrix(moviesData[, 4:ncol(moviesData)])), <br>                        method = &quot;jaccard&quot;)<br>moviesDistance &lt;- as.matrix(moviesDistance)<br>rm(moviesData); gc()<br># - save objects and clear:<br>numMovies &lt;- length(unique(ratingsData$MovieID))<br>write_csv(as.data.frame(moviesDistance), <br>          path = paste0(getwd(),'/moviesDistance.csv'), <br>          append = F, col_names = T)<br>rm(moviesDistance); gc()
</pre>
<p>Matrix P2 – the User-User Proximity Matrix:<br>
the Jaccard distances between the users, computed from the binarized<br>
features representing whether a particular user has or has not rated a<br>
particular item only. This matrix plays the role of the content-based representation of users:</p>
<pre>### --- produce binary User-Item Matrix (who rated what only):<br>userItemMat &lt;- matrix(rep(0, dim(usersData)[1]*numMovies),<br>                      nrow = dim(usersData)[1],<br>                      ncol = numMovies)<br>userItemMat[as.matrix(ratingsData[c('UserID', 'MovieID')])] &lt;- 1<br>rm('w', 'ratingsData', 'usersData'); gc()<br>### --- Compute userDistance w. Jaccard {text2vec}<br>userItemMat &lt;- Matrix(userItemMat)<br>usersDistance &lt;- dist2(userItemMat, <br>                       method = &quot;jaccard&quot;)<br>rm(userItemMat); gc()<br>usersDistance &lt;- as.matrix(usersDistance)<br>write_csv(as.data.frame(usersDistance), <br>          path = paste0(getwd(),'/usersDistance.csv'), <br>          append = F, col_names = T)<br>rm(usersDistance); gc()
</pre>
<p>Note that I’ve missed to use the user demographics<br>
available in MovieLens 100K. In this project, I will not use them at<br>
all, but the general ratio is: because I am going after an ordinal<br>
logistic regression model, I can use these information as regressors<br>
there. Also, as I have computed the user-user distance by looking at<br>
what movies ratings they have in common, presumeably capturing some<br>
information on shared tastes, I could have done the some for the items,<br>
by inspecting what movies have been rated by the same users. However, a<br>
hunch told me that I might be end up using redundand feature<br>
representations in the same model; I’ve decided to describe the movie<br>
similarities from content-based features directly (genre and the decade<br>
of production), and user similarities from content-based features that<br>
encompass the similarity in tastes (the common movie ratings among<br>
them).</p>
<p>The following two similarity matrices are those<br>
typically used in user and item-based CF; they comprise Pearson<br>
correlation coefficients computed directly from the ratings matrix for<br>
1681 items (note: one movie was ‘unknown’, so I have removed it from the<br>
dataset and cleaned up from the ratings matrix accordingly) and 943<br>
users. Given the moderate size of the desired correlation matrices, I<br>
have used the {base} cor() function on pairwise complete observations to<br>
compute them. For digging out correlation or cosine matrices from<br>
problems larger than the MovieLens 100K, you might want to take a look<br>
at the functions provided by my colleague Stefan Nikolić in his “<a target="_blank" rel="nofollow" href="https://www.smartcat.io/blog/2016/recommender-systems-matrix-operations-for-fast-calculation-of-similarities/">Recommender Systems: Matrix operations for fast calculation of similarities</a>” (supporting the computations for CF in an order of magnitude lesser time than the popular <a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/packages/recommenderlab/index.html">{recommenderlab}</a> R package). The code produces the remaining two matrices:</p>
<p>Matrix S1 – the User-User Similarity Matrix:<br>
the pairwise complete Pearson correlation coefficients between the<br>
users’ ratings from the ratings matrix directly. This matrix plays the<br>
role of the memory-based representation of users:</p>
<pre>### --- Compute User-User and Item-Item Ratings Similarity Matrices<br>ratingsData &lt;- read_csv(&quot;ratingsData_Model.csv&quot;,<br>                        col_names = T)<br>ratingsData$X1 &lt;- NULL<br># - User-Item Ratings Matrix<br>ratingsData$Timestamp &lt;- NULL<br>ratingsData &lt;- ratingsData %>% <br>  spread(key = MovieID,<br>         value = Rating, <br>         sep = &quot;_&quot;) %>% <br>  arrange(UserID)<br># - Pearson Correlations: User-User Sim Matrix<br>UserUserSim &lt;- ratingsData %>% <br>  select(starts_with(&quot;Movie&quot;))<br>UserUserSim &lt;- t(UserUserSim)<br>UserUserSim &lt;- cor(UserUserSim, <br>                   use = 'pairwise.complete.obs')<br>UserUserSim &lt;- as.data.frame(UserUserSim)<br>write_csv(UserUserSim, <br>          path = paste0(getwd(),'/UserUserSim.csv'), <br>          append = F, col_names = T)<br>rm(UserUserSim); gc()
</pre>
<p>Matrix S2 – the Item-Item Similarity Matrix:<br>
the pairwise complete Pearson correlation coefficients between the<br>
item’ ratings from the ratings matrix directly. This matrix plays the<br>
role of the memory-based representation of items:</p>
<pre># - Pearson Correlations: Item-Item Sim Matrix<br>ItemItemSim &lt;- ratingsData %>% <br>  select(starts_with(&quot;Movie&quot;))<br>rm(ratingsData); gc()<br>ItemItemSim &lt;- cor(ItemItemSim,<br>                   use = 'pairwise.complete.obs')<br>ItemItemSim &lt;- as.data.frame(as.matrix(ItemItemSim))<br>write_csv(ItemItemSim, <br>          path = paste0(getwd(),'/ItemItemSim.csv'), <br>          append = F, col_names = T)<br>rm(ItemItemSim); gc()
</pre>
<p><b>Summary</b></p>
<p>The idea is to combine the content-based with<br>
memory-based information in a unified model to solve the recommendation<br>
problem for online recommender systems. I have used the Jaccard<br>
distances to describe the content-based proximity between users, based<br>
on what items they have rated or not; on the other hand, the Jaccard<br>
distances were also used to described the proximity of items in a<br>
content-based space derived from the “essentialistic” item features like<br>
movie genres and the decade of their production. Additionally, the<br>
typical Pearson correlation similarity matrices – computed directly from<br>
the ratings matrix – were produced for both users and items. The two<br>
user-user and item-item matrices share the same respective<br>
dimensionality. In the next step, the idea is to use the neighborhoods<br>
of different size from these matrices (say, take p similar users from the content-based proximity and the memory-based user-user similarity matrices, and q<br>
similar items from the content-based proximity and the memory-based<br>
item-item matrices) as regressors in an ordinal logistic model to<br>
predict the ordinal 5-point ratings of novel user-item combinations. By<br>
inspecting the regression coefficients from these predictive models<br>
while varying the size of the neighborhood sets of users and items whose<br>
ratings will be used as regressors, I hope to be able to provide some<br>
conclusion on the relative importance of content-based and memory-based<br>
information for building efficient recommendation engines. We’ll see.<br>
But before skipping to  <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_3.R">OrdinalRecommenders_3.R</a> to play with the beautiful {ordinal} models, you might wish to run  <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_2.R">OrdinalRecommenders_2.R</a><br>
that will prepare the first 100 neighbours from the proximity and<br>
similarity matrices for you. As I told you, there’s a for loop there –<br>
and a bad one.</p>
<p><b>References</b></p>
<p>[1] Desrosiers, C. &amp; Karypis, G. (2010). A comprehensive survey of neighborhood-based recommendation methods. In Ricci, F., Rokach, L., Shapira, B. &amp; Kantor, P. B. (Eds), Recommender Systems Handbook. Springer: US. DOI:10.1007/978-0-387-85820-3.</p>
<p>[2] F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems (TiiS) 5, 4, Article 19 (December 2015), 19 pages. DOI=http://dx.doi.org/10.1145/2827872.</p>
<p>Goran S. Milovanović, PhD<br><a target="_blank" rel="nofollow" href="http://t.umblr.com/redirect?z=http%3A%2F%2FSmartCat&t=ZDE1YzY0NTYzZDVjZjRmYjRhOTlkYWMxMzAxZGYxN2U1NmFmODE3ZixRcWhrR0dpQw%3D%3D&b=t%3AfxfLHZZpYe9t9Oek1n3iNg&p=http%3A%2F%2Fwww.exactness.net%2Fpost%2F157582845120%2Fasknasa-whats-the-optimal-time-for-aliens-to&m=1">Data Science Consultant, SmartCat</a></p>
<figure data-orig-height="180" data-orig-width="180"><img src="https://i1.wp.com/68.media.tumblr.com/a5d2491db08ab66670efe27e60c1ee9c/tumblr_inline_oof3npzi1Q1qa0hyw_540.png?resize=180%2C180" data-orig-height="180" data-recalc-dims="1" data-orig-width="180"></figure>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.exactness.net/post/159576751640"> The Exactness of Mind</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/9mCpH21oUEo"></a></div>