<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/hybrid-content-based-and-collaborative-filtering-recommendations-with-ordinal-logistic-regression-2-recommendation-as-discrete-choice/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.exactness.net/post/159577222155"> The Exactness of Mind</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<figure data-orig-height="600" data-orig-width="450"><img alt="image" src="https://i2.wp.com/68.media.tumblr.com/5f488c9c79c56c163f19df8b0806c49f/tumblr_inline_oof3behCeg1qa0hyw_540.jpg?resize=450%2C600" data-orig-height="600" data-recalc-dims="1" data-orig-width="450"></figure>
<p>In this continuation of “<a target="_blank" rel="nofollow" href="https://www.smartcat.io/blog/2017/hybrid-content-based-and-collaborative-filtering-recommendations-with-ordinal-logistic-regression-1-feature-engineering/">Hybrid content-based and collaborative filtering recommendations with {ordinal} logistic regression (1): Feature engineering</a>” I will describe the application of the <a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/packages/ordinal/index.html">{ordinal}</a><br>
clm() function to test a new, hybrid content-based, collaborative<br>
filtering approach to recommender engines by fitting a class of ordinal<br>
logistic (aka ordered logit) models to ratings<br>
data from the MovieLens 100K dataset. All R code used in this project<br>
can be obtained from the respective GitHub repository; the chunks of<br>
code present in the body of the post illustrate the essential steps<br>
only. The MovieLens 100K dataset <a target="_blank" rel="nofollow" href="https://grouplens.org/datasets/movielens/100k/">can be obtained from the GroupLens research laboratory</a><br>
of the Department of Computer Science and Engineering at the University<br>
of Minnesota. This second part of the study relies on the R code in <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_3.R">OrdinalRecommenders_3.R</a><br>
and presents the model training, cross-validation, and the analyses.<br>
But before we proceed to approach the recommendation problem from a<br>
viewpoint of discrete choice modeling, let me briefly remind you of the<br>
results of the feature engineering phase and explain what happens in <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_2.R">OrdinalRecommenders_2.R</a> which prepares the data frames that are passed to clm(). </p>
<p><b>Enter explanatory variables</b></p>
<p>In the feature engineering phase (<a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_1.R">OrdinalRecommender_1.R</a>)<br>
we have produced four – two user-user and two item-item – matrices. Of<br>
these four matrices, two are proximity matrices with Jaccard distances<br>
derived from binarized content-based features: for users, we have used<br>
the binary-featured vectors that describe each user by what movies he or<br>
she has or has not rated, while for items we have derived the proximity<br>
matrix from the binarized information on movie genre and the decade of<br>
the production. In addition, we have two Pearson correlation matrices<br>
derived from the ratings matrix directly, one describing the user-user<br>
similarity and the other the similarity between the items. The later two<br>
matrices present the memory-based component of the model, while the<br>
former carry content-based information. In <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_2.R">OrdinalRecommenders_2.R</a>, these four matrices are:</p>
<ul>
<li>usersDistance – user-user Jaccard distances,</li>
<li>UserUserSim, – user-user Pearson correlations,</li>
<li>itemsDistance – item-item Jaccard distances, and</li>
<li>ItemItemSim – item-item Pearson correlations.</li>
</ul>
<p>The <a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_2.R">OrdinalRecommenders_2.R</a> essentially performs only the following operations over these matrices: for each user-item ratings present in ratingsData (the full ratings matrix),</p>
<ul>
<li>from the user-user proximity/similarity matrix,<br>
select 100 most users who have rated the item under consideration and<br>
who are closest to the user under consideration;</li>
<li>place them in increasing order of distance/decreasing order of similarity from the current user;</li>
<li>copy their respective ratings of the item under consideration.</li>
</ul>
<p>For the item-item matrices, the procedure is similar:</p>
<ul>
<li>from the item-item proximity/similarity matrix,<br>
select 100 items that the user under consideration has rated and that<br>
are closest to the item under consideration;</li>
<li>place them in increasing order of distance/decreasing order of similarity from the current item;</li>
<li>copy the respective ratings of the item under consideration.</li>
</ul>
<p>Note that this procedure of selecting the nearest<br>
neighbours’ ratings produces missing data almost by necessity: a<br>
particular user doesn’t have to have 100 ratings of the items similar to<br>
the item under consideration, and for a particular user and item we<br>
don’t necessarily find 100 similar users’ ratings of the same item. This<br>
would reflect in a decrease of the sample size as one builds<br>
progressively more and more feature-encompassing ordered logit models.<br>
In order to avoid this problem, I have first reduced the sample size so<br>
to keep only complete observations for the most feature-encompassing<br>
model that I have planned to test: the one relying on 30 nearest<br>
neighbors from all four matrices (e.g., encompassing 4 * 30 = 120<br>
regressors, add four intercepts needed for the ordered logit over a<br>
5-point rating scale); 87,237 observations from the dataset were kept.</p>
<p>For the sake of completeness, I have also copied the<br>
respective distance and similarity measures from the matrices; while the<br>
first 100 columns of the output matrices present ratings only, the<br>
following 100 columns are the respective distance/similarity measures.<br>
The later will not be used in the remainder of the study. The resulting<br>
four matrices are:</p>
<ul>
<li>proxUsersRatingsFrame – for each<br>
user-item combination in the data set, these are the ratings of the item<br>
under consideration from the 100 closest neighbours in the user-user<br>
content-based proximity matrix;</li>
<li>simUsersRatingsFrame, – for each<br>
user-item combination in the data set, these are the ratings of the item<br>
under consideration from the 100 closest neighbours in the user-user<br>
memory-based similarity matrix;</li>
<li>proxItemsRatingsFrame – for each<br>
user-item combination in the data set, these are the ratings from the<br>
user under consideration of the 100 closest neighbours in the item-item<br>
content-based proximity matrix; and</li>
<li>simItemsRatingsFrame – for<br>
each user-item combination in the data set, these are the ratings from<br>
the user under consideration of the 100 closest neighbours in the<br>
item-item memory-based similarity matrix.</li>
</ul>
<p>Of course it can be done faster than for-looping over<br>
100,000 ratings. Next, the strategy. The strategy here is to<br>
progressively select a larger number of numSims – the number of nearest neighboours taken from proxUsersRatingsFrame, simUsersRatingsFrame, proxItemsRatingsFrame, and simItemsRatingsFrame – and pull them together with the ratings from the ratingsData. For example, if numSims == 5, we obtain a predictive model Rating ~ 4 matrices * 5 regressors which is 20 regressors in total, if numSims == 10, we obtain a predictive model Rating ~ 4 matrices * 10 regressors which is 40 regressors in total, etc.</p>
<p><b>Enter Ordered Logit Model</b></p>
<p>The idea to think of the recommendation problem as<br>
discrete choice is certainly not new; look at the very introductory<br>
lines of [1] where the Netflix competition problem was used to motivate<br>
the introduction of ordered regressions in discrete choice. We assume<br>
that each user has an unobservable, continuous utility function that<br>
maps onto his or her item preferences, and of which we learn only by<br>
observing the discrete information that is captured by his or her<br>
ratings on a 5-point scale. We also assume that the mapping of the<br>
underlying utility on the categories of the rating scale is controlled<br>
by threshold parameters, of which we fix the first to have a value of<br>
zero, and estimate the thresholds for 1→2, 2→3, 3→4, and 4→5 scale<br>
adjustments; these play a role of category-specific intercepts, and are<br>
usually considered as nuisance parameters in ordered logistic<br>
regression. In terms of the unobserved user’s utility for items, these<br>
threshold parameters “disect” the hypothesized continuous utility scale<br>
into fragments that map onto the discrete rating responses (see Chapter<br>
3, Chapter 4.8 in [1]). Additionally, we estimate a vector of regression<br>
coefficients, β, one for each explanatory variable in the model: in our case, the respective ratings from the numSims<br>
closest neighbours from the similarity and proximity matrices as<br>
described in the previous section. In the ordered logit setting – and<br>
with the useful convention of writing minus in front of the xiTβ term used in {ordinal} (see [2]) – we have:</p>
<figure data-orig-height="24" data-orig-width="188"><img alt="image" src="https://i1.wp.com/68.media.tumblr.com/d6b0c688eccc0902b09228c0010e14de/tumblr_inline_oof35eTQAi1qa0hyw_540.png?resize=188%2C24" data-orig-height="24" data-recalc-dims="1" data-orig-width="188"></figure>
<p>and given that</p>
<figure data-orig-height="19" data-orig-width="172"><img alt="image" src="https://i0.wp.com/68.media.tumblr.com/bcf7b2d5be077e3bb48d76e3e077c49f/tumblr_inline_oof35t0A6b1qa0hyw_540.png?resize=172%2C19" data-orig-height="19" data-recalc-dims="1" data-orig-width="172"></figure>
<p>the logit is cumulative; θj stands for the j-th response category-specific threshold, while i is the index of each observation in the dataset. Vectors x and β play the usual role. The interpretation of the regression coefficients in β becomes clear after realizing that the cumulative odds ratios take the form of:</p>
<figure data-orig-height="49" data-orig-width="182"><img alt="image" src="https://i0.wp.com/68.media.tumblr.com/38aecaf66c7099ce4188c8539baf3146/tumblr_inline_oof368KomZ1qa0hyw_540.png?resize=182%2C49" data-orig-height="49" data-recalc-dims="1" data-orig-width="182"></figure>
<p>and is independent of the response category j, so that for a unit change in x the cumulative odds ratio of the change in probability P(Yi ≥ j) increases by a multiplicative factor of exp(β).<br>
Back to the recommendation problem: by progressively adding more<br>
nearest neighbours’’ ratings from the user-user and item-item proximity<br>
and similarity matrices, and tracking the behavior of regression<br>
coefficients in predictions from the respective ordered logit models,<br>
one can figure out the relative importance of the memory-based<br>
information (from the similarity matrices) and the content-based<br>
information (from proximity matrices).</p>
<p>I have varied the numSims (the number of nearest neighbours used) as seq(5, 30, by = 5)<br>
to develop six ordinal models with 20, 40, 60, 80, 100, and 120<br>
regressors respectively, and performed a 10-fold cross-validation over<br>
87,237 user-item ratings from MovieLens 100K (<a target="_blank" rel="nofollow" href="https://github.com/smartcat-labs/OrdinalRecommenders/blob/master/OrdinalRecommenders_3.R">OrdinalRecommenders_3.R</a>):</p>
<pre> 
### --- 10-fold cross-validation for each:<br>### --- numSims &lt;- seq(5, 30, by = 5)<br>numSims &lt;- seq(5, 30, by = 5)<br>meanRMSE &lt;- numeric(length(numSims))<br>totalN &lt;- dim(ratingsData)[1]<br>n &lt;- numeric()<br>ct &lt;- 0<br>## -- Prepare modelFrame:<br># - select variables so to match the size needed for the most encompassing clm() model:<br>f1 &lt;- select(proxUsersRatingsFrame, <br>             starts_with('proxUsersRatings_')[1:numSims[length(numSims)]])<br>f2 &lt;- select(simUsersRatingsFrame, <br>             starts_with('simUsersRatings_')[1:numSims[length(numSims)]])<br>f3 &lt;- select(proxItemsRatingsFrame, <br>             starts_with('proxItemsRatings_')[1:numSims[length(numSims)]])<br>f4 &lt;- select(simItemsRatingsFrame, <br>             starts_with('simItemsRatings_')[1:numSims[length(numSims)]])<br># - modelFrame:<br>mFrame &lt;- cbind(f1, f2, f3, f4, ratingsData$Rating)<br>colnames(mFrame)[dim(mFrame)[2]] &lt;- 'Rating'<br># - Keep complete observations only;<br># - to match the size needed for the most encompassing clm() model:<br>mFrame &lt;- mFrame[complete.cases(mFrame), ]<br># - store sample size:<br>n &lt;- dim(mFrame)[1]<br># - Rating as ordered factor for clm():<br>mFrame$Rating &lt;- factor(mFrame$Rating, ordered = T)<br># - clean up a bit:<br>rm('f1', 'f2', 'f3', 'f4'); gc()<br>## -- 10-fold cross-validation<br>set.seed(10071974)<br># - folds:<br>foldSize &lt;- round(length(mFrame$Rating)/10)<br>foldRem &lt;- length(mFrame$Rating) - 10*foldSize<br>foldSizes &lt;- rep(foldSize, 9)<br>foldSizes[10] &lt;- foldSize + foldRem<br>foldInx &lt;- numeric()<br>for (i in 1:length(foldSizes)) {<br>  foldInx &lt;- append(foldInx, rep(i,foldSizes[i]))<br>}<br>foldInx &lt;- sample(foldInx)<br># CV loop:<br>for (k in numSims) {<br>  <br>  ## -- loop counter<br>  ct &lt;- ct + 1<br>  <br>  ## -- report<br>  print(paste0(&quot;Ordinal Logistic Regression w. &quot;,<br>               k, &quot; nearest neighbours running:&quot;))<br>  <br>  ### --- select k neighbours:<br>  modelFrame &lt;- mFrame[, c(1:k, 31:(30+k), 61:(60+k), 91:(90+k))]<br>  modelFrame$Rating &lt;- mFrame$Rating<br>  <br>  # - model for the whole data set (no CV):<br>  mFitAll &lt;- clm(Rating ~ .,<br>                 data = modelFrame)<br>  saveRDS(mFitAll, paste0(&quot;OrdinalModel_NoCV_&quot;, k, &quot;Feats.Rds&quot;))<br>  <br>  RMSE &lt;- numeric(10)<br>  for (i in 1:10) {<br>    # - train and test data sets<br>    trainFrame &lt;- modelFrame[which(foldInx != i), ]<br>    testFrame &lt;- modelFrame[which(foldInx == i), ]<br>    # - model<br>    mFit &lt;- clm(Rating ~ .,<br>                data = trainFrame)<br>    <br>    # - predict test set:<br>    predictions &lt;- predict(mFit, newdata = testFrame, <br>                           type = &quot;class&quot;)$fit<br>    <br>    # - RMSE:<br>    dataPoints &lt;- length(testFrame$Rating)<br>    RMSE[i] &lt;- <br>      sum((as.numeric(predictions)-as.numeric(testFrame$Rating))^2)/dataPoints<br>    print(paste0(i, &quot;-th fold, RMSE:&quot;, RMSE[i]))<br>  }<br>  <br>  ## -- store mean RMSE over 10 folds:<br>  meanRMSE[ct] &lt;- mean(RMSE)<br>  print(meanRMSE[ct])<br>  <br>  ## -- clean up:<br>  rm('testFrame', 'trainFrame', 'modelFrame'); gc()<br>}
</pre>
<p>The outer loop varies k in numSims,<br>
the number of nearest neighbours selected, then prepares the dataset,<br>
picks ten folds for cross-validation, and fits the ordered logit to the<br>
whole dataset before the cross-validation; the inner loop performs a<br>
10-fold cross-validation and stores the RMSE from each test fold; the<br>
average RMSE from ten folds for each k in numSims is recorded. This procedure is later modified to test models that work with proximity or similarity matrices exclusively.</p>
<p><b>Results</b></p>
<p>The following figure presents the average RMSE from<br>
10-fold CV after selecting 5, 10, 15, 20, 25, and 30 nearest neighbours<br>
from the user-user and item-item proximity and similarity matrices. Once<br>
again, the similarity matrices represent the memory-based information,<br>
while the proximity matrices carry the content-based information. The<br>
four different sources of features are termed “feature categories” in<br>
the graph.</p>
<figure data-orig-height="428" data-orig-width="450"><img alt="image" src="https://i0.wp.com/68.media.tumblr.com/f207d1eb8428be5488b8d2fc3d5d44e5/tumblr_inline_oof37krFpn1qa0hyw_540.png?resize=450%2C428" data-orig-height="428" data-recalc-dims="1" data-orig-width="450"></figure>
<p>Figure 1. The average RMSE from the 10-fold cross-validations from clm() <br>with 5, 10, 15, 20, 25, and 30 nearest neighbours.</p>
<p>The model with 20 features only<br>
 and exactly 24 parameters (4 intercepts were estimated plus the<br>
coefficients for 5 ratings from four matrices) achieves an average RMSE<br>
of around .52. For comparison, in <a target="_blank" rel="nofollow" href="https://www.smartcat.io/blog/2017/improved-r-implementation-of-collaborative-filtering/">a recent 10-fold cross-validation over MovieLens 100K with {recommenderlab}</a><br>
the CF algorithms achieved an RMSE of .991 with 100 nearest neighbours<br>
in user-based CF, and an RMSE of .940 with 100 nearest neighbours in<br>
item-based CF: our approach almost double downs the error with a<br>
fivefold reduction in the number of features used. With 120 features (30<br>
nearest neighbours from all four matrices) and 124 parameters, the<br>
models achieves a RMSE of approximately .43. With these results, the new<br>
approach surpasses many recommender engines that were tested on<br>
MovieLens 100K in the previous years; for example, compare the results<br>
presented here – taking into consideration the fact that the model with<br>
20 features only achieves an RMSE of .517 on the whole dataset (ie.<br>
without cross-validation) – with the <a target="_blank" rel="nofollow" href="http://www.cs.carleton.edu/cs_comps/0607/recommend/recommender/results.html">results</a><br>
from the application of several different approaches presented by the<br>
Recommender Systems project of the Carleton College, Northfield, MN.</p>
<p>What about the relative importance of memory and<br>
content-based information in the prediction of user ratings? The<br>
following figure presents the median values and range of exp(β) for<br>
progressively more complex ordinal models that were tested here:</p>
<figure data-orig-height="428" data-orig-width="450"><img alt="image" src="https://i1.wp.com/68.media.tumblr.com/ac95b988a6dc2734daad0a004bef35b6/tumblr_inline_oof38cxUfq1qa0hyw_540.png?resize=450%2C428" data-orig-height="428" data-recalc-dims="1" data-orig-width="450"></figure>
<p>Figure 2. The median exp(β) from the clm() ordered logit  models estimated <br>without cross-validation and encompassing 5, 10, 15, 20, 25, and 30 nearest neighbours from the memory-based and content-based matrices.</p>
<p>As it is readily observed, the memory-based features (simItemsCoeff for item-item similarity neighbours, and simUsersCoeff<br>
for user-user similarity neighbours in the graph legend) weigh more<br>
heavily in the ratings prediction than the content-based features (proxItemsCoeff for the item-item proximity neighbours, and proxUsersCoeff<br>
for the user-user proximity neighbours). I have performed another round<br>
of the experiment by fitting ordinal models with the information from<br>
the memory-based, similarity matrices only; Figure 3 presents the<br>
average RMSE obtained from 10-fold CVs of these models:</p>
<figure data-orig-height="390" data-orig-width="450"><img alt="image" src="https://i2.wp.com/68.media.tumblr.com/db6a5de3970f241792dec314cdfdf075/tumblr_inline_oof393n8SW1qa0hyw_540.png?resize=450%2C390" data-orig-height="390" data-recalc-dims="1" data-orig-width="450"></figure>
<p>Figure 3. The average RMSE from the 10-fold cross-validations from clm() <br>with 5, 10, 15, 20, 25, and 30 nearest neighbours for models with memory-based <br>information only.</p>
<p>The results seem to indicate almost no change in the<br>
RMSEs across the models (but note how the model starts overfitting when<br>
more than 25 features per matrix are used). Could it be that the<br>
content-based information – encoded by the proximity matrices – is not a<br>
necessary ingredient in my predictive model at all? No. But we will<br>
certainly not use RMSE to support this conclusion:</p>
<pre>modelFiles &lt;- list.files()<br>modelRes &lt;- matrix(rep(0,6*7), nrow = 6, ncol = 7)<br>numSims &lt;- seq(5, 30, by = 5)<br>ct = 0<br>for (k in numSims) {<br>  ct = ct + 1<br>  fileNameSimProx &lt;- modelFiles[which(grepl(paste0(&quot;CV_&quot;,k,&quot;F&quot;), modelFiles, fixed = T))]<br>  modelSimProx &lt;- readRDS(fileNameSimProx)<br>  fileNameSim &lt;- modelFiles[which(grepl(paste0(&quot;Sim_&quot;,k,&quot;F&quot;), modelFiles, fixed = T))]<br>  modelSim &lt;- readRDS(fileNameSim)<br>  testRes &lt;- anova(modelSimProx, modelSim)<br>  modelRes[ct, ] &lt;- c(testRes$no.par, testRes$AIC, testRes$LR.stat[2],<br>                      testRes$df[2], testRes$`Pr(>Chisq)`[2])<br>}<br>modelRes &lt;- as.data.frame(modelRes)<br>colnames(modelRes) &lt;- c('Parameters_Model1', 'Parameters_Model2',<br>                        'AIC_Model1', 'AIC_Model2',<br>                        'LRTest','DF','Pr_TypeIErr')<br>write.csv(modelRes, 'ModelSelection.csv')
</pre>
<p> The results of the likelihood ratio tests,<br>
together with the values of the Akaike Information Criterion (AIC) for<br>
all models are provided in Table 1; a comparison between the AIC values<br>
are provided in Figure 4. I have compared the models that encompass both<br>
the top neighbours from the memory and the content-based matrices, on<br>
one, with the models encompassing only information from the memory-based<br>
matrices, on the other hand. As you can see, the AIC is consistently<br>
lower for the models that incorporate both the content and the<br>
memory-based information (with the significant LRTs testifying that the<br>
contribution of content-based information cannot be rejected).</p>
<figure data-orig-height="196" data-orig-width="450"><img alt="image" src="https://i0.wp.com/68.media.tumblr.com/2bf9d16d9b9f4cf3c4c1c6805125ef7f/tumblr_inline_oof39ymZl91qa0hyw_540.png?resize=450%2C196" data-orig-height="196" data-recalc-dims="1" data-orig-width="450"></figure>
<figure data-orig-height="339" data-orig-width="450"><img alt="image" src="https://i0.wp.com/68.media.tumblr.com/a3fbbfdf618bb19ab4eb5e2dee515c16/tumblr_inline_oof3afwKZG1qa0hyw_540.png?resize=450%2C339" data-orig-height="339" data-recalc-dims="1" data-orig-width="450"></figure>
<p>Figure 4. Model selection: AIC for ordered logit models<br>with 5, 10, 15, 20, 25, and 30 nearest neighbours: memory-based <br>information only (Sim) vs. memory-based and content-based models (Sim+Prox).</p>
<p><b>Summary</b></p>
<p>Some say that comparing model-based recommendation<br>
systems with the CF approaches is not fair in the first place. True.<br>
However, achieving a reduction in error of almost 50% by using 80% less<br>
features is telling. I think the result has two important stories to<br>
tell, in fact. The first one is that the comparison is really not fair<br>
:-). The second story is, however, much more important. It is not about<br>
the application of the relevant model; since the development of discrete<br>
choice models in economics, many behavioral scientist who see a Likert<br>
type response scale probably tends to think of some appropriate ordinal<br>
model for the problem at hand. It is about the <i>formulation of the problem</i>: as ever in science, the crucial question turns out to be the one on the choice of the most relevant description of the problem; in cognitive and data science, <i>the question of what representation</i> enables for its solution by as simple as possible means of mathematical modeling.</p>
<p>References</p>
<p>[1] Greene, W. &amp; Hensher, D. (2010). Modeling Ordered Choices. Cambridge University Press, 2010.</p>
<p>[2] Christensen, R. H. B. (June 28, 2015). Analysis<br>
of ordinal data with cumulative link models – estimation with the<br>
R-package ordinal. Accessed on March 03, 2017, from the The Comprehensive R Archive Network. URL: <a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/packages/ordinal/vignettes/clm_intro.pdf">https://cran.r-project.org/web/packages/ordinal/vignettes/clm_intro.pdf</a></p>
<p>Goran S. Milovanović, PhD<br><a target="_blank" rel="nofollow" href="http://t.umblr.com/redirect?z=http%3A%2F%2FSmartCat&t=ZDE1YzY0NTYzZDVjZjRmYjRhOTlkYWMxMzAxZGYxN2U1NmFmODE3ZixRcWhrR0dpQw%3D%3D&b=t%3AfxfLHZZpYe9t9Oek1n3iNg&p=http%3A%2F%2Fwww.exactness.net%2Fpost%2F157582845120%2Fasknasa-whats-the-optimal-time-for-aliens-to&m=1">Data Science Consultant, SmartCat</a></p>
<figure data-orig-height="180" data-orig-width="180"><img alt="image" src="https://i2.wp.com/68.media.tumblr.com/a5d2491db08ab66670efe27e60c1ee9c/tumblr_inline_oof3ctXBjg1qa0hyw_540.png?resize=180%2C180" data-orig-height="180" data-recalc-dims="1" data-orig-width="180"></figure>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.exactness.net/post/159577222155"> The Exactness of Mind</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/WvRTiDB93lo"></a></div>