<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/web-scraping-and-invalid-multibyte-string/"></div></div><div><a data-url="https://www.r-bloggers.com/web-scraping-and-invalid-multibyte-string/" data-counturl="https://www.r-bloggers.com/web-scraping-and-invalid-multibyte-string/" data-count="horizontal" data-text="Web Scraping and “invalid multibyte string”" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.exegetic.biz/blog/2016/08/web-scraping-invalid-multibyte-string/"> R – Exegetic Analytics</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>A couple of my collaborators have had trouble using <code>read_html()</code> from the <a target="_blank" href="https://github.com/hadley/readr">readr package</a> to access <a target="_blank" href="https://en.wikipedia.org/wiki/List_of_countries_by_population_(United_Nations)">this Wikipedia page</a>. Specifically they have been getting errors like this:</p>
<pre>
Error in utils::type.convert(out[, i], as.is = TRUE, dec = dec) :
  invalid multibyte string at '&lt;e2>&lt;94>'
</pre>
<p>Since I couldn’t reproduce these errors on my machine it appeared to be something relating to their particular machine setup. Looking at their locale provided a clue:</p>
<pre>
> Sys.getlocale()
[1] &quot;LC_COLLATE=Korean_Korea.949;LC_CTYPE=Korean_Korea.949;LC_MONETARY=Korean_Korea.949;
LC_NUMERIC=C;LC_TIME=Korean_Korea.949&quot;
</pre>
<p>whereas on my machine I have:</p>
<pre>
> Sys.getlocale()
[1] &quot;LC_CTYPE=en_ZA.UTF-8;LC_NUMERIC=C;LC_TIME=en_ZA.UTF-8;LC_COLLATE=en_ZA.UTF-8;
LC_MONETARY=en_ZA.UTF-8;LC_MESSAGES=en_ZA.UTF-8;LC_PAPER=en_ZA.UTF-8;LC_NAME=C;LC_ADDRESS=C;
LC_TELEPHONE=C;LC_MEASUREMENT=en_ZA.UTF-8;LC_IDENTIFICATION=C&quot;
</pre>
<p>The document that they were trying to scrape is encoded in UTF-8, which I see in my locale but not in theirs. Perhaps changing locale will sort out the problem? Since the <code>en_ZA</code> locale is a bit of an acquired taste (unless you’re South African, in which case it’s <em>de rigueur</em>!), the following should resolve the problem:</p>
<pre>
> Sys.setlocale(&quot;LC_CTYPE&quot;, &quot;en_US.UTF-8&quot;)
</pre>
<p>This might precipitate an error stating that the directive cannot be honoured by your system. Do not fear, all is not lost. Try the following (which seems to work almost universally!):</p>
<pre>
Sys.setlocale(&quot;LC_ALL&quot;, &quot;English&quot;)
</pre>
<p>Try scraping again. Your issues should be resolved.</p>
<p>The post <a target="_blank" rel="nofollow" href="http://www.exegetic.biz/blog/2016/08/web-scraping-invalid-multibyte-string/">Web Scraping and “invalid multibyte string”</a> appeared first on <a target="_blank" rel="nofollow" href="http://www.exegetic.biz/blog">Exegetic Analytics</a>.</p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/web-scraping-and-invalid-multibyte-string/"></div></div><div><a data-url="https://www.r-bloggers.com/web-scraping-and-invalid-multibyte-string/" data-counturl="https://www.r-bloggers.com/web-scraping-and-invalid-multibyte-string/" data-count="vertical" data-text="Web Scraping and “invalid multibyte string”" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.exegetic.biz/blog/2016/08/web-scraping-invalid-multibyte-string/"> R – Exegetic Analytics</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/NLqMVw1cupE"></a></div>