<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/retrieving-reading-levels-with-r/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.moreorlessnumbers.com/2017/04/retrieving-reading-levels-with-r.html"> More or Less Numbers</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<div dir="ltr">For those that don’t work in education or aren’t aware, there is a measurement for a child’s reading level called a <a target="_blank" rel="nofollow" href="https://lexile.com/">Lexile ® Level</a>.  There are ways in which this level can be retrieved using different reading assessments.  The measurement can be used to match a child’s reading level to books that are the same level.  So the company maintains a database of books that they have assigned levels.  There are other assessments that measure how well a child is reading, but I’m not aware of any systems that assign books as comprehensively.  The library at the school I work at lacked <a target="_blank" rel="nofollow" href="https://lexile.com/">Lexile Levels</a> for their books.  This is unfortunate because of the fact that teachers are not able provide students with books for their respective Lexile Level.  Fortunately though the library had a list of ISBNs for the books.
<p>On the <a target="_blank" rel="nofollow" href="https://lexile.com/">Lexile website</a> there is a way to search for books using ISBN numbers to retrieve their Lexile ® Level if the book is available in their database.  Entering every ISBN number available is a task fit for something not human. </p>
<p><a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/packages/rvest/rvest.pdf">rvest</a> to the rescue. </p>
<p>Below is the script to retrieve the Lexile Levels of books if a list of ISBNs is available.  This was an incredible time save provided by some R code and hopefully someone else out there could use it.<br>  <br>
<table data-tab-size="8">
<tbody>
<tr>
<td> </td>
</tr>
<tr>
<td data-line-number="5"></td>
<td>library(<span>rvest</span>) library(httr)</td>
</tr>
<tr>
<td data-line-number="6"></td>
<td>library(<span>htmltools</span>)</td>
</tr>
<tr>
<td data-line-number="7"></td>
<td>library(<span>dplyr</span>)</td>
</tr>
<tr>
<td data-line-number="8"></td>
<td><span><span>#</span>#Prep for things used later</span></td>
</tr>
<tr>
<td data-line-number="9"></td>
<td><span>url</span><span>&lt;-</span><span><span>“</span>https://www.lexile.com/fab/results/?keyword=<span>“</span></span></td>
</tr>
<tr>
<td data-line-number="10"></td>
<td><span>url2</span><span>&lt;-</span><span><span>“</span>https://lexile.com/book/details/<span>“</span></span></td>
</tr>
<tr>
<td data-line-number="11"></td>
<td><span><span>#</span>#CSV file with ISBN numbers</span></td>
</tr>
<tr>
<td data-line-number="12"></td>
<td><span>dat1</span><span>&lt;-</span>read.csv(<span><span>“</span>~isbns.csv<span>“</span></span>,<span>header</span><span>=</span><span>FALSE</span>)</td>
</tr>
<tr>
<td data-line-number="13"></td>
<td><span><span>#</span>#dat1&lt;-data.frame(dat1[203:634,])</span></td>
</tr>
<tr>
<td data-line-number="14"></td>
<td><span>dat</span><span>&lt;-</span>as.character(<span>dat1</span>[,<span>1</span>])%<span>></span>%trimws()</td>
</tr>
<tr>
<td data-line-number="15"></td>
<td><span><span>#</span>#dat&lt;-dat[41:51]</span></td>
</tr>
<tr>
<td data-line-number="16"></td>
<td><span>blank</span><span>&lt;-</span>as.character(<span><span>“</span>NA<span>“</span></span>)</td>
</tr>
<tr>
<td data-line-number="17"></td>
<td><span>blank1</span><span>&lt;-</span>as.character(<span><span>“</span>NA<span>“</span></span>)</td>
</tr>
<tr>
<td data-line-number="18"></td>
<td><span><span>#</span>#blank2&lt;-as.character(“NA”)</span></td>
</tr>
<tr>
<td data-line-number="19"></td>
<td><span><span>#</span>#blank3&lt;-as.character(“NA”)</span></td>
</tr>
<tr>
<td data-line-number="20"></td>
<td><span>all</span><span>&lt;-</span><span>data.frame</span>(<span><span>“</span>A<span>“</span></span>,<span><span>“</span>B<span>“</span></span>,<span><span>“</span>C<span>“</span></span>)</td>
</tr>
<tr>
<td data-line-number="21"></td>
<td>colnames(<span>all</span>)<span>&lt;-</span>c(<span><span>“</span>name<span>“</span></span>,<span><span>“</span>lexiledat<span>“</span></span>,<span><span>“</span>num<span>“</span></span>)</td>
</tr>
<tr>
<td data-line-number="22"></td>
<td><span>all</span><span>&lt;-</span><span>data.frame</span>(<span>all</span>[<span>–</span><span>1</span>,])</td>
</tr>
<tr>
<td data-line-number="23"></td>
<td></td>
</tr>
<tr>
<td data-line-number="24"></td>
<td><span>for</span>(<span>i</span> <span>in</span> <span>dat</span>) {</td>
</tr>
<tr>
<td data-line-number="25"></td>
<td> </td>
</tr>
<tr>
<td data-line-number="26"></td>
<td><span>sites</span><span>&lt;-</span>paste(<span>url</span>,<span>i</span>,<span>sep</span><span>=</span><span><span>“</span><span>“</span></span>)</td>
</tr>
<tr>
<td data-line-number="27"></td>
<td></td>
</tr>
<tr>
<td data-line-number="28"></td>
<td><span>x</span> <span>&lt;-</span> GET(<span>sites</span>, add_headers(<span><span>‘</span>user-agent<span>‘</span></span> <span>=</span> <span><span>‘</span>r<span>‘</span></span>))</td>
</tr>
<tr>
<td data-line-number="29"></td>
<td></td>
</tr>
<tr>
<td data-line-number="30"></td>
<td><span>webpath</span><span>&lt;-</span><span>x</span><span>$</span><span>url</span>%<span>></span>%<span>includeHTML</span>%<span>></span>%read_html()</td>
</tr>
<tr>
<td data-line-number="31"></td>
<td></td>
</tr>
<tr>
<td data-line-number="32"></td>
<td><span><span>#</span>#Book Name</span></td>
</tr>
<tr>
<td data-line-number="33"></td>
<td><span>name</span><span>&lt;-</span><span>webpath</span>%<span>></span>%html_nodes(<span>xpath</span><span>=</span><span><span>“</span>///div[2]/div/div[2]/h4/a<span>“</span></span>)%<span>></span>%html_text()%<span>></span>%trimws()</td>
</tr>
<tr>
<td data-line-number="34"></td>
<td></td>
</tr>
<tr>
<td data-line-number="35"></td>
<td><span><span>#</span>#Lexile Range</span></td>
</tr>
<tr>
<td data-line-number="36"></td>
<td><span>lexile</span><span>&lt;-</span><span>webpath</span>%<span>></span>%html_nodes(<span>xpath</span><span>=</span><span><span>“</span>///div[2]/div/div[3]/div[1]<span>“</span></span>)%<span>></span>%html_text()%<span>></span>%trimws()%<span>></span>%as.character()</td>
</tr>
<tr>
<td data-line-number="37"></td>
<td></td>
</tr>
<tr>
<td data-line-number="38"></td>
<td><span><span>#</span>#CSS change sometimes </span></td>
</tr>
<tr>
<td data-line-number="39"></td>
<td><span>lexiledat</span><span>&lt;-</span>ifelse(is.na(<span>lexile</span>[<span>2</span>])<span>==</span><span>TRUE</span>,<span>lexile</span>,<span>lexile</span>[<span>2</span>])</td>
</tr>
<tr>
<td data-line-number="40"></td>
<td></td>
</tr>
<tr>
<td data-line-number="41"></td>
<td><span>test1</span><span>&lt;-</span><span>data.frame</span>(<span>lexiledat</span>,<span>NA</span>)</td>
</tr>
<tr>
<td data-line-number="42"></td>
<td></td>
</tr>
<tr>
<td data-line-number="43"></td>
<td><span><span>#</span>#Breaks every now and then when adding Author/Pages</span></td>
</tr>
<tr>
<td data-line-number="44"></td>
<td><span><span>#</span>#Author Name</span></td>
</tr>
<tr>
<td data-line-number="45"></td>
<td><span><span>#</span>#author&lt;-webpath%>%html_nodes(xpath=’///div[2]/div/div[2]/span’)%>%html_text()%>%as.character()%>%trimws()</span></td>
</tr>
<tr>
<td data-line-number="46"></td>
<td></td>
</tr>
<tr>
<td data-line-number="47"></td>
<td><span><span>#</span>#author&lt;-sub(“by: “,””,author)</span></td>
</tr>
<tr>
<td data-line-number="48"></td>
<td></td>
</tr>
<tr>
<td data-line-number="49"></td>
<td><span><span>#</span>#Pages</span></td>
</tr>
<tr>
<td data-line-number="50"></td>
<td><span><span>#</span>#pages&lt;-webpath%>%html_nodes(xpath=’///div[2]/div/div[2]/div/div[1]’)%>%html_text()%>%as.character()%>%trimws()</span></td>
</tr>
<tr>
<td data-line-number="51"></td>
<td></td>
</tr>
<tr>
<td data-line-number="52"></td>
<td><span><span>#</span>#pages&lt;-sub(“Pages: “,””,pages)</span></td>
</tr>
<tr>
<td data-line-number="53"></td>
<td></td>
</tr>
<tr>
<td data-line-number="54"></td>
<td></td>
</tr>
<tr>
<td data-line-number="55"></td>
<td><span><span>#</span>#Some books not found, this excludes them and replaces with NA values</span></td>
</tr>
<tr>
<td data-line-number="56"></td>
<td><span>df</span><span>&lt;-</span><span>if</span>(is.na(<span>test1</span>)) <span>data.frame</span>(<span>blank</span>,<span>blank1</span>) <span>else</span> <span>data.frame</span>(<span>name</span>,<span>lexiledat</span>,<span>stringsAsFactors</span> <span>=</span> <span>FALSE</span>)</td>
</tr>
<tr>
<td data-line-number="57"></td>
<td></td>
</tr>
<tr>
<td data-line-number="58"></td>
<td>colnames(<span>df</span>)<span>&lt;-</span>c(<span><span>“</span>name<span>“</span></span>,<span><span>“</span>lexiledat<span>“</span></span>)</td>
</tr>
<tr>
<td data-line-number="59"></td>
<td><span>df</span><span>$</span><span>num</span> <span>&lt;-</span> <span>i</span></td>
</tr>
<tr>
<td data-line-number="60"></td>
<td></td>
</tr>
<tr>
<td data-line-number="61"></td>
<td><span>all</span><span>&lt;-</span>bind_rows(<span>all</span>,<span>df</span>)</td>
</tr>
<tr>
<td data-line-number="62"></td>
<td></td>
</tr>
<tr>
<td data-line-number="63"></td>
<td>}</td>
</tr>
<tr>
<td data-line-number="64"></td>
<td></td>
</tr>
<tr>
<td data-line-number="65"></td>
<td><span>master</span><span>&lt;-</span>rbind(<span>all1</span>,<span>all</span>)</td>
</tr>
</tbody>
</table>
<p>   <br><a target="_blank" rel="nofollow" href="https://github.com/justlebeau/Blog-Code/blob/master/LexileLevelScrape.R">Link to code</a></p>
<p></div>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.moreorlessnumbers.com/2017/04/retrieving-reading-levels-with-r.html"> More or Less Numbers</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/GvS3Ok3AlZI"></a></div>