<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/introduction-of-the-pdfsearch-package/"></div></div><div><a data-url="https://www.r-bloggers.com/introduction-of-the-pdfsearch-package/" data-counturl="https://www.r-bloggers.com/introduction-of-the-pdfsearch-package/" data-count="horizontal" data-text="Introduction of the pdfsearch package" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://educate-r.org//2016/12/02/intro_pdfsearch.html"> Educate-R - R</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>I’m happy to introduce an add-on package, <code>pdfsearch</code>, that adds the ability to do keyword searches on pdf files. This add-on package uses the excellent <code>pdftools</code> package from the <a target="_blank" rel="nofollow" href="https://ropensci.org/">ropensci</a> project to read in pdf files and perform keyword searches based character strings of interest.</p>
<h2>Installation</h2>
<p>The package is currently only hosted on <a target="_blank" rel="nofollow" href="https://github.com/lebebr01/pdfsearch">github</a> and can be installed with the devtools library.</p>
<pre><code>devtools::install_github('lebebr01/pdfsearch')
</code></pre>
<h2>Basic Example</h2>
<p>Doing a simple keyword search on a single pdf file uses the <code>keyword_search</code> function. The following is a simple example using a pdf from <a target="_blank" rel="nofollow" href="https://arxiv.org/">arXiv</a>.</p>
<pre><code>library(pdfsearch)

file &lt;- system.file('pdf', '1501.00450.pdf', package = 'pdfsearch')

key_res &lt;- keyword_search(file, 
                          keyword = c('repeated measures', 'mixed effects'),
                          path = TRUE)
</code></pre>
<p>In the following example, the function <code>keyword_search</code> takes two required arguments, the path to the pdf file and the keyword(s) to search for in the pdf. The optional argument shown above, <code>path</code> tells the function to read in the raw pdf using the <code>pdftools</code> package.</p>
<pre><code>data.frame(key_res)
</code></pre>
<pre><code>##              keyword page_num line_num
## 1  repeated measures        1       24
## 2  repeated measures        2       57
## 3  repeated measures        2      108
## 4  repeated measures        2      110
## 5  repeated measures        2      125
## 6  repeated measures        6      444
## 7  repeated measures        6      445
## 8  repeated measures        6      474
## 9  repeated measures        6      485
## 10 repeated measures        9      708
##                                                                                                                                 line_text
## 1  cally the repeated measures design, including the crossover           get false confidence about lack of negative effects. Statistical
## 2             fast iterations and testing many ideas can reap the most         erations to repeated measures design, with variants to the
## 3             repeated measures design in different stages of treatment        in this section we assume all users appear in all periods,
## 4               ing the repeated measures analysis, reporting a &lt;U+0093>per week&lt;U+0094>       to metrics that are defined as simple average and assume
## 5               In fact, the crossover design is a type of repeated measures     designs considered can be examined in the same framework
## 6          values and the absence in a specific time window can still          It is common to analyze data from repeated measures design
## 7              provide information on the user behavior and in reality there       with the repeated measures ANOVA model and the F-test,
## 8                                      \022             \023                      As an example, one possible model for repeated measures
## 9          a similar example; also see (Van der Vaart 2000) for a text         possible. In repeated measures data, users might appear in
## 10                 framework of repeated measures design. Experimenters should          &lt;U+0095> Wash-out and decide: If we have little informa-
</code></pre>
<pre><code>head(key_res$line_text, n = 2)
</code></pre>
<pre><code>## [[1]]
## [1] &quot;cally the repeated measures design, including the crossover           get false confidence about lack of negative effects. Statistical&quot;
## 
## [[2]]
## [1] &quot;fast iterations and testing many ideas can reap the most         erations to repeated measures design, with variants to the&quot;
</code></pre>
<p>The output includes the keyword, the page number it is located, the line number the keyword was found, and the line of text. By default, only the line matching the keyword is returned. If the context of the result is desired, there is an optional argument <code>surround_lines</code> that can include the lines around the line of the matching keyword.</p>
<pre><code>key_res &lt;- keyword_search(file, 
                          keyword = c('repeated measures', 'mixed effects'),
                          path = TRUE, 
                          surround_lines = 2)
head(key_res$line_text, n = 2)
</code></pre>
<pre><code>## [[1]]
## [1] &quot;to be evaluated, and the speed new feature iterations. We             Running under powered experiments have many perils. Not&quot;         
## [2] &quot;introduce more sophisticated experimental designs, specifi-           only would we miss potentially beneficial effects, we may also&quot;  
## [3] &quot;cally the repeated measures design, including the crossover           get false confidence about lack of negative effects. Statistical&quot;
## [4] &quot;design and related variants, to increase KPI sensitivity with         power increases with larger effect size, and smaller variances.&quot; 
## [5] &quot;the same traffic size and duration of experiment. In this pa-         Let us look at these aspects in turn.&quot;                           
## 
## [[2]]
## [1] &quot;benefits (see Kohavi et al. 2012, Section 3.4). This poses&quot;                                                                     
## [2] &quot;a limitation to any online experimentation platform, where       within-subject variation. We also discuss practical consid-&quot;   
## [3] &quot;fast iterations and testing many ideas can reap the most         erations to repeated measures design, with variants to the&quot;    
## [4] &quot;rewards.                                                         crossover design to study the carry over effect, including the&quot;
## [5] &quot;&lt;U+0093>re-randomized&lt;U+0094> design (row 5 in table 1).&quot;
</code></pre>
<h2>Directory Search</h2>
<p>This package also has the ability to loop over a directory of pdf files in a single run. To do this, the <code>keyword_directory</code> function is of interest. Much of the arguments are the same, except a directory is specified instead of a single path to the location of the pdf files.</p>
<pre><code># find directory
directory &lt;- system.file('pdf', package = 'pdfsearch')

# do search over two files
head(keyword_directory(directory, 
       keyword = c('repeated measures', 'measurement error'),
       surround_lines = 1, full_names = TRUE), n = 12)
</code></pre>
<pre><code>##    ID       pdf_name           keyword page_num line_num
## 1   1 1501.00450.pdf repeated measures        1       24
## 2   1 1501.00450.pdf repeated measures        2       57
## 3   1 1501.00450.pdf repeated measures        2      108
## 4   1 1501.00450.pdf repeated measures        2      110
## 5   1 1501.00450.pdf repeated measures        2      125
## 6   1 1501.00450.pdf repeated measures        6      444
## 7   1 1501.00450.pdf repeated measures        6      445
## 8   1 1501.00450.pdf repeated measures        6      474
## 9   1 1501.00450.pdf repeated measures        6      485
## 10  1 1501.00450.pdf repeated measures        9      708
## 11  2 1610.00147.pdf measurement error        1        5
## 12  2 1610.00147.pdf measurement error        1       19
##                                                                                                                                                                                                                                                                                                                                                                                                              line_text
## 1  introduce more sophisticated experimental designs, specifi-           only would we miss potentially beneficial effects, we may also, cally the repeated measures design, including the crossover           get false confidence about lack of negative effects. Statistical, design and related variants, to increase KPI sensitivity with         power increases with larger effect size, and smaller variances.
## 2                           a limitation to any online experimentation platform, where       within-subject variation. We also discuss practical consid-, fast iterations and testing many ideas can reap the most         erations to repeated measures design, with variants to the, rewards.                                                         crossover design to study the carry over effect, including the
## 3                                 In this paper we extend the idea further by employing the        weeks. To facilitate our illustration, in all the derivation, repeated measures design in different stages of treatment        in this section we assume all users appear in all periods,, assignment. The traditional A/B test can be analyzed us-         i.e. no missing measurement. We also restrict ourselves
## 4                                       assignment. The traditional A/B test can be analyzed us-         i.e. no missing measurement. We also restrict ourselves, ing the repeated measures analysis, reporting a &lt;U+0093>per week&lt;U+0094>       to metrics that are defined as simple average and assume, treatment effect, as show in row 3 &lt;U+0093>parallel&lt;U+0094> design in ta-      treatment and control have the same sample size. We fur-
## 5                                                                     each user serves as his/her own control in the measurement.      fixed effects in the model in this section. This way, various, In fact, the crossover design is a type of repeated measures     designs considered can be examined in the same framework, design commonly used in biomedical research to control for       and easily compared.
## 6                                                       to realize infrequent users are more likely to have missing         5.1 Review of Existing Methods, values and the absence in a specific time window can still          It is common to analyze data from repeated measures design, provide information on the user behavior and in reality there       with the repeated measures ANOVA model and the F-test,
## 7                        values and the absence in a specific time window can still          It is common to analyze data from repeated measures design, provide information on the user behavior and in reality there       with the repeated measures ANOVA model and the F-test,, might be other factors causing user to be missing that are          under certain assumptions, such as normality, sphericity (ho-
## 8                                                                                                                                                                                                                    0 0, \022             \023                      As an example, one possible model for repeated measures, Xi Xi0                             using lme4&lt;U+0092>s formula syntax (Bates et al. 2012a;b) is
## 9                      the delta-method; see Deng et al. (2013, Appendix B) for            Random effect makes modeling within-subject variability, a similar example; also see (Van der Vaart 2000) for a text         possible. In repeated measures data, users might appear in, book treatment of the delta-method.                                 multiple periods, represented as multiple rows in the dataset.
## 10                                         At the design stage, we face a few choices under the same               measure it directly and should be used here., framework of repeated measures design. Experimenters should          &lt;U+0095> Wash-out and decide: If we have little informa-, use domain knowledge and past experiments to inform the                 tion to judge carry over effect, we can run the first
## 11                                                                                                                                                                                                                                         Abstract, Often in surveys, key items are subject to measurement errors. Given just the, data, it can be difficult to determine the distribution of this error process, and
## 12                                                                                                                                                                                               National Survey of College Graduates. We also present a process for assessing, the sensitivity of various analyses to different choices for the measurement error, models. Supplemental material is available online.
</code></pre>
<p>Two relavent arguments for the <code>keyword_directory</code> function are <code>full_names</code> and <code>recursive</code>. These functions ask whether the full path for the pdf files in the directory will be used and whether subfolders within the directory will also be searched.</p>
<h2>Uses for pdfsearch</h2>
<p>This package may be extremely useful when conducting research syntheses or meta analyses, particularly when screening articles for inclusion into the research synthesis or meta analysis. This aim is hopeful to be explored later in more depth.</p>
<h3>Limitations</h3>
<p>The limitations of the package and the quality of text matches will depend on the pdfs being searched. For example, words that wrap across lines (i.e. hyphenated words) will not be included in the matches as entire words are currently being searched to be matched.</p>
<h2>Moving Forward</h2>
<p>The package will be submitted to CRAN next week, however, any bugs or problems can be submitted to the github site <a target="_blank" rel="nofollow" href="https://github.com/lebebr01/pdfsearch/issues">https://github.com/lebebr01/pdfsearch/issues</a>.</p>
<p><div><div><div><div data-width="55" data-show-faces="false" data-send="true" data-layout="box_count" data-height="62" data-href="https://www.r-bloggers.com/introduction-of-the-pdfsearch-package/"></div></div><div><a data-url="https://www.r-bloggers.com/introduction-of-the-pdfsearch-package/" data-counturl="https://www.r-bloggers.com/introduction-of-the-pdfsearch-package/" data-count="vertical" data-text="Introduction of the pdfsearch package" class="twitter-share-button" data-via="rbloggers" href="https://twitter.com/share"></a></div><div></div></div><div></div></div>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://educate-r.org//2016/12/02/intro_pdfsearch.html"> Educate-R - R</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/?s=data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/?s=Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/?s=map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/?s=animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/?s=sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/?s=LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/?s=SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/?s=eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/?s=git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/?s=hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/?s=Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/?s=regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/?s=PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/?s=time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/?s=trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/a3YEQASBi-k"></a></div>