<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/building-and-maintaining-exams-with-dynamic-content/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://msperlin.github.io/2017-01-30-Exams-with-dynamic-content/"> R and Finance</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p> An introduction to package exams – </p>
<p>Part of my job as a researcher and teacher is to periodically apply and<br>
grade exams in my classroom. Being constantly in the shoes of an<br>
examiner, you soon quickly realize that students are clever in finding<br>
ways to do well in an exam without effort. These days, photos and pdf<br>
versions of past exams and exercises are shared online in facebook,<br>
whatsapp groups, instagram and what not. As weird as it may sound, the<br>
distribution of information in the digital era creates a problem for<br>
examiners. If you use the same exam from past year, it is likely that<br>
students will simply memorize the answers from a digital record.<br>
Moreover, some students will also cheat by looking for answers during<br>
the test. Either way, keeping the same exam over time and across<br>
students, is not advisable.</p>
<p>This issue really bothered me. For large classes, there isn’t a way to<br>
evaluate the work of students as cost effective as online or printed<br>
exams. I’m strongly in favor of meritocracy in academia and I think that<br>
a grade in an exam should, on average, be good indicator of the<br>
knowledge that the students retained during coursework. Otherwise,<br>
what’s the point of doing all of it?</p>
<p>In the past, I manually created different versions of questions and<br>
wrote new ones in order to avoid cheating and memorization of questions.<br>
But, year after year, it became clear to me that this was a time<br>
consuming task that took more energy than what I would like to invest.<br>
Besides teaching, I also do research and work on administrative issues<br>
within my department. Sometimes, specially around deadlines, you simply<br>
don’t have the time and mental energy to come up with different versions<br>
of an existing exam.</p>
<p>Back in 2016 I decided to invest some to time to automatize this process<br>
and try to come up with an elegant solution. Since I had all my exams in<br>
a latex template called <code>examdesign</code>, I wrote package<br>
<a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=RndTexExams">RndTexExams</a> that took<br>
as input a .tex file and created <code>n</code> versions of exams by randomly<br>
defining the order of questions, the answer list and textual content<br>
based on a simple markup language. If you know latex, it is basically a<br>
problem of finding regex patterns and restructuring a character object<br>
that is later saved in a new and compilable latex file.</p>
<p>The package I wrote worked pretty well for me but, as with any first<br>
version of a software, it had missing features. The output was only a<br>
pdf file based on a template, it did not work with standard academic<br>
platforms such as Blackboard and Moodle and, the most problematic in my<br>
opinion, it was not designed to run embedded R code that could be parsed<br>
by <code>knitr</code>, like in a Rmarkdown file.</p>
<p>This is when I tried out the package<br>
<a target="_blank" rel="nofollow" href="https://cran.r-project.org/package=exams">exams</a>. While my solution<br>
with <code>RndTexExams</code> was alright for a latex user, package exams is much<br>
better at solving the problem of dynamic content in exams. Using the<br>
<code>knitr</code> and <code>sweave</code> engines, the level of randomization and creation of<br>
dynamic content is really amazing. By combining R code (and all the<br>
capabilities of CRAN packages), you can do do anything your want in an<br>
exam. You can get information on the web, use completely different<br>
datasets for each exam and so on. The limit is set by your imagination.</p>
<h2>An example of exam with dynamic content</h2>
<p>As a quick example, I am going to show one question from the exercise<br>
chapter of my book. When it is ready, I will be serving the exercises<br>
with a web based shiny app, meaning that the reader will download a pdf<br>
file with unique questions that is processed in a shiny server.</p>
<p>In this example questions, I’m asking the reader to use R to solve the<br>
following problem:</p>
<div>
<pre><code>How many packages you can find today (2017-01-30) in CRAN?
Use repository https://cloud.r-project.org/ for the solution.
</code></pre>
</div>
<p>The solution is pretty simple, all you need to do is to ask for the<br>
number of rows for the object output from a call to<br>
<code>available.packages()</code>. The reader can find the solution with the<br>
command<br>
<code>nrow(available.packages(repos='https://cloud.r-project.org/'))</code>.</p>
<p>Now, lets build the content of this simple question in a separate file.<br>
You can either use .Rnw or .Rmd files with exam. I will choose the later<br>
just to keep it simple. Here are the contents of a file called<br>
<strong>Question.Rmd</strong>, available <a target="_blank" rel="nofollow" href="https://github.com/msperlin/msperlin.github.io/blob/master/otherfiles/Question.Rmd">here</a>.</p>
<div>
<pre><code>cat(paste0(readLines('Question.Rmd'), collapse = '\n'))

## ```{r data generation, echo = FALSE, results = &quot;hide&quot;}
## #possible.repo &lt;- getCRANmirrors()$URL  # doenst work well for all repos
## 
## possible.repo &lt;- c('https://cloud.r-project.org/',
##                   'http://mirror.fcaglp.unlp.edu.ar/CRAN/',
##                   'http://cran-r.c3sl.ufpr.br/',
##                   'http://cran.stat.sfu.ca/',
##                   'https://mirrors.dotsrc.org/cran/',
##                   'https://mirrors.cicku.me/CRAN/',
##                   'https://cran.ism.ac.jp/')
## 
## my.repo &lt;- sample(possible.repo,1)
## 
## n.pkgs &lt;- nrow(available.packages(repos = my.repo))
## 
## sol.q &lt;- n.pkgs
## rnd.vec &lt;- c(0, sample(-5000:-1,4))
## 
## my.answers &lt;- paste0(sol.q+rnd.vec, ' packages')
## ```
## 
## Question
## ========
## 
## How many packages you can find today (`r Sys.Date()`) in CRAN? 
## 
## Use repository `r my.repo` for the solution.
## 
## ```{r questionlist, echo = FALSE, results = &quot;asis&quot;}
## exams::answerlist(my.answers, markup = &quot;markdown&quot;)
## ```
## 
## Meta-information
## ================
## extype: schoice
## exsolution: 10000
## exname: numbero of cran pkgs
## exshuffle: TRUE
</code></pre>
</div>
<p>For the last piece of code, notice that I’ve set the solution of the<br>
question in object <code>sol.q</code>. Later, in object <code>my.answers</code>, I use it<br>
together with a random vector of integers to create five alternative<br>
answers to the questions, where the first one is the correct. This<br>
operation results in the following objects:</p>
<div>
<pre><code>my.repo &lt;- 'https://cloud.r-project.org/'
n.pkgs &lt;- nrow(available.packages(repos = my.repo))
  
sol.q &lt;- n.pkgs
rnd.vec &lt;- c(0, sample(-5000:-1,4))
  
my.answers &lt;- paste0(sol.q+rnd.vec, ' packages')
print(my.answers)

## [1] &quot;10001 packages&quot; &quot;8687 packages&quot;  &quot;9883 packages&quot;  &quot;7157 packages&quot; 
## [5] &quot;8513 packages&quot;
</code></pre>
</div>
<p>To conclude the question, I simply use <code>Sys.Date()</code> to get the system’s<br>
date and later set the correct answers using function <code>answerlist</code>. Some<br>
metadata is also inserted at the last section of <code>Question.Rmd</code>. The<br>
line <code>exshuffle: TRUE</code> sets a random order of possible answers in each<br>
exam for this questions. Do notice that the solution is registered in<br>
line <code>exsolution: 10000</code>, where the 1 in 10000 means correct answer in<br>
the first element of <code>my.answers</code> and the 0s represent incorrect<br>
answers.</p>
<p>Now that the file with content of the question is finished, let’s set<br>
some options and build the exam with <code>exams</code>. For simplicity, we will<br>
repeate the same question five times.</p>
<div>
<pre><code>library(exams)

my.f &lt;-'Question.Rmd'
n.ver &lt;- 1
name.exam &lt;- 'exam_sample'
my.dir &lt;- 'exam-out/'

my.exam &lt;- exams2pdf(file = rep(my.f,5),
                     n = n.ver, 
                     dir = my.dir,
                     name = name.exam, 
                     verbose = TRUE)

## Exams generation initialized.
## 
## Output directory: /home/msperlin/Dropbox/My Blog/exam-out
## Exercise directory: /home/msperlin/Dropbox/My Blog
## Supplement directory: /tmp/RtmpaDj6Ju/file27145b4e0310
## Temporary directory: /tmp/RtmpaDj6Ju/file2714cb4b300
## Exercises: Question, Question, Question, Question, Question
## 
## Generation of individual exams.
## Exam 1: Question (srt) Question_1 (srt) Question_2 (srt) Question_3 (srt) Question_4 (srt) ... w ... done.

f.out &lt;- paste0(my.dir,name.exam,'1','.pdf')
file.exists(f.out)

## [1] TRUE
</code></pre>
</div>
<p>The result of the previous code is a pdf file with the following<br>
content:</p>
<p><img alt="" src="https://i1.wp.com/msperlin.github.io/img/exam_sample.png?w=456" data-recalc-dims="1"></p>
<p>One interesting information from this post is that you can find a small<br>
difference in the number of packages in between the CRAN mirrors. My<br>
best guess is that they synchronize with the master server in different<br>
times of the day/week.</p>
<p>Looking at the contents of the pdf file, clearly some things are missing<br>
from the exam, such as the title page and the instructions. You can add<br>
all the bells and whistles with the inputs of function <code>exams2pdf</code> or<br>
change it directly in the different file templates. One quick tip for<br>
new users is that the answer sheet can be found by looping over the<br>
values of the output from <code>exams2pdf</code>:</p>
<div>
<pre><code>df.answer.key &lt;- data.frame()
n.q &lt;- 5 # number of questions
for (i.ver in seq(n.ver)){
  
  exam.now &lt;- my.exam[[i.ver]] 
  
  for (i.q in seq(n.q)){
    
    sol.now &lt;- letters[which(exam.now[[i.q]]$metainfo$solution)]
    
    temp &lt;- data.frame(i.ver = i.ver, i.q = i.q, solution = sol.now)
    df.answer.key &lt;- rbind(df.answer.key, temp)  
  }
  
}

df.answer.key.wide &lt;- tidyr::spread(df.answer.key, key = i.q, value = solution)
df.answer.key.wide

##   i.ver 1 2 3 4 5
## 1     1 d e a c a
</code></pre>
</div>
<p>By using package <code>exams2pdf</code>, I can code different questions in the<br>
<code>exams</code> format and not worry whether someone is going to copy it over<br>
and distribute it in the internet. Students may know the content of each<br>
question, but they will have to learn how to get to the correct answer<br>
in order to solve it for their exam. Cheating is also impossible since<br>
each student will have different versions and different answer sheets.<br>
If I have a class of 100 students, I will build 100 different exams,<br>
each one with unique answers.</p>
<p>As for maintainability, the time value of my exam questions increases<br>
significantly. I can use them over and over, now that I can effortlessly<br>
create as many versions of it as I need. Since it is all based in R<br>
code, I can use the code from the class material in my exams. Going<br>
further, I can also automatically grade the exams using the internet<br>
(see the <a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/packages/RndTexExams/vignettes/rte-vignette_creating_exams.html">vignette of<br>
<code>RndTexExams</code></a><br>
for information on how to do that with Google spreadsheets.)</p>
<p>In this post I only scratched the surface of <code>exams</code>. Adding to the<br>
description of its capabilities, you can <strong>export</strong> exams to standard<br>
academic systems such as Moodle, Blackboard and others. You can also<br>
print the exam in pdf, nops (a pdf that allows easy scanning), or html.<br>
If you know a bit of latex or html, it is easy to customize the<br>
templates to the needs of your particular exam.</p>
<p>As with all technical things, not everything is perfect. In my oppinio,<br>
the main issue with the <code>exams</code> template is that requires some knowledge<br>
of R and Knitr. While this is Ok for most people reading this blog, it<br>
is not the case for the <em>average</em> professor. It may sound surprising to<br>
the quantitative inclined people, but the great majority of professors<br>
still use .docx and .xlsx files to write academic work such as articles<br>
and exams. Why they don’t use or learn better tools? Well, this is a<br>
long answer, best suited for another post.</p>
<p>Package <code>exams</code> had a <strong>big and positive impact on how I do my work</strong>.<br>
Based on a large database of questions that I’ve built, I can create a<br>
new exam in 5 minutes and grade it for a large class in less than 1<br>
minute. I am very thankful to its authors and this is one of the reasons<br>
why I love posting packages in CRAN. It is my way of giving it back to<br>
the community.</p>
<p>Concluding, package <code>exams</code> is great and I believe that every examiner<br>
and professor should be using it. Thinking about the future, the<br>
template of questions in <code>exams</code> has the potential of setting the<br>
<strong>language of exams</strong>, a structure that could allow the user to output<br>
questions in any format he wants, just as you can use Markdown to output<br>
latex or word.</p>
<p>Sharing questions in a collaborative platform, such as Quora, should be<br>
something for the developers (or R community) to think of. Questions<br>
could be ranked according to popular vote. Users could contribute by<br>
posting question files for other to use. Users would get a feedback on<br>
their work and, at the same time, be able to use other people questions.<br>
Students could also have access to it and independently study to a<br>
particular topic, by building custom made exams with randomized content.</p>
<p>Summing up, if you are a teacher or examiner, I hope that this post<br>
convinces you to try out package <code>exams</code>.</p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://msperlin.github.io/2017-01-30-Exams-with-dynamic-content/"> R and Finance</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/8zOVQ3U02r0"></a></div>