<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/r-exams-erum-2018/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.r-exams.org/general/erum2018/"> R/exams</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>Keynote lecture about R/exams at eRum 2018 (European R Users Meeting) in Budapest: Slides, video, e-learning, replication materials.</p>
<p><img alt="R/exams @ eRum 2018" src="https://i0.wp.com/www.r-exams.org/images/erum2018.title.png?w=456" data-recalc-dims="1"></p>
<h3>Keynote lecture at eRum 2018</h3>
<p><a rel="nofollow" href="http://www.r-exams.org/" target="_blank">R/exams</a> was presented in a keynote lecture by <a rel="nofollow" href="https://eeecon.uibk.ac.at/~zeileis/" target="_blank">Achim Zeileis</a> at <a rel="nofollow" href="https://2018.erum.io/" target="_blank">eRum 2018</a>, the European R Users Meeting, this time organized by a team around <a rel="nofollow" href="https://twitter.com/daroczig" target="_blank">Gergely Daróczi</a> in Budapest. It was a great event with many exciting presentations, reflecting the vibrant R community in Europe (and beyond).</p>
<p>This blog post provides various resources accompanying the presentation which may be of interest to those who did not attend the meeting as well as those who did and who want to explore the materials in more detail.</p>
<p>Most importantly the presentation slides are available in PDF format (under CC-BY):</p>
<p><a rel="nofollow" href="http://www.r-exams.org/assets/posts/2018-05-20-erum2018/slides.pdf" target="_blank"><img alt="slides.pdf" src="https://i0.wp.com/www.r-exams.org/assets/posts/2018-05-20-erum2018/slides.png?w=456" data-recalc-dims="1"></a></p>
<h3>Video</h3>
<p>The eRum organizers did a great job in making the meeting accessible to those useRs who could not make it to Budapest. All presentations were available in a livestream on YouTube where also videos of all lectures were made available after the meeting (Standard YouTube License):</p>
<p><a rel="nofollow" href="https://www.youtube.com/watch?v=NMIzbrklBEE" target="_blank"><img alt="YouTube" src="https://i0.wp.com/www.r-exams.org/assets/posts/2018-05-20-erum2018/youtube.png?w=456" data-recalc-dims="1"></a></p>
<h3>E-Learning</h3>
<p>To illustrate the e-learning capabilities supported by R/exams, the presentation started with a live quiz using the audience response system <a rel="nofollow" href="https://arsnova.eu/" target="_blank">ARSnova</a>. The original version of the quiz was hosted on the ARSnova installation at Universität Innsbruck. To encourage readers to try out ARSnova for their own purposes, a copy of the quiz was also posted on the official ARSnova server at Technische Hochschule Mittelhessen (where ARSnova is developed under the General Public License, GPL):</p>
<p><a rel="nofollow" href="https://arsnova.eu/mobile/#id/39755874" target="_blank"><img alt="ARSnova" src="https://i0.wp.com/www.r-exams.org/assets/posts/2018-05-20-erum2018/arsnova.png?w=456" data-recalc-dims="1"></a></p>
<p>The presentation briefly also showed an online test generated by R/exams and imported into <a rel="nofollow" href="https://www.openolat.com/" target="_blank">OpenOLAT</a>, an open-source learning management system (available under the Apache License). The online test is made available again here for anonymous guest access. <em>(Note however, that the system only has one guest user so that when you start the test there may already be some test results from a previous guest session. In that case you can finish the test and also start it again.)</em></p>
<p><a rel="nofollow" href="https://lms.uibk.ac.at/url/RepositoryEntry/2823520256/CourseNode/97603810083315?guest=true" target="_blank"><img alt="OpenOLAT" src="https://i1.wp.com/www.r-exams.org/assets/posts/2018-05-20-erum2018/openolat.png?w=456" data-recalc-dims="1"></a></p>
<h3>Replication code</h3>
<p>The presentation slides show how to set up an exam using the R package and then rendering it into different output formats. In order to allow the same exam to be rendered into a wide range of different output formats, only single-choice and multiple-choice exercises were employed (see the <code>choice</code> list below). However, in the e-learning test shown in OpenOLAT all exercises types are supported (see the <code>elearn</code> list below). All these exercises are readily provided in the package and also introduced online: <a rel="nofollow" href="http://www.r-exams.org/templates/deriv/" target="_blank">deriv</a>/<a rel="nofollow" href="http://www.r-exams.org/templates/deriv2/" target="_blank">deriv2</a>, <a rel="nofollow" href="http://www.r-exams.org/templates/fruit/" target="_blank">fruit</a>/<a rel="nofollow" href="http://www.r-exams.org/templates/fruit2/" target="_blank">fruit2</a>, <a rel="nofollow" href="http://www.r-exams.org/templates/ttest/" target="_blank">ttest</a>, <a rel="nofollow" href="http://www.r-exams.org/templates/boxplots/" target="_blank">boxplots</a>, <a rel="nofollow" href="http://www.r-exams.org/templates/cholesky/" target="_blank">cholesky</a>, <a rel="nofollow" href="http://www.r-exams.org/templates/lm/" target="_blank">lm</a>, <a rel="nofollow" href="http://www.r-exams.org/templates/function/" target="_blank">function</a>. The code below uses the R/LaTeX (.Rnw) version but the R/Markdown version (.Rmd) could also be used instead.</p>
<pre><code>## package
library(&quot;exams&quot;)

## single-choice and multiple-choice only
choice &lt;- list(&quot;deriv2.Rnw&quot;, &quot;fruit2.Rnw&quot;, c(&quot;ttest.Rnw&quot;, &quot;boxplots.Rnw&quot;))

## e-learning test (all exercise types)
elearn &lt;- c(&quot;deriv.Rnw&quot;, &quot;fruit.Rnw&quot;, &quot;ttest.Rnw&quot;, &quot;boxplots.Rnw&quot;,
  &quot;cholesky.Rnw&quot;, &quot;lm.Rnw&quot;, &quot;function.Rnw&quot;)
</code></pre>
<p>First, the exam with the choice-based questions can be easily turned into a PDF exam in <a rel="nofollow" href="http://www.r-exams.org/tutorials/exams2nops/" target="_blank">NOPS format</a> using <code>exams2nops</code>, here using Hungarian language for illustration. Exams in this format can be easily scanned and evaluated within R.</p>
<pre><code>set.seed(2018-05-16)
exams2nops(choice, institution = &quot;eRum 2018&quot;, language = &quot;hu&quot;)
</code></pre>
<p>Second, the choice-based exam version can be exported into the JSON format for ARSnova: <a rel="nofollow" href="http://www.r-exams.org/assets/posts/2018-05-20-erum2018/Rexams-1.json" target="_blank">Rexams-1.json</a>. This contains an entire ARSnova session that can be directly imported into the ARSnova system as shown above. It employs a custom exercise set up just for eRum (<a rel="nofollow" href="http://www.r-exams.org/assets/posts/2018-05-20-erum2018/conferences.Rmd" target="_blank">conferences.Rmd</a>) as well as a slightly tweaked exercise (<a rel="nofollow" href="http://www.r-exams.org/assets/posts/2018-05-20-erum2018/fruit3.Rmd" target="_blank">fruit3.Rmd</a>) that displays better in ARSnova.</p>
<pre><code>set.seed(2018-05-16)
exams2arsnova(list(&quot;conferences.Rmd&quot;, choice[[1]], &quot;fruit3.Rmd&quot;, choice[[3]]),
  name = &quot;R/exams&quot;, abstention = FALSE, fix_choice = TRUE)
</code></pre>
<p>Third, the e-learning exam can be generated in QTI 1.2 format for OpenOLAT, as shown above: <a rel="nofollow" href="http://www.r-exams.org/assets/posts/2018-05-20-erum2018/eRum-2018.zip" target="_blank">eRum-2018.zip</a>. The <code>exams2openolat</code> command below is provided starting from the current <a rel="nofollow" href="http://www.r-exams.org/general/cran_release_231/" target="_blank">R/exams version 2.3-1</a>. It essentially just calls <code>exams2qti12</code> but slightly tweaks the MathJax output from pandoc so that it is displayed properly by OpenOLAT.</p>
<pre><code>set.seed(2018-05-16)
exams2openolat(elearn, name = &quot;eRum-2018&quot;, n = 10, qti = &quot;1.2&quot;)
</code></pre>
<h3>What else?</h3>
<p>In the last part of the presentation a couple of new and ongoing efforts within the R/exams project are highlighted. First, the <a rel="nofollow" href="http://www.r-exams.org/tutorials/nops_language/" target="_blank">natural language support in NOPS exams</a> is mentioned which was recently described in more detail in this blog. Second, the relatively new “stress tester” was illustrated with the following example. (A more detailed blog post will follow soon.)</p>
<pre><code>s &lt;- stresstest_exercise(&quot;deriv2.Rnw&quot;)
plot(s)
</code></pre>
<p>Finally, a psychometric analysis illustrated how to examine exams regarding: Exercise difficulty, student performance, unidimensionality, fairness. The replication code for the results from the slides is included below (omitting some graphical details for simplicity, e.g., labeling or color).</p>
<pre><code>## load data and exclude extreme scorers
library(&quot;psychotools&quot;)
data(&quot;MathExam14W&quot;, package = &quot;psychotools&quot;)
mex &lt;- subset(MathExam14W, nsolved > 0 &amp; nsolved &lt; 13)

## raw data
plot(mex$solved)

## Rasch model parameters
mr &lt;- raschmodel(mex$solved)
plot(mr, type = &quot;profile&quot;)

## points per student
MathExam14W &lt;- transform(MathExam14W,
  points = 2 * nsolved - 0.5 * rowSums(credits == 1)
)
hist(MathExam14W$points, breaks = -4:13 * 2 + 0.5, col = &quot;lightgray&quot;)
abline(v = 12.5, lwd = 2, col = 2)

## person-item map
plot(mr, type = &quot;piplot&quot;)

## principal component analysis
pr &lt;- prcomp(mex$solved, scale = TRUE)
plot(pr)
biplot(pr, col = c(&quot;transparent&quot;, &quot;black&quot;),
  xlim = c(-0.065, 0.005), ylim = c(-0.04, 0.065))

## differential item functioning
mr1 &lt;- raschmodel(subset(mex, group == 1)$solved)
mr2 &lt;- raschmodel(subset(mex, group == 2)$solved)
ma &lt;- anchortest(mr1, mr2, adjust = &quot;single-step&quot;)

## anchored item difficulties
plot(mr1, parg = list(ref = ma$anchor_items), ref = FALSE, ylim = c(-2, 3), pch = 19)
plot(mr2, parg = list(ref = ma$anchor_items), ref = FALSE, add = TRUE, pch = 19, border = 4)
legend(&quot;topleft&quot;, paste(&quot;Group&quot;, 1:2), pch = 19, col = c(1, 4), bty = &quot;n&quot;)

## simultaneous Wald test for pairwise differences
plot(ma$final_tests)
</code></pre>
	<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.r-exams.org/general/erum2018/"> R/exams</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a rel="nofollow" href="https://www.r-project.org/" title="The R Project for Statistical Computing">R</a> news and <a rel="nofollow" href="https://www.r-bloggers.com/search/tutorial" title="R tutorials">tutorials</a> on topics such as: <a rel="nofollow" href="https://www.r-bloggers.com/search/data%20science" title="Data science">Data science</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data" title="Big Data">Big Data, <a rel="nofollow" href="https://www.r-users.com/" title="R jobs">R jobs</a>, visualization (<a rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2" title="ggplot and ggplot2 tutorials">ggplot2</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/boxplot" title="Boxplots using lattice and ggplot2 tutorials">Boxplots</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/map" title="Maps and gis">maps</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/animation" title="Animation in R">animation</a>), programming (<a rel="nofollow" href="https://www.r-bloggers.com/search/RStudio" title="RStudio IDE for R">RStudio</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/sweave" title="Sweave and literate programming">Sweave</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX" title="LaTeX in R">LaTeX</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/SQL" title="SQL and databases">SQL</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/eclipse" title="Eclipse IDE for R">Eclipse</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/git" title="git and github, Version Control System">git</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/hadoop" title="Large data in R using Hadoop">hadoop</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping" title="Web Scraping of google, facebook, yahoo, twitter and more using R">Web Scraping</a>) statistics (<a rel="nofollow" href="https://www.r-bloggers.com/search/regression" title="Regressions and ANOVA analysis tutorials">regression</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/PCA" title="principal component analysis tutorial">PCA</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/time+series" title="Time series">time series</a>, <a rel="nofollow" href="https://www.r-bloggers.com/search/trading" title="finance trading">trading</a>) and more...
<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/hWBuID1oDdI" height="1"></a></div>