<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/taking-steps-in-xml/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://nsaunders.wordpress.com/2017/02/01/taking-steps-in-xml/"> R – What You're Doing Is Rather Desperate</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>So the votes are in:</p>
<div>
<blockquote data-width="550" class="twitter-tweet">
<p dir="ltr" lang="en">Your established blog is mostly about your work. Your work changes. Do you continue at the current blog or start a new one?</p>
<p>— Neil Saunders (@neilfws) <a target="_blank" rel="nofollow" href="https://twitter.com/neilfws/status/823492672987930626">January 23, 2017</a></p>
</blockquote>
<p></div>
<p>I thank you, kind readers. So here’s the plan: (1) keep blogging here as frequently as possible (perhaps monthly), (2) on more general “how to do cool stuff with data and R” topics, (3) which may still include biology from time to time. Sounds OK? Good.</p>
<p>So: let’s use R to analyse data from the iOS Health app.</p>
<p><span></span></p>
<p>I own an iPhone. It comes with a Health app installed by default. Not being a big user of mobile apps, it was several months before I opened it and realised that it had been collecting data. Which makes me wonder what else the phone does without my knowledge…but back to the topic. It turns out that health data can be exported by tapping at top-right on the overview page, then choosing export.</p>
<p><a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/2017/02/01/taking-steps-in-xml/#gallery-4467-1-slideshow">Click to view slideshow.</a></p>
<p>This generates a compressed file, <code>ios_health_export.zip</code>. Upload it from your phone to your destination of choice; I went with Google Drive.</p>
<p>Being Apple, I’d assumed that the contents might be some hideous proprietary binary format but in fact unzipping the file reveals a directory, <code>apple_health_export</code>, in which reside two XML files. The larger <code>export.xml</code> contains your health data.</p>
<p>Records in the XML file consist of lines that specify the record type (measurement), source, three timestamps for creation, start and end, and the value of the measurement. Most of my records are step counts, which look like this:</p>
<pre>
&lt;Record type=&quot;HKQuantityTypeIdentifierStepCount&quot; sourceName=&quot;Health&quot; unit=&quot;count&quot; creationDate=&quot;2014-09-24 09:25:06 +1100&quot; startDate=&quot;2014-09-23 18:01:22 +1100&quot; endDate=&quot;2014-09-23 18:01:24 +1100&quot; value=&quot;9&quot;/>
</pre>
<p>And so to R. In the past I would have used the <code>XML</code> package but in my ongoing effort to convert to the “tidyverse”, I’ll try <code>xml2</code> instead. We’ll use <code>purrr</code> too for reasons that will become apparent, <code>ggplot2</code> for plotting and <code>dplyr</code> because it is awesome.</p>
<p>Reading in the file could not be easier:</p>
<pre>
library(xml2)
library(purrr)
library(ggplot2)
library(dplyr)

health_data &lt;- read_xml(&quot;export.xml&quot;)
</pre>
<p>Nor could extracting the records that contain step counts. We use an xpath expression, then pipe the result to purr’s mapping functions to go straight from XML attributes to a data frame, <a target="_blank" rel="nofollow" href="http://stackoverflow.com/questions/41287397/r-how-to-best-extract-two-xml-attributes-from-a-node">as described here</a>.</p>
<pre>
steps &lt;- xml_find_all(health_data, &quot;.//Record[@type='HKQuantityTypeIdentifierStepCount']&quot;) %>% map(xml_attrs) %>% map_df(as.list)

glimpse(steps)
Observations: 188,677
Variables: 9
$ type          &lt;chr> &quot;HKQuantityTypeIdentifierStepCount&quot;, &quot;HKQuantityTypeIdentifierStepCount&quot;, &quot;HKQuantityTypeIdentifierStepCount&quot;, &quot;HKQuantityTypeIdentifierStepCount&quot;, ...
$ sourceName    &lt;chr> &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;, &quot;Health&quot;...
$ unit          &lt;chr> &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;count&quot;, &quot;cou...
$ creationDate  &lt;chr> &quot;2014-09-24 09:25:06 +1100&quot;, &quot;2014-09-24 09:25:06 +1100&quot;, &quot;2014-09-24 09:25:06 +1100&quot;, &quot;2014-09-24 09:25:06 +1100&quot;, &quot;2014-09-24 09:25:06 +1100&quot;, &quot;20...
$ startDate     &lt;chr> &quot;2014-09-23 17:58:58 +1100&quot;, &quot;2014-09-23 17:59:08 +1100&quot;, &quot;2014-09-23 17:59:18 +1100&quot;, &quot;2014-09-23 17:59:28 +1100&quot;, &quot;2014-09-23 17:59:58 +1100&quot;, &quot;20...
$ endDate       &lt;chr> &quot;2014-09-23 17:59:03 +1100&quot;, &quot;2014-09-23 17:59:13 +1100&quot;, &quot;2014-09-23 17:59:23 +1100&quot;, &quot;2014-09-23 17:59:33 +1100&quot;, &quot;2014-09-23 18:00:03 +1100&quot;, &quot;20...
$ value         &lt;chr> &quot;12&quot;, &quot;5&quot;, &quot;17&quot;, &quot;1&quot;, &quot;14&quot;, &quot;4&quot;, &quot;10&quot;, &quot;2&quot;, &quot;4&quot;, &quot;2&quot;, &quot;9&quot;, &quot;7&quot;, &quot;4&quot;, &quot;9&quot;, &quot;7&quot;, &quot;6&quot;, &quot;11&quot;, &quot;13&quot;, &quot;6&quot;, &quot;8&quot;, &quot;5&quot;, &quot;8&quot;, &quot;6&quot;, &quot;9&quot;, &quot;1&quot;, &quot;7&quot;, &quot;13&quot;, &quot;6&quot;, &quot;...
$ sourceVersion &lt;chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, ...
$ device        &lt;chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, ...
</pre>
<p>To illustrate an example analysis, let’s aggregate steps to a monthly count and plot counts by month. We’ll assume that <code>startDate</code> is a proxy for day (<em>i.e.</em> I’m not walking at midnight so steps don’t straddle day boundaries). We’ll also assign the monthly count to the first day of the month, to avoid having to figure out what number day ends the month <img alt="🙂" class="wp-smiley" src="https://s.w.org/images/core/emoji/2/72x72/1f642.png"></p>
<p>So, to recode step count as an integer, convert the start date to a date object, summarise by month and plot, let’s see <code>dplyr</code> in action:</p>
<pre>
steps %>% select(startDate, value) %>%
group_by(Date = as.Date(paste(substr(startDate, 1, 7), &quot;01&quot;, sep = &quot;-&quot;)))
%>% summarise(count = sum(as.numeric(value))) %>%
ggplot(aes(Date, count)) + geom_col(fill = &quot;skyblue3&quot;) + theme_bw() + labs(y = &quot;monthly step count&quot;, title = &quot;Steps by month September 2014 - January 2017 as measured by iOS&quot;)
</pre>
<p>Result:</p>
<p><img sizes="(max-width: 1040px) 100vw, 1040px" data-image-description="" data-orig-file="https://i2.wp.com/nsaunders.files.wordpress.com/2017/02/ios_steps.png?w=456&amp;ssl=1" data-orig-size="1098,804" data-image-title="ios_steps" class="wp-image-4553" data-comments-opened="1" data-recalc-dims="1" data-attachment-id="4553" data-permalink="https://nsaunders.wordpress.com/2017/02/01/taking-steps-in-xml/ios_steps/" alt="ios_steps" data-medium-file="https://i2.wp.com/nsaunders.files.wordpress.com/2017/02/ios_steps.png?w=456&amp;ssl=1?w=300" data-large-file="https://i2.wp.com/nsaunders.files.wordpress.com/2017/02/ios_steps.png?w=456&amp;ssl=1?w=1024" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" src="https://i2.wp.com/nsaunders.files.wordpress.com/2017/02/ios_steps.png?w=456&ssl=1"></p>
<p>As to how accurate the counts are: that’s for another day.</p>
<p>Filed under: <a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/category/personal/">personal</a>, <a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/category/statistics/r/">R</a>, <a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/category/statistics/">statistics</a>, <a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/category/this-blog/">this blog</a> Tagged: <a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/tag/health/">health</a>, <a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/tag/ios/">iOS</a>, <a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/tag/parsing/">parsing</a>, <a target="_blank" rel="nofollow" href="https://nsaunders.wordpress.com/tag/xml/">xml</a> <a target="_blank" rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/nsaunders.wordpress.com/4467/"><img alt="" src="http://feeds.wordpress.com/1.0/comments/nsaunders.wordpress.com/4467/" border="0"></a> <img alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" border="0" data-recalc-dims="1"></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="https://nsaunders.wordpress.com/2017/02/01/taking-steps-in-xml/"> R – What You're Doing Is Rather Desperate</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/Ji8xk450wAo"></a></div>