<p>Someone asked on Twitter about the characteristics of New Zealand First voters. While some crude conclusions can be drawn from <a target="_blank" rel="nofollow" href="http://ellisp.github.io/blog/2016/04/09/nzelect3">examining votes by location cast</a> and then comparing that with census data, we really need individual level data to answer the question properly. I set myself this task as a motivation for exploring the <a target="_blank" rel="nofollow" href="http://www.nzes.org/">New Zealand Election Study</a>.</p>
<p>This turned into a fairly lengthy blog post and includes more than 500 lines of code. Skip down to the very end if you just want to see how things turn out for the four main parties and the main conclusions from that. But first here’s the best version of the result with relation to New Zealand First:</p>
<p><img width="100%" src="http://ellisp.github.io/img/0096-svyglm-boot.svg"></p>
<p>Later down there’s a chart comparing this to the voters for the National, Labour and Green parties.</p>
<h2>Functionality</h2>
<p>Here’s the R packages I needed to do this analysis, plus a few convenience functions I wrote specifically for it.</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>tidyverse</span><span>)</span><span>
</span><span>library</span><span>(</span><span>scales</span><span>)</span><span>
</span><span>library</span><span>(</span><span>foreign</span><span>)</span><span> </span><span># for importing SPSS data
</span><span>library</span><span>(</span><span>survey</span><span>)</span><span> </span><span># for survey weighting and analysis
</span><span>library</span><span>(</span><span>forcats</span><span>)</span><span> </span><span># for manipulating factor levels
</span><span>library</span><span>(</span><span>mgcv</span><span>)</span><span> </span><span># for generalized additive models
</span><span>library</span><span>(</span><span>glmnet</span><span>)</span><span> </span><span># for elastic net regularisation
</span><span>library</span><span>(</span><span>mice</span><span>)</span><span> </span><span># for imputation
</span><span>library</span><span>(</span><span>testthat</span><span>)</span><span>
</span><span>library</span><span>(</span><span>broom</span><span>)</span><span> </span><span># for reshaping model outputs
</span><span>library</span><span>(</span><span>stringr</span><span>)</span><span> </span><span># for str_wrap
</span><span>library</span><span>(</span><span>boot</span><span>)</span><span>
</span><span>library</span><span>(</span><span>doParallel</span><span>)</span><span>
</span><span>library</span><span>(</span><span>gridExtra</span><span>)</span><span>
</span><span>library</span><span>(</span><span>nzelect</span><span>)</span><span> </span><span># for party colours
</span><span>library</span><span>(</span><span>lme4</span><span>)</span><span> </span><span># for mixed effects / multilevel modelling
</span><span>
</span><span>#-------------convenience functions--------------
</span><span>camel_to_english</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>camelCase</span><span>){</span><span>
</span><span>return</span><span>(</span><span>gsub</span><span>(</span><span>&quot;([a-z])([A-Z])&quot;</span><span>,</span><span> </span><span>&quot;\\1 \\L\\2&quot;</span><span>,</span><span> </span><span>camelCase</span><span>,</span><span> </span><span>perl</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>))</span><span>
</span><span>}</span><span>
</span><span># Convert five category membership question (for trade unions, business
# associations, etc) into Yes or No.
</span><span>membership</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>x</span><span>){</span><span>
</span><span>tmp</span><span> </span><span>&lt;-</span><span> </span><span>fct_recode</span><span>(</span><span>x</span><span>,</span><span>
</span><span>Yes</span><span> </span><span>=</span><span> </span><span>&quot;I belong, but no one else in the house does&quot;</span><span>,</span><span>
</span><span>Yes</span><span> </span><span>=</span><span> </span><span>&quot;I do, plus another in the house&quot;</span><span>,</span><span>
</span><span>Yes</span><span> </span><span>=</span><span> </span><span>&quot;Another person belongs, but not me&quot;</span><span>,</span><span>
</span><span>No</span><span> </span><span>=</span><span> </span><span>&quot;No, no one belongs&quot;</span><span>,</span><span>
</span><span>No</span><span> </span><span>=</span><span> </span><span>&quot;Don't know&quot;</span><span>)</span><span> </span><span>tmp</span><span> </span><span>&lt;-</span><span> </span><span>ifelse</span><span>(</span><span>tmp</span><span> </span><span>==</span><span> </span><span>&quot;Yes&quot;</span><span>,</span><span> </span><span>1</span><span>,</span><span> </span><span>0</span><span>)</span><span>
</span><span># Uncomment the next line if we want to turn NA into 0.
</span><span> </span><span># tmp &lt;-ifelse(is.na(tmp), 0, tmp)
</span><span> </span><span>return</span><span>(</span><span>tmp</span><span>)</span><span>
</span><span>}</span><span>
</span><span># Draw confidence interval segment chart for two types of models:
# those with variance-covariance matrices, and hence can use confint()
# on them; and pooled models created by with.mice() after multiple
# imputation.
</span><span>sum_plot</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>model</span><span>,</span><span> </span><span>title</span><span> </span><span>=</span><span> </span><span>NULL</span><span>,</span><span> </span><span>party</span><span> </span><span>=</span><span> </span><span>gsub</span><span>(</span><span>&quot;PartyVote&quot;</span><span>,</span><span> </span><span>&quot;&quot;</span><span>,</span><span> </span><span>as.character</span><span>(</span><span>model</span><span>$</span><span>formula</span><span>)[</span><span>2</span><span>]),</span><span>
</span><span>colour</span><span> </span><span>=</span><span> </span><span>&quot;#000000&quot;</span><span>,</span><span>
</span><span>type</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>&quot;vcov&quot;</span><span>,</span><span> </span><span>&quot;pool&quot;</span><span>)){</span><span>
</span><span>type</span><span> </span><span>&lt;-</span><span> </span><span>match.arg</span><span>(</span><span>type</span><span>)</span><span>
</span><span>if</span><span>(</span><span>type</span><span> </span><span>==</span><span> </span><span>&quot;vcov&quot;</span><span>){</span><span>
</span><span>conf_ints</span><span> </span><span>&lt;-</span><span> </span><span>cbind</span><span>(</span><span>tidy</span><span>(</span><span>confint</span><span>(</span><span>model</span><span>)),</span><span> </span><span>coef</span><span>(</span><span>model</span><span>))</span><span>
</span><span>}</span><span> </span><span>else</span><span> </span><span>{</span><span>
</span><span>conf_ints</span><span> </span><span>&lt;-</span><span> </span><span>tidy</span><span>(</span><span>summary</span><span>(</span><span>pool</span><span>(</span><span>model</span><span>)))</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>.rownames</span><span>,</span><span> </span><span>lo.95</span><span>,</span><span> </span><span>hi.95</span><span>,</span><span> </span><span>est</span><span>)</span><span>
</span><span>}</span><span>
</span><span>tmp</span><span> </span><span>&lt;-</span><span> </span><span>conf_ints</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>.rownames</span><span> </span><span>!=</span><span> </span><span>&quot;(Intercept)&quot;</span><span>)</span><span>
</span><span>names</span><span>(</span><span>tmp</span><span>)</span><span> </span><span>&lt;-</span><span> </span><span>c</span><span>(</span><span>&quot;var&quot;</span><span>,</span><span> </span><span>&quot;lower&quot;</span><span>,</span><span> </span><span>&quot;upper&quot;</span><span>,</span><span> </span><span>&quot;point&quot;</span><span>)</span><span>
</span><span>p</span><span> </span><span>&lt;-</span><span> </span><span>tmp</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>var</span><span> </span><span>=</span><span> </span><span>camel_to_english</span><span>(</span><span>var</span><span>),</span><span>
</span><span>var</span><span> </span><span>=</span><span> </span><span>fct_reorder</span><span>(</span><span>var</span><span>,</span><span> </span><span>point</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>y</span><span> </span><span>=</span><span> </span><span>var</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_vline</span><span>(</span><span>xintercept</span><span>=</span><span> </span><span>0</span><span>)</span><span> </span><span>+</span><span>
</span><span>geom_segment</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>lower</span><span>,</span><span> </span><span>xend</span><span> </span><span>=</span><span> </span><span>upper</span><span>,</span><span> </span><span>yend</span><span> </span><span>=</span><span> </span><span>var</span><span>),</span><span> </span><span>size</span><span> </span><span>=</span><span> </span><span>3</span><span>,</span><span> </span><span>colour</span><span> </span><span>=</span><span> </span><span>colour</span><span>,</span><span> </span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>)</span><span> </span><span>+</span><span>
</span><span>geom_point</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>point</span><span>))</span><span> </span><span>+</span><span>
</span><span>ggtitle</span><span>(</span><span>title</span><span>)</span><span> </span><span>+</span><span>
</span><span>labs</span><span>(</span><span>caption</span><span> </span><span>=</span><span> </span><span>&quot;New Zealand Election Survey, analysed at https://ellispgithub.io&quot;</span><span>,</span><span>
</span><span>x</span><span> </span><span>=</span><span> </span><span>paste</span><span>(</span><span>&quot;Impact on log odds of voting&quot;</span><span>,</span><span> </span><span>party</span><span>),</span><span>
</span><span>y</span><span> </span><span>=</span><span> </span><span>str_wrap</span><span>(</span><span>&quot;Compared to 'no religion', ''age30-55', 'high household income', 'school qualification', 'working full time&quot;</span><span>,</span><span> </span><span>50</span><span>))</span><span>
</span><span>return</span><span>(</span><span>p</span><span>)</span><span>
</span><span>}</span></code></pre>
</figure>
<h2>Sourcing the New Zealand Election Study data</h2>
<p>The New Zealand Election Study generously makes its <a target="_blank" rel="nofollow" href="http://www.nzes.org/exec/show/data">data available for free</a>. The code snippet below assumes the zip file with the SPSS version of the data has been downloaded into the location specified on d: drive; obviously change this to wherever you have it saved.</p>
<figure>
<pre><code data-lang="r"><span>#------------------data download-----------------
</span><span>
</span><span># Data downloaded from http://www.nzes.org/exec/show/data and because
# they want you to fill in a form to know who is using the data, I
# won't re-publish it myself
</span><span>unzip</span><span>(</span><span>&quot;D:/Downloads/NZES2014GeneralReleaseApril16.sav.zip&quot;</span><span>)</span><span>
</span><span>nzes_orig</span><span> </span><span>&lt;-</span><span> </span><span>read.spss</span><span>(</span><span>&quot;NZES2014GeneralReleaseApril16.sav&quot;</span><span>,</span><span> </span><span>to.data.frame</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>,</span><span> </span><span>trim.factor.names</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span><span>
</span><span>varlab</span><span> </span><span>&lt;-</span><span> </span><span>cbind</span><span>(</span><span>attributes</span><span>(</span><span>nzes_orig</span><span>)</span><span>$</span><span>variable.labels</span><span>)</span><span>
</span><span>DT</span><span>::</span><span>datatable</span><span>(</span><span>varlab</span><span>)</span></code></pre>
</figure>
<p>SPSS data includes both short column names and full verbose questions, so the data frame of metadata called <code>varlab</code> in the R snippet above looks like this:</p>
<p><iframe width="450" src="http://ellisp.github.io/img/0096-nzes-variables.html"></iframe></p>
<h2>Data prep</h2>
<p>The sample size is 2,835 and it includes 234 respondents who claim they party-voted for New Zealand First (note that nine of these were identified by the researchers as actually not voting at all; I made the call to count people on the basis of their self-report). I’m interested in a demographic characterisation of the New Zealand First voters compared to the rest of the New Zealand population (rather than, for example, compared to the voters of a single other party) so I’m going to be using a logistic regression with a response variable of 1 if the respondent voted for NZ First and 0 otherwise.</p>
<blockquote>
<p>It’s often not appreciated that, on a rule of thumb, you need more data per explanatory variable for logistic regression than a continuous response: twice as much if the response is split 50/50, 10 times as much if it’s split 90/10.</p>
</blockquote>
<p>Based on Frank Harrell’s sample size advice in <em>Regression Modeling Strategies</em> I’ve got somewhere between 11 and 23 degrees of freedom to “spend” on explanatory variables; that is 1/10 or 1/20 of the number of cases of the least frequent value of the categorical response variable. I certainly can’t afford to use all 437 columns in the data. Many of those columns are attitudinal variables that are out of scope for today (hope to come back later), but there’s still far too much socio-economic and demographic data to use it all in a model without more data. So I follow Harrell’s general approach of working out how many degrees of freedom I have to spend; which explanatory variables are committed to be in (no sneaky stepwise reductions based on p values); and how to allocate the degrees of freedom to those variables.</p>
<p>My options were restricted by what is available, and with what is available I had to make choices. For example, I collapsed down the nine household income categories into just two: “lower” and “higher”. In some cases, not being an expert in the exactly best data for theoretically interesting questions, I made some fairly arbitrary choices eg to use the answer to “what is your housing status?” to identify owner-occupiers, rather than “do you or any family member own a residence?”</p>
<p>Here’s the code that created the explanatory variables I wanted, a data set that is just a subset of the explanatory variables, the survey weights, and four possible response variables (party vote for the four highest voting parties).</p>
<figure>
<pre><code data-lang="r"><span>#============rationalised version of feature creation===========
# This is a single 100 line command to aggregate various answers into
# simpler cateogrisations, because we don't have enough data to # analyse the original granular detail.
</span><span>nzes</span><span> </span><span>&lt;-</span><span> </span><span>nzes_orig</span><span> </span><span>%>%</span><span>
</span><span># party vote:
</span><span> </span><span>mutate</span><span>(</span><span>dpartyvote2</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>is.na</span><span>(</span><span>dpartyvote</span><span>),</span><span> </span><span>&quot;Did not vote&quot;</span><span>,</span><span> </span><span>as.character</span><span>(</span><span>dpartyvote</span><span>)),</span><span>
</span><span>dpartyvote2</span><span> </span><span>=</span><span> </span><span>gsub</span><span>(</span><span>&quot;M.ori&quot;</span><span>,</span><span> </span><span>&quot;Maori&quot;</span><span>,</span><span> </span><span>dpartyvote2</span><span>),</span><span>
</span><span>dpartyvote2</span><span> </span><span>=</span><span> </span><span>gsub</span><span>(</span><span>&quot;net.Man&quot;</span><span>,</span><span> </span><span>&quot;net-Man&quot;</span><span>,</span><span> </span><span>dpartyvote2</span><span>),</span><span>
</span><span>dpartyvote2</span><span> </span><span>=</span><span> </span><span>fct_infreq</span><span>(</span><span>dpartyvote2</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>PartyVoteNZFirst</span><span> </span><span>=</span><span> </span><span>(</span><span>dpartyvote2</span><span> </span><span>==</span><span> </span><span>&quot;NZ First&quot;</span><span>),</span><span>
</span><span>PartyVoteLabour</span><span> </span><span>=</span><span> </span><span>(</span><span>dpartyvote2</span><span> </span><span>==</span><span> </span><span>&quot;Labour&quot;</span><span>),</span><span>
</span><span>PartyVoteNational</span><span> </span><span>=</span><span> </span><span>(</span><span>dpartyvote2</span><span> </span><span>==</span><span> </span><span>&quot;National&quot;</span><span>),</span><span>
</span><span>PartyVoteGreen</span><span> </span><span>=</span><span> </span><span>(</span><span>dpartyvote2</span><span> </span><span>==</span><span> </span><span>&quot;Green&quot;</span><span>))</span><span> </span><span>%>%</span><span>
</span><span># voted at all (needed for weighting):
</span><span> </span><span>mutate</span><span>(</span><span>Voted</span><span> </span><span>=</span><span> </span><span>1</span><span> </span><span>*</span><span> </span><span>(</span><span>ddidvote</span><span> </span><span>==</span><span> </span><span>1</span><span>))</span><span> </span><span>%>%</span><span>
</span><span># Two degrees of freedom for ethnicity:
</span><span> </span><span>mutate</span><span>(</span><span>NotEuropean</span><span> </span><span>=</span><span> </span><span>1</span><span> </span><span>-</span><span> </span><span>dethnicity_e</span><span>,</span><span>
</span><span>Maori</span><span> </span><span>=</span><span> </span><span>dethnicity_m</span><span>)</span><span> </span><span>%>%</span><span>
</span><span># Two degrees of freedom for income (lower, higher, don't know):
</span><span> </span><span>mutate</span><span>(</span><span>HHIncome</span><span> </span><span>=</span><span> </span><span>fct_recode</span><span>(</span><span>dhhincome</span><span>,</span><span>
</span><span>Lower</span><span> </span><span>=</span><span> </span><span>&quot;No income&quot;</span><span>,</span><span>
</span><span>Lower</span><span> </span><span>=</span><span> </span><span>&quot;$NZ23,000 or less&quot;</span><span>,</span><span>
</span><span>Lower</span><span> </span><span>=</span><span> </span><span>&quot;$NZ23,001-$NZ31,000&quot;</span><span>,</span><span>
</span><span>Lower</span><span> </span><span>=</span><span> </span><span>&quot;$NZ31,001-$NZ39,800&quot;</span><span>,</span><span>
</span><span>Lower</span><span> </span><span>=</span><span> </span><span>&quot;$NZ39,801-$NZ55,000&quot;</span><span>,</span><span>
</span><span>Higher</span><span> </span><span>=</span><span> </span><span>&quot;$NZ55,001-$NZ76,100&quot;</span><span>,</span><span>
</span><span>Higher</span><span> </span><span>=</span><span> </span><span>&quot;$NZ76,101-$NZ110,800&quot;</span><span>,</span><span>
</span><span>Higher</span><span> </span><span>=</span><span> </span><span>&quot;$NZ110,801-$NZ147,699&quot;</span><span>,</span><span>
</span><span>Higher</span><span> </span><span>=</span><span> </span><span>&quot;$NZ147,700 or over&quot;</span><span>,</span><span>
</span><span>`Don't know / NA`</span><span> </span><span>=</span><span> </span><span>&quot;Don't know&quot;</span><span>),</span><span>
</span><span>HHIncome</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>is.na</span><span>(</span><span>HHIncome</span><span>),</span><span> </span><span>&quot;Don't know / NA&quot;</span><span>,</span><span> </span><span>as.character</span><span>(</span><span>HHIncome</span><span>)),</span><span>
</span><span>HHIncome</span><span> </span><span>=</span><span> </span><span>fct_infreq</span><span>(</span><span>HHIncome</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>## Two - four degrees of freedom for associations?
</span><span> </span><span>mutate</span><span>(</span><span>HHMemberTradeUnion</span><span> </span><span>=</span><span> </span><span>membership</span><span>(</span><span>dtradeunion</span><span>),</span><span>
</span><span>HHMemberProfAssoc</span><span> </span><span>=</span><span> </span><span>membership</span><span>(</span><span>dprofassoc</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>## One degree of freedom for born in NZ
</span><span> </span><span>mutate</span><span>(</span><span>NZBorn</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>dnzborn</span><span> </span><span>==</span><span> </span><span>&quot;New Zealand&quot;</span><span>,</span><span> </span><span>1</span><span>,</span><span> </span><span>0</span><span>)</span><span>
</span><span># uncomment the next line if you want to turn NA into zero:
</span><span> </span><span># , NZBorn = ifelse(is.na(NZBorn), 0, NZBorn)
</span><span> </span><span>)</span><span> </span><span>%>%</span><span>
</span><span>## One for sex
</span><span> </span><span>mutate</span><span>(</span><span>Male</span><span> </span><span>=</span><span> </span><span>1</span><span> </span><span>*</span><span> </span><span>(</span><span>dsex</span><span> </span><span>==</span><span> </span><span>&quot;Male&quot;</span><span>),</span><span>
</span><span>Male</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>dsex</span><span> </span><span>==</span><span> </span><span>&quot;Transsexual or transgender&quot;</span><span>,</span><span> </span><span>NA</span><span>,</span><span> </span><span>Male</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>## Two-four for age
</span><span> </span><span>mutate</span><span>(</span><span>Age</span><span> </span><span>=</span><span> </span><span>fct_collapse</span><span>(</span><span>as.character</span><span>(</span><span>dage</span><span>),</span><span>
</span><span>`18-29`</span><span> </span><span>=</span><span> </span><span>as.character</span><span>(</span><span>18</span><span>:</span><span>29</span><span>),</span><span>
</span><span>`30-55`</span><span> </span><span>=</span><span> </span><span>as.character</span><span>(</span><span>30</span><span>:</span><span>55</span><span>),</span><span>
</span><span>`56+`</span><span> </span><span>=</span><span> </span><span>as.character</span><span>(</span><span>56</span><span>:</span><span>100</span><span>)),</span><span>
</span><span># Uncomment the next line if you want to explicitly code the NAs
</span><span> </span><span># Age = ifelse(is.na(dage), &quot;unknown&quot;, as.character(Age)),
</span><span> </span><span>Age</span><span> </span><span>=</span><span> </span><span>fct_relevel</span><span>(</span><span>Age</span><span>,</span><span> </span><span>&quot;30-55&quot;</span><span>)</span><span>
</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>## One for housing. Note there is also an alternative question &quot;do you or any family member own a residence&quot;
</span><span> </span><span>mutate</span><span>(</span><span>OwnHouseOrFlat</span><span> </span><span>=</span><span> </span><span>1</span><span> </span><span>*</span><span> </span><span>grepl</span><span>(</span><span>&quot;Own house or flat&quot;</span><span>,</span><span> </span><span>dhousing</span><span>))</span><span> </span><span>%>%</span><span>
</span><span># Two for religion
</span><span> </span><span>mutate</span><span>(</span><span>Religion</span><span> </span><span>=</span><span> </span><span>fct_lump</span><span>(</span><span>dreligion</span><span>,</span><span> </span><span>2</span><span>))</span><span> </span><span>%>%</span><span>
</span><span># One for marital status
</span><span> </span><span>mutate</span><span>(</span><span>Marital</span><span> </span><span>=</span><span> </span><span>1</span><span> </span><span>*</span><span> </span><span>(</span><span>dmarital</span><span> </span><span>==</span><span> </span><span>&quot;Married, in a civil union, or living with a partner&quot;</span><span>))</span><span> </span><span>%>%</span><span>
</span><span># One for self-identified class
</span><span> </span><span>mutate</span><span>(</span><span>IdentifyWorkingClass</span><span> </span><span>=</span><span> </span><span>1</span><span> </span><span>*</span><span> </span><span>(</span><span>dclass</span><span> </span><span>==</span><span> </span><span>&quot;Working class&quot;</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>## Two for education (University, None, Other)
</span><span> </span><span>mutate</span><span>(</span><span>HighestQual</span><span> </span><span>=</span><span> </span><span>fct_collapse</span><span>(</span><span>dedcons</span><span>,</span><span> </span><span>University</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>&quot;Undergraduate Degree&quot;</span><span>,</span><span> </span><span>&quot;Postgraduate degree&quot;</span><span>,</span><span> </span><span>&quot;PhD&quot;</span><span>)),</span><span>
</span><span>HighestQual</span><span> </span><span>=</span><span> </span><span>fct_lump</span><span>(</span><span>HighestQual</span><span>,</span><span> </span><span>2</span><span>),</span><span>
</span><span>HighestQual</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>dedcons</span><span> </span><span>==</span><span> </span><span>&quot;Not known&quot;</span><span>,</span><span> </span><span>NA</span><span>,</span><span> </span><span>as.character</span><span>(</span><span>HighestQual</span><span>)),</span><span>
</span><span>HighestQual</span><span> </span><span>=</span><span> </span><span>fct_relevel</span><span>(</span><span>HighestQual</span><span>,</span><span> </span><span>&quot;Other&quot;</span><span>)</span><span>
</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>## Two for working status
</span><span> </span><span>mutate</span><span>(</span><span>WorkStatus</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>!</span><span>is.na</span><span>(</span><span>dwkpt</span><span>),</span><span> </span><span>&quot;Part time&quot;</span><span>,</span><span> </span><span>NA</span><span>),</span><span>
</span><span>WorkStatus</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>!</span><span>is.na</span><span>(</span><span>dwkft</span><span>),</span><span> </span><span>&quot;Full time&quot;</span><span>,</span><span> </span><span>WorkStatus</span><span>),</span><span>
</span><span>WorkStatus</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>is.na</span><span>(</span><span>WorkStatus</span><span>),</span><span> </span><span>&quot;Other or unknown&quot;</span><span>,</span><span> </span><span>WorkStatus</span><span>),</span><span>
</span><span>WorkStatus</span><span> </span><span>=</span><span> </span><span>fct_infreq</span><span>(</span><span>WorkStatus</span><span>),</span><span>
</span><span>Student</span><span> </span><span>=</span><span> </span><span>1</span><span> </span><span>*</span><span> </span><span>(</span><span>!</span><span>is.na</span><span>(</span><span>dwksch</span><span>)))</span><span> </span><span>%>%</span><span>
</span><span>## One for occupation
</span><span> </span><span>mutate</span><span>(</span><span>SuperviseAnyone</span><span> </span><span>=</span><span> </span><span>1</span><span> </span><span>*</span><span> </span><span>(</span><span>dsupervise</span><span> </span><span>==</span><span> </span><span>&quot;Yes&quot;</span><span>))</span><span> </span><span>%>%</span><span>
</span><span># Note there is detailed occupation information (for respondent and partner)
</span><span> </span><span># but I don't think we hav eneough data to use this in the model.
</span><span> </span><span>## None for industry.
</span><span> </span><span># We have industry of employhment for respondent and partner
</span><span> </span><span># but I don't think we have enough data to use this. Plus many people
</span><span> </span><span># don't know what industry they are in anyway.
</span><span> </span><span>#
</span><span> </span><span>## One degree of freedom for area lived in?
</span><span> </span><span># Five nice categories here, not enough data so we'll split into one
</span><span> </span><span>mutate</span><span>(</span><span>City</span><span> </span><span>=</span><span> </span><span>1</span><span> </span><span>*</span><span> </span><span>(</span><span>dregsize</span><span> </span><span>==</span><span> </span><span>&quot;A major city (over 100,000 population)&quot;</span><span>))</span><span> </span><span>#============filter and stockcheck missing data===========
# nzes_subset is a subset of all the columns in the original nzes.
# we want a dataframe with only the variables we intend to use;
# will use it down the track for imputation.
</span><span>nzes_subset</span><span> </span><span>&lt;-</span><span> </span><span>nzes</span><span> </span><span>%>%</span><span>
</span><span>select</span><span>(</span><span>PartyVoteNZFirst</span><span>,</span><span>
</span><span>PartyVoteNational</span><span>,</span><span>
</span><span>PartyVoteLabour</span><span>,</span><span>
</span><span>PartyVoteGreen</span><span>,</span><span>
</span><span>NotEuropean</span><span>,</span><span> </span><span>Maori</span><span>,</span><span>
</span><span>HHIncome</span><span>,</span><span> </span><span>OwnHouseOrFlat</span><span>,</span><span>
</span><span>HHMemberTradeUnion</span><span>,</span><span> </span><span>HHMemberProfAssoc</span><span>,</span><span>
</span><span>NZBorn</span><span>,</span><span>
</span><span>Religion</span><span>,</span><span>
</span><span>Marital</span><span>,</span><span>
</span><span>HighestQual</span><span>,</span><span>
</span><span>IdentifyWorkingClass</span><span>,</span><span>
</span><span>WorkStatus</span><span>,</span><span>
</span><span>Student</span><span>,</span><span>
</span><span>SuperviseAnyone</span><span>,</span><span>
</span><span>City</span><span>,</span><span>
</span><span>Male</span><span>,</span><span>
</span><span>Age</span><span>,</span><span>
</span><span>dage</span><span>,</span><span>
</span><span>dwtfin</span><span>,</span><span>
</span><span>Voted</span><span>)</span><span>
</span><span># 29% rows missing at least one value:
</span><span>sum</span><span>(</span><span>complete.cases</span><span>(</span><span>nzes_subset</span><span>))</span><span> </span><span>/</span><span> </span><span>nrow</span><span>(</span><span>nzes_subset</span><span>)</span></code></pre>
</figure>
<p>As the last comment in the code above notes, about 29% of rows are missing at least one value. There are three ways of dealing with this:</p>
<ol>
<li>knock out all rows missing a value</li>
<li>explicitly include a “missing” dummy variable for each factor (which means about a dozen or more extra degrees of freedom)</li>
<li>multiple imputation (creating multiple datasets with different imputed values, reflecting uncertainty in their expected values as well as population randomness – and pooling models estimated from each dataset).</li>
</ol>
<p>Imputation is definitely the best and I will come back to it, but in the early stages I am knocking out those 29% of the data, for workflow simplicity.</p>
<h2>glm versus svyglm versus glm with weights</h2>
<p>The NZES data deliberately oversampled some groups (conspicuously, Māori) and used weights to control for this and for the incidental imbalance by sex, age, education, and non-voting (voters were also oversampled).</p>
<p>My first model was deliberately naive and I chose to ignore the weights, something I wouldn’t do but I wanted to see what impact it had. This gave me these results:</p>
<p><img width="100%" src="http://ellisp.github.io/img/0096-glm-simple.svg"></p>
<p>When I correctly used the survey weights and estimated the model via Thomas Lumley’s <code>survey</code> package I got this, slightly differing set of results:</p>
<p><img width="100%" src="http://ellisp.github.io/img/0096-svyglm.svg"></p>
<p>To check this, I also tried the base function <code>glm</code> with a <code>weights = </code> argument. The publishers normalised the weights so their mean value is nearly exactly 1 (which wouldn’t be the case with a survey used to estimate population totals for official statistics purposes), so in principle this should be basically ok:</p>
<p><img width="100%" src="http://ellisp.github.io/img/0096-glm-weights.svg"></p>
<p>The naive method returns different results from the two that correctly adjust for the weights. In particular, it over-estimates the evidence linking Māori ethnicity and unknown household income to a New Zealand First vote. The two methods that adjust for weights give basically identical results to eachother.</p>
<p>Here’s the code that fitted those first three trial models:</p>
<figure>
<pre><code data-lang="r"><span># For convenience, here is a model formulation we'll be using several times:
</span><span>form</span><span> </span><span>&lt;-</span><span> </span><span>as.formula</span><span>(</span><span>PartyVoteNZFirst</span><span> </span><span>~</span><span> </span><span>NotEuropean</span><span> </span><span>+</span><span> </span><span>Maori</span><span> </span><span>+</span><span> </span><span>HHIncome</span><span> </span><span>+</span><span> </span><span>OwnHouseOrFlat</span><span> </span><span>+</span><span>
</span><span>HHMemberTradeUnion</span><span> </span><span>+</span><span> </span><span>HHMemberProfAssoc</span><span> </span><span>+</span><span>
</span><span>NZBorn</span><span> </span><span>+</span><span>
</span><span>Religion</span><span> </span><span>+</span><span>
</span><span>Marital</span><span> </span><span>+</span><span> </span><span>HighestQual</span><span> </span><span>+</span><span>
</span><span>IdentifyWorkingClass</span><span> </span><span>+</span><span>
</span><span>WorkStatus</span><span> </span><span>+</span><span>
</span><span>Student</span><span> </span><span>+</span><span>
</span><span>SuperviseAnyone</span><span> </span><span>+</span><span>
</span><span>City</span><span> </span><span>+</span><span>
</span><span>Male</span><span> </span><span>+</span><span>
</span><span>Age</span><span>)</span><span>
</span><span>#---------glm v svyglm with knocking out the NAs--------------------
</span><span>model_naive</span><span> </span><span>&lt;-</span><span> </span><span>glm</span><span>(</span><span>form</span><span>,</span><span> </span><span>data</span><span> </span><span>=</span><span> </span><span>nzes</span><span>,</span><span> </span><span>family</span><span> </span><span>=</span><span> </span><span>&quot;binomial&quot;</span><span>)</span><span>
</span><span>sum_plot</span><span>(</span><span>model_naive</span><span>,</span><span> </span><span>&quot;Generalized linear model, no weights or imputation&quot;</span><span>)</span><span>
</span><span>nzes_svy1</span><span> </span><span>&lt;-</span><span> </span><span>svydesign</span><span>(</span><span>~</span><span>1</span><span>,</span><span> </span><span>data</span><span> </span><span>=</span><span> </span><span>nzes</span><span>,</span><span> </span><span>weights</span><span> </span><span>=</span><span> </span><span>~</span><span>dwtfin</span><span>)</span><span>
</span><span>model_svy1</span><span> </span><span>&lt;-</span><span> </span><span>svyglm</span><span>(</span><span>form</span><span>,</span><span> </span><span>design</span><span> </span><span>=</span><span> </span><span>nzes_svy1</span><span>,</span><span> </span><span>family</span><span> </span><span>=</span><span> </span><span>&quot;quasibinomial&quot;</span><span>)</span><span>
</span><span>sum_plot</span><span>(</span><span>model_svy1</span><span>,</span><span> </span><span>&quot;Survey-specific `svyglm``, survey weights, no imputation&quot;</span><span>)</span><span>
</span><span>model_glmw</span><span> </span><span>&lt;-</span><span> </span><span>glm</span><span>(</span><span>form</span><span>,</span><span> </span><span>data</span><span> </span><span>=</span><span> </span><span>nzes</span><span>,</span><span> </span><span>family</span><span> </span><span>=</span><span> </span><span>&quot;quasibinomial&quot;</span><span>,</span><span> </span><span>weights</span><span> </span><span>=</span><span> </span><span>dwtfin</span><span>)</span><span>
</span><span>sum_plot</span><span>(</span><span>model_glmw</span><span>,</span><span> </span><span>&quot;'Usual' `glm`, survey weights, no imputation&quot;</span><span>)</span></code></pre>
</figure>
<h2>Non-linear age effect</h2>
<p>The data provide an age in years for each respondent (except missing values), which unlike the other variables leaves open the possibility of modelling a non-lineary curved relationship between age and the logarithm of the odds of voting New Zealand First. In my standard models I had split age into three categories (18-29, 30-55, and 56+, with 30-55 the reference point) which would allow some crude non-linearity but it is of interest to see if there is something else going on. To test this, I fit a generalized additive model (GAM) with the <code>mgcv</code> package, which allows the user to specify weights. The results for all the variables of interest were very similar and there are complexities in producing confidence intervals for fixed effects out of a GAM, so I’m not presenting them here. Instead, here’s the relationship between age and voting New Zealand First:</p>
<p><img width="100%" src="http://ellisp.github.io/img/0096-gam.svg"></p>
<p>There’s a definite non-linearity here, but it’s not strong or defined enough for me to worry about, and I decided it’s easier to stick to my three simple categories of young, middle and old from here on. It looks like the strong impact really kicks in at about aged 65 rather 55, but I can’t go back and change my category coding just to accommodate that or I’ll be guilty of p-hacking.</p>
<p>Here’s the code for fitting the GAM:</p>
<figure>
<pre><code data-lang="r"><span>#---------------GAM with weights------------------------------
</span><span>model_gam</span><span> </span><span>&lt;-</span><span> </span><span>gam</span><span>(</span><span>PartyVoteNZFirst</span><span> </span><span>~</span><span> </span><span>NotEuropean</span><span> </span><span>+</span><span> </span><span>Maori</span><span> </span><span>+</span><span> </span><span>HHIncome</span><span> </span><span>+</span><span> </span><span>OwnHouseOrFlat</span><span> </span><span>+</span><span>
</span><span>HHMemberTradeUnion</span><span> </span><span>+</span><span> </span><span>HHMemberProfAssoc</span><span> </span><span>+</span><span>
</span><span>NZBorn</span><span> </span><span>+</span><span>
</span><span>Religion</span><span> </span><span>+</span><span>
</span><span>Marital</span><span> </span><span>+</span><span> </span><span>HighestQual</span><span> </span><span>+</span><span>
</span><span>IdentifyWorkingClass</span><span> </span><span>+</span><span>
</span><span>WorkStatus</span><span> </span><span>+</span><span>
</span><span>Student</span><span> </span><span>+</span><span>
</span><span>SuperviseAnyone</span><span> </span><span>+</span><span>
</span><span>City</span><span> </span><span>+</span><span>
</span><span>Male</span><span> </span><span>+</span><span>
</span><span>s</span><span>(</span><span>dage</span><span>),</span><span>
</span><span>data</span><span> </span><span>=</span><span> </span><span>nzes</span><span>,</span><span> </span><span>family</span><span> </span><span>=</span><span> </span><span>&quot;quasibinomial&quot;</span><span>,</span><span> </span><span>weights</span><span> </span><span>=</span><span> </span><span>dwtfin</span><span>)</span><span>
</span><span>summary</span><span>(</span><span>model_gam</span><span>)</span><span>
</span><span>plot</span><span>(</span><span>model_gam</span><span>,</span><span> </span><span>shade</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>,</span><span> </span><span>main</span><span> </span><span>=</span><span> </span><span>&quot;Non-linear impact of age on voting for NZ First&quot;</span><span>)</span></code></pre>
</figure>
<h2>Elastic net regularization</h2>
<p>To get a feel for the robustness of the results, I also tried using both the “lasso” and “ridge regression” to shrink coefficient estimates towards zero. Results not shown here but basically all coefficient estimates were reduced to zero in either method. This intrigues me… but a thorough investigation will have to wait for a later post (maybe).</p>
<h2>Mixed-effects (multilevel) modelling with a regional electorate effect</h2>
<p>I’ve just finished Andrew Gelman’s <a target="_blank" rel="nofollow" href="http://press.princeton.edu/titles/9030.html">Red State, Blue State, Rich State, Poor State: Why Americans Vote the Way They Do</a>. One of my key takeaways was that while richer states vote Democrats over Republicans (and poor states vice versa), within each state richer individuals vote Republican over Democrats (and poorer individuals vice versa); and the slopes vary in each state. That is, there’s an interaction effect between region and individual income as explanatory variables, and also quite a noticeable region effect that is independent of individual characteristics.</p>
<p>More generally, Gelman uses multi-level models with a regional (and district, perhaps?) random effect as well as individual randomness, when he can get the data. My first ever analysis of voting behaviour was working with David Charnock when he was building on similar research he’d done with <a target="_blank" rel="nofollow" href="http://www.tandfonline.com/doi/abs/10.1080/10361149750922">Australian federal politics</a>.</p>
<p>We don’t have anywhere near enough data to look into the interaction effects, but we can look to see if there is a regional voting pattern that can’t be explained by individual characteristics.</p>
<p>The NZES data has too many missing values for Regional Council, but has near-complete data on which electorate each respondent is in. So I fit a multi-level model with an electorate-level random effect. There was a noticeable amount of variation by electorate – standard deviation of 0.33 – in the tendency to vote New Zealand First in a model with no individual explanatory variables. However, this went all the way down to zero when the individual variables were included.</p>
<blockquote>
<p>Interestingly, once we control for individual characteristics, we see no evidence of a residual region-based electorate effect in tendency to vote New Zealand First.</p>
</blockquote>
<p>As an aside, this <em>doesn’t</em> apply to the tendency to vote National, Labour or Green (ie there <em>is</em> a residual electorate effect for those three parties), but investigating why would take me too far afield.</p>
<p>Code for the multilevel models:</p>
<figure>
<pre><code data-lang="r"><span>#------------------regional multilevel model-----------------------
</span><span>table</span><span>(</span><span>nzes</span><span>$</span><span>kregcon</span><span>,</span><span> </span><span>useNA</span><span> </span><span>=</span><span> </span><span>&quot;always&quot;</span><span>)</span><span> </span><span># too many missing regions to use
</span><span>table</span><span>(</span><span>nzes</span><span>$</span><span>delect</span><span>,</span><span> </span><span>useNA</span><span> </span><span>=</span><span> </span><span>&quot;always&quot;</span><span>)</span><span> </span><span># only four missing electorates
</span><span>
</span><span># Note that glmer interprets &quot;weights&quot; as meaning number of trials
# for a binomial glm, so I'm ignoring them
</span><span>
</span><span>nzes</span><span> </span><span>&lt;-</span><span> </span><span>nzes</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>Electorate</span><span> </span><span>=</span><span> </span><span>ifelse</span><span>(</span><span>is.na</span><span>(</span><span>delect</span><span>),</span><span> </span><span>&quot;unknown&quot;</span><span>,</span><span> </span><span>delect</span><span>))</span><span>
</span><span>model_ml_null</span><span> </span><span>&lt;-</span><span> </span><span>glmer</span><span>(</span><span>PartyVoteNZFirst</span><span> </span><span>~</span><span> </span><span>(</span><span>1</span><span> </span><span>|</span><span> </span><span>Electorate</span><span>),</span><span> </span><span>data</span><span> </span><span>=</span><span> </span><span>nzes</span><span>,</span><span> </span><span>family</span><span> </span><span>=</span><span> </span><span>&quot;binomial&quot;</span><span>)</span><span>
</span><span>summary</span><span>(</span><span>model_ml_null</span><span>)</span><span> </span><span># standard deviation of 0.3297, quite noticeable
</span><span>
</span><span>model_ml</span><span> </span><span>&lt;-</span><span> </span><span>glmer</span><span>(</span><span>PartyVoteNZFirst</span><span> </span><span>~</span><span> </span><span>NotEuropean</span><span> </span><span>+</span><span> </span><span>Maori</span><span> </span><span>+</span><span> </span><span>HHIncome</span><span> </span><span>+</span><span> </span><span>OwnHouseOrFlat</span><span> </span><span>+</span><span> </span><span>HHMemberTradeUnion</span><span> </span><span>+</span><span> </span><span>HHMemberProfAssoc</span><span> </span><span>+</span><span> </span><span>NZBorn</span><span> </span><span>+</span><span> </span><span>Religion</span><span> </span><span>+</span><span> </span><span>Marital</span><span> </span><span>+</span><span> </span><span>HighestQual</span><span> </span><span>+</span><span> </span><span>IdentifyWorkingClass</span><span> </span><span>+</span><span> </span><span>WorkStatus</span><span> </span><span>+</span><span> </span><span>Student</span><span> </span><span>+</span><span> </span><span>SuperviseAnyone</span><span> </span><span>+</span><span> </span><span>City</span><span> </span><span>+</span><span> </span><span>Male</span><span> </span><span>+</span><span> </span><span>Age</span><span> </span><span>+</span><span> </span><span>(</span><span>1</span><span> </span><span>|</span><span> </span><span>Electorate</span><span>),</span><span> </span><span>data</span><span> </span><span>=</span><span> </span><span>nzes</span><span>,</span><span> </span><span>family</span><span> </span><span>=</span><span> </span><span>&quot;binomial&quot;</span><span>)</span><span>
</span><span>summary</span><span>(</span><span>model_ml</span><span>)</span><span>
</span><span># but &quot;electorate effect&quot; completely disappears to nearly zero when individual variables were included
# - at least for NZ First. For other parties there is a) problem converging and b)
</span><span>#</span><span> </span><span>there</span><span> </span><span>does</span><span> </span><span>seem</span><span> </span><span>to</span><span> </span><span>be</span><span> </span><span>a</span><span> </span><span>residual</span><span> </span><span>electorate</span><span> </span><span>effect</span></code></pre>
</figure>
<h1>Bootstrap with imputation and recalibration of survey weights each resample</h1>
<p>All the various models described above I regarded as basically exploratory. My “real” model and estimation process was always going to involve a bootstrap, and imputation of the missing values in a way that captures the randomness that would have been expected to be seen if we had a full set of data.</p>
<h2>Recalibrating survey weights</h2>
<p>The bootstrap involves a resample with replacement of a full sized sample from the sample we’ve got. One result of doing such a resample is that the weights will no longer correctly represent the proportions of people of various age, sex, ethnicity and voting status (did or didn’t vote) as the original weighted sample did. Fixing this requires creating a new set of weights for each bootstrap resample.</p>
<p>To calculate those weights, I need some more information on how the original weights were set. I haven’t been able to find (admittedly, after minimal effort) a definitive description of the weighting scheme of the NZES, but it <a target="_blank" rel="nofollow" href="http://www.nzes.org/data/NZES_weights.pdf">definitely includes</a> ethnicity, age, sex, and voting status; and I <em>thought</em> it included education. In fact, I particularly hope it included education, because a <a target="_blank" rel="nofollow" href="http://www.pewresearch.org/fact-tank/2016/11/09/why-2016-election-polls-missed-their-mark/">much-awaited review out today</a> on the polling performance leading up to the 2016 US Presidential election pinged the failure of many state polls to weight by education as a major contributor to error (more educated people respond more to surveys; and in that particular year, education was more strongly related to vote than had been the case in previous elections).</p>
<p>We can get a glimpse of the sampling scheme (both deliberate and incidental under and oversampling) by a regression of the published survey weight on our explanatory variables. Looking at this, I’m sure that the weights are taking education into account:</p>
<p><img width="100%" src="http://ellisp.github.io/img/0096-weights.svg"></p>
<p>This graphic nicely shows that Maori, voters, older people, and highly educated people were over-sampled and hence have low weights; whereas young people, men and students were under-sampled (probably through lower response rates rather than research design) and hence get slightly higher weights than usual.</p>
<figure>
<pre><code data-lang="r"><span>mod</span><span> </span><span>&lt;-</span><span> </span><span>lm</span><span>(</span><span>dwtfin</span><span> </span><span>~</span><span> </span><span>.</span><span> </span><span>-</span><span>dage</span><span>,</span><span> </span><span>data</span><span> </span><span>=</span><span> </span><span>nzes_subset</span><span>)</span><span>
</span><span>tidy</span><span>(</span><span>mod</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>term</span><span> </span><span>!=</span><span> </span><span>&quot;(Intercept)&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>term</span><span> </span><span>=</span><span> </span><span>fct_reorder</span><span>(</span><span>term</span><span>,</span><span> </span><span>estimate</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>estimate</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>term</span><span>))</span><span> </span><span>+</span><span> </span><span>geom_point</span><span>()</span><span> </span><span>+</span><span>
</span><span>labs</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>&quot;Impact on weight&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>ggtitle</span><span>(</span><span>&quot;Relative weights of different demographics, NZ Election Study 2014&quot;</span><span>,</span><span>
</span><span>&quot;Voters, M\u0101ori, women, older, and university qualified people are over-sampled and have low weights to compensate&quot;</span><span>)</span></code></pre>
</figure>
<p>Most commonly, in a bootstrap of a complex survey one would calculate a set of “replicate weights”, which have a one-off calibration to the correct population totals using Lumley’s <code>survey</code> package. However, I wanted to do the calibration for each resample separately, for convenience, because I was going to perform imputation separately for each resample, which adds quite a complication. Imputation depends on the observed data, hence should be performed for each resample to ensure the full randomness of the approach is taken into account.</p>
<p>To be ready for this recalibration, I needed the sum of weights for various marginal totals from the very first sample – these totals will reveal to me the implicit known, correct population totals the original researchers used in their weighting scheme. I won’t be doing it exactly the way they did, becuase I’m using my own simplified versions of age, qualifications, etc.; but it should be close enough for my purposes. Here’s the code that calculates those marginal totals for me, for later use in the bootstrap cycle:</p>
<figure>
<pre><code data-lang="r"><span>#--------------define population marginal totals that we will force weights to add up to-------------
</span><span>age_pop</span><span> </span><span>&lt;-</span><span> </span><span>nzes_subset</span><span> </span><span>%>%</span><span>
</span><span>group_by</span><span>(</span><span>Age</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>summarise</span><span>(</span><span>Freq</span><span> </span><span>=</span><span> </span><span>sum</span><span>(</span><span>dwtfin</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>!</span><span>is.na</span><span>(</span><span>Age</span><span>))</span><span>
</span><span>male_pop</span><span> </span><span>&lt;-</span><span> </span><span>nzes_subset</span><span> </span><span>%>%</span><span>
</span><span>group_by</span><span>(</span><span>Male</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>summarise</span><span>(</span><span>Freq</span><span> </span><span>=</span><span> </span><span>sum</span><span>(</span><span>dwtfin</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>!</span><span>is.na</span><span>(</span><span>Male</span><span>))</span><span>
</span><span>qual_pop</span><span> </span><span>&lt;-</span><span> </span><span>nzes_subset</span><span> </span><span>%>%</span><span>
</span><span>group_by</span><span>(</span><span>HighestQual</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>summarise</span><span>(</span><span>Freq</span><span> </span><span>=</span><span> </span><span>sum</span><span>(</span><span>dwtfin</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>!</span><span>is.na</span><span>(</span><span>HighestQual</span><span>))</span><span>
</span><span>maori_pop</span><span> </span><span>&lt;-</span><span> </span><span>nzes_subset</span><span> </span><span>%>%</span><span>
</span><span>group_by</span><span>(</span><span>Maori</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>summarise</span><span>(</span><span>Freq</span><span> </span><span>=</span><span> </span><span>sum</span><span>(</span><span>dwtfin</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>!</span><span>is.na</span><span>(</span><span>Maori</span><span>))</span><span>
</span><span>voted_pop</span><span> </span><span>&lt;-</span><span> </span><span>nzes_subset</span><span> </span><span>%>%</span><span>
</span><span>group_by</span><span>(</span><span>Voted</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>summarise</span><span>(</span><span>Freq</span><span> </span><span>=</span><span> </span><span>sum</span><span>(</span><span>dwtfin</span><span>))</span><span> </span></code></pre>
</figure>
<h2>Bootstrap results</h2>
<p>Now, I’m ready to run my loop of bootstrap resamples, to get a decent “best possible” estimate of the relationship between demographic explanatory variables and tendency to party-vote New Zealand First, compared to the rest of the New Zealand adult population. Each resample will:</p>
<ul>
<li>have a single set of imputed values calculated to replace its NAs, using the <code>mice</code> package that is usually used for multiple imputation even though I only want one imputation set per resample. <code>mice</code> adequately captures the existing observed structure in the data (much better than just imputing average values), as well as the observed randomness.</li>
<li>have its weights recalibrated to the correct marginal totals</li>
<li>have a model fit to it with that full set of data and new set of weights</li>
</ul>
<p>The end result has already been shown at the top of this post, but as this is all so long here it is reproduced:</p>
<p><img width="100%" src="http://ellisp.github.io/img/0096-svyglm-boot.svg"></p>
<p>The code that does this exercise took about 90 minutes to run on 7 cores on my laptop:</p>
<figure>
<pre><code data-lang="r"><span>#---------------define a bootstrap function-------------
# Function that does imputation, calibration of weights, and fits
# a survey GLM to the result:
</span><span>imp_cal_fit</span><span> </span><span>&lt;-</span><span> </span><span>function</span><span>(</span><span>x</span><span>,</span><span> </span><span>i</span><span>){</span><span>
</span><span># for dev:
</span><span> </span><span># x &lt;-nzes_subset; i &lt;-sample(1:nrow(x), nrow(x), replace = TRUE)
</span><span> </span><span># Resample the data
</span><span> </span><span>nzes_res</span><span> </span><span>&lt;-</span><span> </span><span>x</span><span>[</span><span>i</span><span>,</span><span> </span><span>]</span><span>
</span><span># Create a single complete, imputed version of the data
</span><span> </span><span>nzes_imp</span><span> </span><span>&lt;-</span><span> </span><span>complete</span><span>(</span><span>mice</span><span>(</span><span>nzes_res</span><span>,</span><span> </span><span>1</span><span>,</span><span> </span><span>printFlag</span><span> </span><span>=</span><span> </span><span>FALSE</span><span>))</span><span>
</span><span>attributes</span><span>(</span><span>nzes_imp</span><span>$</span><span>Religion</span><span>)</span><span>$</span><span>contrasts</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span>attributes</span><span>(</span><span>nzes_imp</span><span>$</span><span>WorkStatus</span><span>)</span><span>$</span><span>contrasts</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span>attributes</span><span>(</span><span>nzes_imp</span><span>$</span><span>HHIncome</span><span>)</span><span>$</span><span>contrasts</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span>attributes</span><span>(</span><span>nzes_imp</span><span>$</span><span>HighestQual</span><span>)</span><span>$</span><span>contrasts</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span>attributes</span><span>(</span><span>nzes_imp</span><span>$</span><span>Age</span><span>)</span><span>$</span><span>contrasts</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span># Set up as a survey object
</span><span> </span><span>nzes_svy</span><span> </span><span>&lt;-</span><span> </span><span>svydesign</span><span>(</span><span>~</span><span>1</span><span>,</span><span> </span><span>data</span><span> </span><span>=</span><span> </span><span>nzes_imp</span><span>,</span><span> </span><span>weights</span><span> </span><span>=</span><span> </span><span>~</span><span>dwtfin</span><span>)</span><span>
</span><span># force the marginal totals to match those from the original weighting
</span><span> </span><span>nzes_svy_cal</span><span> </span><span>&lt;-</span><span> </span><span>calibrate</span><span>(</span><span>nzes_svy</span><span>,</span><span> </span><span>list</span><span>(</span><span>~</span><span>Maori</span><span>,</span><span> </span><span>~</span><span>Age</span><span>,</span><span> </span><span>~</span><span>HighestQual</span><span>,</span><span> </span><span>~</span><span>Male</span><span>,</span><span> </span><span>~</span><span>Voted</span><span>),</span><span> </span><span>list</span><span>(</span><span>maori_pop</span><span>,</span><span> </span><span>age_pop</span><span>,</span><span> </span><span>qual_pop</span><span>,</span><span> </span><span>male_pop</span><span>,</span><span> </span><span>voted_pop</span><span>))</span><span>
</span><span># Fit the model
</span><span> </span><span>model_svy_cal</span><span> </span><span>&lt;-</span><span> </span><span>svyglm</span><span>(</span><span>form</span><span>,</span><span> </span><span>design</span><span> </span><span>=</span><span> </span><span>nzes_svy_cal</span><span>,</span><span> </span><span>family</span><span> </span><span>=</span><span> </span><span>&quot;quasibinomial&quot;</span><span>)</span><span>
</span><span># Return the results
</span><span> </span><span>return</span><span>(</span><span>coef</span><span>(</span><span>model_svy_cal</span><span>))</span><span>
</span><span>}</span><span>
</span><span># Set up a cluster for parallel computing
</span><span>cluster</span><span> </span><span>&lt;-</span><span> </span><span>makeCluster</span><span>(</span><span>7</span><span>)</span><span> </span><span># only any good if you have at least 7 processors
</span><span>registerDoParallel</span><span>(</span><span>cluster</span><span>)</span><span>
</span><span>clusterEvalQ</span><span>(</span><span>cluster</span><span>,</span><span> </span><span>{</span><span>
</span><span>library</span><span>(</span><span>survey</span><span>)</span><span>
</span><span>library</span><span>(</span><span>mice</span><span>)</span><span>
</span><span>})</span><span>
</span><span># to mimic the original deliberate over-sampling of Maori, I make the bootstrap
# resampling stratify itself on Maori too.
# Note that this takes about 90 minutes, even with the parallel computing. Each resample
# has to do the iterative imputation from scratch, then calibrate survey weights.
</span><span>system.time</span><span>({</span><span>
</span><span>nzes_booted</span><span> </span><span>&lt;-</span><span> </span><span>boot</span><span>(</span><span>nzes_subset</span><span>,</span><span> </span><span>imp_cal_fit</span><span>,</span><span> </span><span>R</span><span> </span><span>=</span><span> </span><span>999</span><span>,</span><span> </span><span>strata</span><span> </span><span>=</span><span> </span><span>nzes_subset</span><span>$</span><span>Maori</span><span>,</span><span> </span><span>parallel</span><span> </span><span>=</span><span> </span><span>&quot;snow&quot;</span><span>,</span><span> </span><span>cl</span><span> </span><span>=</span><span> </span><span>cluster</span><span>)</span><span>
</span><span>})</span><span>
</span><span># It's not common to do it this way. More usually you make a set of many replicate
# weights and calibrate them as a once-off job, then use those weights for any future
# estimation. But this doesn't conveniently fit in a workflow where we are doing a # new imputation for each bootstrap resample (which is usually admitted to be good # practice, but very rarely done, precisely because it's inconvenient for workflow).
</span><span>
</span><span># extract the 95% confidence intervals
</span><span>k</span><span> </span><span>&lt;-</span><span> </span><span>ncol</span><span>(</span><span>nzes_booted</span><span>$</span><span>t</span><span>)</span><span>
</span><span>results</span><span> </span><span>&lt;-</span><span> </span><span>as.data.frame</span><span>(</span><span>t</span><span>(</span><span>sapply</span><span>(</span><span>1</span><span>:</span><span>k</span><span>,</span><span> </span><span>function</span><span>(</span><span>i</span><span>){</span><span>boot.ci</span><span>(</span><span>nzes_booted</span><span>,</span><span> </span><span>type</span><span> </span><span>=</span><span> </span><span>&quot;perc&quot;</span><span>,</span><span> </span><span>index</span><span> </span><span>=</span><span> </span><span>i</span><span>)</span><span>$</span><span>percent</span><span>[</span><span>4</span><span>:</span><span>5</span><span>]})))</span><span>
</span><span>names</span><span>(</span><span>results</span><span>)</span><span> </span><span>&lt;-</span><span> </span><span>c</span><span>(</span><span>&quot;lower&quot;</span><span>,</span><span> </span><span>&quot;upper&quot;</span><span>)</span><span>
</span><span>results</span><span>$</span><span>term</span><span> </span><span>&lt;-</span><span> </span><span>names</span><span>(</span><span>coef</span><span>(</span><span>model_naive</span><span>))</span><span>
</span><span># draw graphic:
</span><span>results</span><span> </span><span>%>%</span><span>
</span><span>filter</span><span>(</span><span>term</span><span> </span><span>!=</span><span> </span><span>&quot;(Intercept)&quot;</span><span>)</span><span> </span><span>%>%</span><span>
</span><span>mutate</span><span>(</span><span>term</span><span> </span><span>=</span><span> </span><span>camel_to_english</span><span>(</span><span>term</span><span>),</span><span>
</span><span>mid</span><span> </span><span>=</span><span> </span><span>(</span><span>lower</span><span> </span><span>+</span><span> </span><span>upper</span><span> </span><span>)</span><span> </span><span>/</span><span> </span><span>2</span><span>,</span><span>
</span><span>term</span><span> </span><span>=</span><span> </span><span>fct_reorder</span><span>(</span><span>term</span><span>,</span><span> </span><span>mid</span><span>))</span><span> </span><span>%>%</span><span>
</span><span>ggplot</span><span>(</span><span>aes</span><span>(</span><span>y</span><span> </span><span>=</span><span> </span><span>term</span><span>))</span><span> </span><span>+</span><span>
</span><span>geom_vline</span><span>(</span><span>xintercept</span><span> </span><span>=</span><span> </span><span>0</span><span>)</span><span> </span><span>+</span><span>
</span><span>geom_segment</span><span>(</span><span>aes</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>lower</span><span>,</span><span> </span><span>xend</span><span> </span><span>=</span><span> </span><span>upper</span><span>,</span><span> </span><span>yend</span><span> </span><span>=</span><span> </span><span>term</span><span>),</span><span> </span><span>colour</span><span> </span><span>=</span><span> </span><span>parties_v</span><span>[</span><span>&quot;NZ First&quot;</span><span>],</span><span> </span><span>alpha</span><span> </span><span>=</span><span> </span><span>0.5</span><span>,</span><span> </span><span>size</span><span> </span><span>=</span><span> </span><span>3</span><span>)</span><span> </span><span>+</span><span>
</span><span>ggtitle</span><span>(</span><span>&quot;Party vote for New Zealand First in the 2014 election&quot;</span><span>,</span><span>
</span><span>&quot;Bootstrap, imputation, recalibrated survey weights, svyglm&quot;</span><span>)</span><span> </span><span>+</span><span>
</span><span>labs</span><span>(</span><span>caption</span><span> </span><span>=</span><span> </span><span>&quot;New Zealand Election Survey, analysed at https://ellispgithub.io&quot;</span><span>,</span><span>
</span><span>x</span><span> </span><span>=</span><span> </span><span>&quot;Impact on log odds of voting New Zealand First&quot;</span><span>,</span><span>
</span><span>y</span><span> </span><span>=</span><span> </span><span>str_wrap</span><span>(</span><span>&quot;Compared to 'no religion', 'age30-55', 'high household income', 'school qualification', 'working full time'&quot;</span><span>,</span><span> </span><span>50</span><span>))</span></code></pre>
</figure>
<h1>Multiple imputation without bootstrap</h1>
<p>The bootstrapped, multiply-imputed model gives a warm glow of thorough justificationness (if there is such a word) but is expensive in processor time. I was left with the impression that the standard model was fitting ok enough that even analytically calculated confidence intervals are probably good enough – after all, the bootstrap results looked pretty similar to the original non-bootstrap versions with no imputation at all.</p>
<p>To round out the analysis with fitting similar models to all parties, I skipped the bootstrap part and just used the original sample, but with five different sets of imputed values using the core functionality provided by <code>mice</code>. I didn’t bother to re-calibrate the weights for each set of imputed values because I’m fairly confident changes from the original weights would have been minimal. This means I have five different complete (no NA values) datasets, each of my four models is fit to each dataset, and the five results for each party are pooled using the Barnard-Rubin method.</p>
<p>Here’s the end results (click image to enlarge):</p>
<p><a target="_blank" rel="nofollow" href="http://ellisp.github.io/img/0096-all-parties.svg"><br>
<img width="100%" src="http://ellisp.github.io/img/0096-all-parties.svg"><br>
</a></p>
<p>… and the code to do it is pretty simple:</p>
<figure>
<pre><code data-lang="r"><span>#============multiple imputation all four main parties===============
# see http://r-survey.r-forge.r-project.org/survey/svymi.html for an alternative way to do this
# Also see https://stats.stackexchange.com/questions/149053/questions-on-multiple-imputation-with-mice-for-a-multigroup-sem-analysis-inclu
</span><span>
</span><span>nzes_mi</span><span> </span><span>&lt;-</span><span> </span><span>mice</span><span>(</span><span>nzes_subset</span><span>)</span><span>
</span><span>attributes</span><span>(</span><span>nzes_mi</span><span>$</span><span>data</span><span>$</span><span>Age</span><span>)</span><span>$</span><span>contrasts</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span>attributes</span><span>(</span><span>nzes_mi</span><span>$</span><span>data</span><span>$</span><span>Religion</span><span>)</span><span>$</span><span>contrasts</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span>attributes</span><span>(</span><span>nzes_mi</span><span>$</span><span>data</span><span>$</span><span>HighestQual</span><span>)</span><span>$</span><span>contrasts</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span>attributes</span><span>(</span><span>nzes_mi</span><span>$</span><span>data</span><span>$</span><span>WorkStatus</span><span>)</span><span>$</span><span>contrasts</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span>attributes</span><span>(</span><span>nzes_mi</span><span>$</span><span>data</span><span>$</span><span>HHIncome</span><span>)</span><span>$</span><span>contrasts</span><span> </span><span>&lt;-</span><span> </span><span>NULL</span><span>
</span><span>responses</span><span> </span><span>&lt;-</span><span> </span><span>paste0</span><span>(</span><span>&quot;PartyVote&quot;</span><span>,</span><span> </span><span>c</span><span>(</span><span>&quot;National&quot;</span><span>,</span><span> </span><span>&quot;Labour&quot;</span><span>,</span><span> </span><span>&quot;NZFirst&quot;</span><span>,</span><span> </span><span>&quot;Green&quot;</span><span>))</span><span>
</span><span>colours</span><span> </span><span>&lt;-</span><span> </span><span>parties_v</span><span>[</span><span>c</span><span>(</span><span>&quot;National&quot;</span><span>,</span><span> </span><span>&quot;Labour&quot;</span><span>,</span><span> </span><span>&quot;NZ First&quot;</span><span>,</span><span> </span><span>&quot;Green&quot;</span><span>)]</span><span>
</span><span>p</span><span> </span><span>&lt;-</span><span> </span><span>list</span><span>()</span><span>
</span><span>form_gen</span><span> </span><span>&lt;-</span><span> </span><span>&quot;XXX ~ NotEuropean + Maori + HHIncome + OwnHouseOrFlat +
HHMemberTradeUnion + HHMemberProfAssoc + NZBorn + Religion +
Marital + HighestQual + IdentifyWorkingClass + WorkStatus +
Student + SuperviseAnyone + City + Male + Age&quot;</span><span>
</span><span>for</span><span>(</span><span>i</span><span> </span><span>in</span><span> </span><span>1</span><span>:</span><span>length</span><span>(</span><span>responses</span><span>)){</span><span>
</span><span>form2</span><span> </span><span>&lt;-</span><span> </span><span>gsub</span><span>(</span><span>&quot;XXX&quot;</span><span>,</span><span> </span><span>responses</span><span>[</span><span>i</span><span>],</span><span> </span><span>form_gen</span><span>)</span><span>
</span><span>model_mi</span><span> </span><span>&lt;-</span><span> </span><span>with</span><span>(</span><span>data</span><span> </span><span>=</span><span> </span><span>nzes_mi</span><span>,</span><span>
</span><span>glm</span><span>(</span><span>as.formula</span><span>(</span><span>form2</span><span>),</span><span> </span><span>family</span><span> </span><span>=</span><span> </span><span>&quot;quasibinomial&quot;</span><span>,</span><span> </span><span>weights</span><span> </span><span>=</span><span> </span><span>dwtfin</span><span>))</span><span>
</span><span>p</span><span>[[</span><span>i</span><span>]]</span><span> </span><span>&lt;-</span><span> </span><span>sum_plot</span><span>(</span><span>model_mi</span><span>,</span><span> </span><span>title</span><span> </span><span>=</span><span> </span><span>paste</span><span>(</span><span>gsub</span><span>(</span><span>&quot;PartyVote&quot;</span><span>,</span><span> </span><span>&quot;Party vote for &quot;</span><span>,</span><span> </span><span>responses</span><span>[</span><span>i</span><span>]),</span><span>
</span><span>&quot;compared to rest of population&quot;</span><span>),</span><span>
</span><span>colour</span><span> </span><span>=</span><span> </span><span>colours</span><span>[</span><span>i</span><span>],</span><span>
</span><span>type</span><span> </span><span>=</span><span> </span><span>&quot;pool&quot;</span><span>,</span><span>
</span><span>party</span><span> </span><span>=</span><span> </span><span>gsub</span><span>(</span><span>&quot;PartyVote&quot;</span><span>,</span><span> </span><span>&quot;&quot;</span><span>,</span><span> </span><span>responses</span><span>[</span><span>i</span><span>]))</span><span>
</span><span>}</span><span>
</span><span>grid.arrange</span><span>(</span><span>p</span><span>[[</span><span>1</span><span>]],</span><span> </span><span>p</span><span>[[</span><span>2</span><span>]],</span><span> </span><span>p</span><span>[[</span><span>3</span><span>]],</span><span> </span><span>p</span><span>[[</span><span>4</span><span>]])</span></code></pre>
</figure>
<h1>Discussion</h1>
<p>While I do a chunk of analysis on voting behaviour (eg see my <a target="_blank" rel="nofollow" href="http://ellisp.github.io/elections/elections.html">election forecasts</a>), I’m the wrong person for all sorts of reason to give actual commentary on New Zealand politics. So I’m going to make only the most obvious comments:</p>
<ul>
<li>To answer the original question at the beginning of this post, New Zealand First voters were more likely than the general population to be older, born in New Zealand, identifying as working class and male.</li>
<li>No surprise that people such as home owners, higher income people, European heritage, those who don’t identify as working class, don’t have a trade union member in the household voted for the centre-right National party</li>
<li>Perhaps more surprising that being born in New Zealand is estimated to make people <em>less</em> likely to vote for National after controlling for education, ethnicity, income etc (in fact this one is surprising enough for me I’d like to see it replicated with fresh data before making too much of it)</li>
<li>In addition to the variables that would be expected to lead to Labour vote (union membership, lower income, less qualification), being <em>Maori</em> and <em>female</em> are both estimated to point in that direction</li>
<li>The characteristics of Green voters are an interesting but not that surprising list – one or more of university qualifications, beign a student, part time, union member in the household, living in a city, not identifying as Christian, not of European ethnicity.</li>
</ul>
<p>So there we go. Relationship of individual socio-economi status on voting behaviour in the 2014 New Zealand General Election. Rather sadly, I don’t see much of this amazing dataset <a target="_blank" rel="nofollow" href="https://scholar.google.co.nz/scholar?start=0&q=%22new+zealand+election+study%22&hl=en&as_sdt=0,5&as_ylo=2016">in the academic literature</a>. Maybe I’m looking in the wrong spot.</p>
<p>Corrections and other comments welcomed. Beware – all the above code isn’t checked by anyone except me.</p>
<p>Big thanks to the Electoral Commission for funding the NZES, the NZES researchers themselves, and (as always) everyone behind the R core project and the 16 R packages listed near the beginning of this post.</p>
<div>
<pre><code>> thankr::shoulders() %>%
+    mutate(maintainer = gsub(&quot;&lt;.*>&quot;, &quot;&quot;, maintainer)) %>%
+    group_by(maintainer) %>%
+    summarise(no_packages = sum(no_packages)) %>%
+    arrange(desc(no_packages)) %>%
+    as.data.frame()
             maintainer no_packages
1       Hadley Wickham           18
2          R Core Team           12
3         Brian Ripley            4
4        Kirill Müller            3
5         Rich Calaway            3
6           Yixuan Qiu            3
7    Dirk Eddelbuettel            2
8       Jennifer Bryan            2
9      Martin Maechler            2
10     &quot;Thomas Lumley&quot;            1
11       Achim Zeileis            1
12    Adelchi Azzalini            1
13     Baptiste Auguie            1
14          Ben Bolker            1
15   Charlotte Wickham            1
16      David Robinson            1
17     Deepayan Sarkar            1
18  Duncan Temple Lang            1
19        Gábor Csárdi            1
20        James Hester            1
21         Jelmer Ypma            1
22         Jeroen Ooms            1
23          Jim Hester            1
24          JJ Allaire            1
25           Joe Cheng            1
26 Katharine M. Mullen            1
27        Luke Tierney            1
28    Marek Gagolewski            1
29         Peter Ellis            1
30              R-core            1
31          Simon Wood            1
32     Stef van Buuren            1
33 Stefan Milton Bache            1
34    Terry M Therneau            1
35       Trevor Hastie            1
36       Vitalie Spinu            1
37     William Revelle            1
38       Winston Chang            1
39           Yihui Xie            1
</code></pre>
</div>
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/DJ11G22natY">