<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/logopt-a-journey-in-r-python-finance-and-open-source-2017-05-02-213500/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://optimallog.blogspot.com/2017/05/blog-post.html"> logopt: a journey in R, Python, finance and open source</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>

<div>
<div></div>
<div>
<div>
<h1>A R named list is not a python dictionary<a target="_blank" rel="nofollow" href="http://optimallog.blogspot.com/2017/05/blog-post.html#A-R-named-list-is-not-a-python-dictionary">¶</a></h1>
<p>In the previous post, mpiktas posted a comment that included this question “Isn’t Python dictionary a named list in R?”. The short answer is no, but the reason why is worth some more explanation.</p>
</div>
</div>
</div>
<div>
<div></div>
<div>
<div>
<p>At first sight, a named list does look a little bit as a python dictionary, it allows to retrieve values inside a list using names (i.e. strings) instead of by numerical index. The code below is also a good example on how it is possible to mix R and python code inside a Jupyter notebook</p>
</div>
</div>
</div>
<div>
<div>
<div>In [1]:</div>
<div>
<div>
<div>
<pre><span></span><span>%</span><span>load_ext</span> rpy2.ipython<br></pre>
</div></div>
</div>
</div></div>
<div>
<div>
<div>In [2]:</div>
<div>
<div>
<div>
<pre><span></span><span>%</span><span>R</span> RL &lt;- list('one'=1, 'two'=2, 'three'=3) ; RL[['one']] <br></pre>
</div></div>
</div>
</div>
<div>
<div>
<div>
<div>Out[2]:</div>
<div>
<pre>array([ 1.])</pre>
</div></div>
</div>
</div></div>
<div>
<div>
<div>In [3]:</div>
<div>
<div>
<div>
<pre><span></span><span>PD</span> <span>=</span> <span>{</span><span>'one'</span><span>:</span> <span>1</span><span>,</span> <span>'two'</span><span>:</span><span>2</span><span>,</span> <span>'three'</span><span>:</span> <span>3</span><span>}</span> <span>;</span> <span>PD</span><span>[</span><span>'one'</span><span>]</span><br></pre>
</div></div>
</div>
</div>
<div>
<div>
<div>
<div>Out[3]:</div>
<div>
<pre>1</pre>
</div></div>
</div>
</div></div>
<div>
<div></div>
<div>
<div>
<p>But a dictionary in python can do more, the keys can be any immutable type, so you can use numbers and tuples for keys, even in one dictionary</p>
</div>
</div>
</div>
<div>
<div>
<div>In [4]:</div>
<div>
<div>
<div>
<pre><span></span><span>PD</span> <span>=</span> <span>{</span><span>'one'</span><span>:</span> <span>1</span><span>,</span> <span>2</span><span>:</span><span>2</span><span>,</span> <span>(</span><span>3</span><span>,</span><span>'three'</span><span>)</span> <span>:</span>  <span>3</span> <span>}</span> <span>;</span> <span>PD</span><span>[(</span><span>3</span><span>,</span><span>'three'</span><span>)]</span><br></pre>
</div></div>
</div>
</div>
<div>
<div>
<div>
<div>Out[4]:</div>
<div>
<pre>3</pre>
</div></div>
</div>
</div></div>
<div>
<div></div>
<div>
<div>
<p>And you can add by name into an existing dictionary.</p>
</div>
</div>
</div>
<div>
<div>
<div>In [5]:</div>
<div>
<div>
<div>
<pre><span></span><span>PD</span><span>[</span><span>'four'</span><span>]</span> <span>=</span> <span>4</span> <span>;</span> <span>PD</span><br></pre>
</div></div>
</div>
</div>
<div>
<div>
<div>
<div>Out[5]:</div>
<div>
<pre>{'one': 1, 2: 2, (3, 'three'): 3, 'four': 4}</pre>
</div></div>
</div>
</div></div>
<div>
<div></div>
<div>
<div>
<p>You need a two step approach in R, or at least I don’t know a single step approach to add a named element</p>
</div>
</div>
</div>
<div>
<div>
<div>In [6]:</div>
<div>
<div>
<div>
<pre><span></span><span>%</span><span>R</span> RL[4] &lt;- 4 ; names(RL)[4] &lt;- 'four' ; names(RL)<br></pre>
</div></div>
</div>
</div>
<div>
<div>
<div>
<div>Out[6]:</div>
<div>
<pre>array(['one', 'two', 'three', 'four'], <br>      dtype='&lt;U5')</pre>
</div></div>
</div>
</div></div>
<div>
<div></div>
<div>
<div>
<p>But there is a more fundamental difference, the python dictionary uses a hash table implementation so that access to a key is independent of the size of the dictionary, while the named list in R seems to use a sorted table of names. This makes access by names (very) slow compared to access by index in R. Here is an example, that also shows how to mix intelligently R and python.</p>
</div>
</div>
</div>
<div>
<div></div>
<div>
<div>
<p>The code builds (large) equivalent structures, and a random subset is defined using R (easier in R) and passed to the python code. Then we measure the access time using different approaches. In python, we use both a list and a dictionary.</p>
</div>
</div>
</div>
<div>
<div>
<div>In [7]:</div>
<div>
<div>
<div>
<pre><span></span><span>import</span> <span>time</span><br><span>import</span> <span>pandas</span><br><span>nBits</span> <span>=</span> <span>[</span><span>18</span><span>,</span><span>19</span><span>,</span><span>20</span><span>,</span><span>21</span><span>]</span><br><span>columns</span> <span>=</span> <span>[</span><span>&quot;Size&quot;</span><span>,</span> <span>&quot;Python dict by key&quot;</span><span>,</span> <span>&quot;Python list by index&quot;</span><span>,</span> <span>&quot;R list by index&quot;</span><span>,</span> <span>&quot;R list by key&quot;</span><span>]</span><br><span>results</span> <span>=</span> <span>pandas</span><span>.</span><span>DataFrame</span><span>(</span><span>columns</span><span>=</span><span>columns</span><span>,</span> <span>index</span><span>=</span><span>nBits</span><span>)</span><br><span>nSamples</span> <span>=</span> <span>1000</span><br><span>for</span> <span>nBit</span> <span>in</span> <span>[</span><span>18</span><span>,</span><span>19</span><span>,</span><span>20</span><span>,</span><span>21</span><span>]:</span><br>    <span>n</span> <span>=</span> <span>(</span><span>1</span><span>&lt;&lt;</span><span>nBit</span><span>)</span><br>    <span>results</span><span>[</span><span>&quot;Size&quot;</span><span>][</span><span>nBit</span><span>]</span> <span>=</span> <span>n</span><br>    <span>%</span><span>R</span> -i nSamples -o samples -i n   samples &lt;- sample(1:(n-1), nSamples, replace = FALSE) <br>    <span>%</span><span>R</span> -o keys                       keys &lt;- as.character(samples)<br>    <span>keys</span> <span>=</span> <span>list</span><span>(</span><span>keys</span><span>)</span><br>    <span>PL</span> <span>=</span> <span>list</span><span>(</span><span>range</span><span>(</span><span>n</span><span>))</span><br>    <span>start</span> <span>=</span> <span>time</span><span>.</span><span>perf_counter</span><span>()</span><br>    <span>S</span> <span>=</span> <span>sum</span><span>([</span><span>PL</span><span>[</span><span>i</span><span>]</span> <span>for</span> <span>i</span> <span>in</span> <span>samples</span> <span>])</span><br>    <span>stop</span> <span>=</span> <span>time</span><span>.</span><span>perf_counter</span><span>()</span><br>    <span>results</span><span>[</span><span>&quot;Python list by index&quot;</span><span>][</span><span>nBit</span><span>]</span> <span>=</span> <span>(</span><span>stop</span> <span>-</span> <span>start</span><span>)</span> <span>/</span> <span>nSamples</span> <span>*</span> <span>1e6</span><br>    <span>PD</span> <span>=</span> <span>{</span> <span>str</span><span>(</span><span>x</span><span>):</span><span>x</span> <span>for</span> <span>x</span> <span>in</span> <span>PL</span><span>}</span><br>    <span>start</span> <span>=</span> <span>time</span><span>.</span><span>perf_counter</span><span>()</span><br>    <span>S</span> <span>=</span> <span>sum</span><span>([</span><span>PD</span><span>[</span><span>k</span><span>]</span> <span>for</span> <span>k</span> <span>in</span> <span>keys</span><span>])</span><br>    <span>stop</span> <span>=</span> <span>time</span><span>.</span><span>perf_counter</span><span>()</span><br>    <span>results</span><span>[</span><span>&quot;Python dict by key&quot;</span><span>][</span><span>nBit</span><span>]</span> <span>=</span> <span>(</span><span>stop</span> <span>-</span> <span>start</span><span>)</span> <span>/</span> <span>nSamples</span> <span>*</span> <span>1e6</span><br>    <span>%</span><span>R</span> -i n -o RL RL &lt;- as.list(1:n)<br>    <span>%</span><span>R</span>            start &lt;- Sys.time() ; S &lt;- 0 ; for (i in samples) { S &lt;- S + RL[[i]] } ; stop &lt;- Sys.time()<br>    <span>%</span><span>R</span> -o idelay  idelay &lt;- stop - start<br>    <span>%</span><span>R</span>            names(RL) &lt;- as.character(RL)<br>    <span>%</span><span>R</span>            start &lt;- Sys.time() ; S &lt;- 0 ; for (k in keys ) { S &lt;- S + RL[[k]] } ; stop &lt;- Sys.time()<br>    <span>%</span><span>R</span> -o kdelay  kdelay &lt;- stop - start  <br>    <span>results</span><span>[</span><span>&quot;R list by index&quot;</span><span>][</span><span>nBit</span><span>]</span> <span>=</span> <span>idelay</span><span>[</span><span>0</span><span>]</span> <span>/</span> <span>nSamples</span> <span>*</span> <span>1e6</span><br>    <span>results</span><span>[</span><span>&quot;R list by key&quot;</span><span>][</span><span>nBit</span><span>]</span> <span>=</span> <span>kdelay</span><span>[</span><span>0</span><span>]</span> <span>/</span> <span>nSamples</span> <span>*</span> <span>1e6</span><br><span>results</span><br></pre>
</div></div>
</div>
</div>
<div>
<div>
<div>
<div>Out[7]:</div>
<div>
<div>
<table border="1">
<thead>
<tr>
<th></th>
<th>Size</th>
<th>Python dict by key</th>
<th>Python list by index</th>
<th>R list by index</th>
<th>R list by key</th>
</tr>
</thead>
<tbody>
<tr>
<th>18</th>
<td>262144</td>
<td>0.447617</td>
<td>0.257533</td>
<td>0.998974</td>
<td>1298</td>
</tr>
<tr>
<th>19</th>
<td>524288</td>
<td>0.659162</td>
<td>0.341844</td>
<td>2.00105</td>
<td>3028.03</td>
</tr>
<tr>
<th>20</th>
<td>1048576</td>
<td>0.758547</td>
<td>0.357429</td>
<td>8.03399</td>
<td>6471.99</td>
</tr>
<tr>
<th>21</th>
<td>2097152</td>
<td>0.857677</td>
<td>0.415936</td>
<td>1.02687</td>
<td>13096</td>
</tr>
</tbody>
</table>
</div>
</div></div>
</div>
</div></div>
<div>
<div></div>
<div>
<div>
<p>The results have sometimes outliers, but the main result is that access by key in R is both slow and increasing with the list size.</p>
</div>
</div>
</div>
<div>
<div></div>
<div>
<div>
<p>There is more to say, in particular there <em>are</em> data structures in R that are based on hash tables, but they remain less flexible than a python dictionary. I am not the first person making this type of analysis, you can check e.g.</p>
<ul>
<li><a target="_blank" rel="nofollow" href="http://jeffreyhorner.tumblr.com/post/114524915928/hash-table-performance-in-r-part-i">http://jeffreyhorner.tumblr.com/post/114524915928/hash-table-performance-in-r-part-i</a> and sequels</li>
<li><a target="_blank" rel="nofollow" href="https://opendatagroup.wordpress.com/2009/07/26/hash-package-for-r/">https://opendatagroup.wordpress.com/2009/07/26/hash-package-for-r/</a> introducing <a target="_blank" rel="nofollow" href="https://cran.r-project.org/web/packages/hash/">https://cran.r-project.org/web/packages/hash/</a> </li>
</ul></div>
</div>
</div>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://optimallog.blogspot.com/2017/05/blog-post.html"> logopt: a journey in R, Python, finance and open source</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/mA_xF6aXDhg"></a></div>