<div></div><div><div><h1>A R named list is not a python dictionary<a href="http://optimallog.blogspot.com/search/label/R#A-R-named-list-is-not-a-python-dictionary">¶</a></h1><p>In the previous post, mpiktas posted a comment that included this question &quot;Isn't Python dictionary a named list in R?&quot;. The short answer is no, but the reason why is worth some more explanation.</p> </div></div><div><div></div><div><div><p>At first sight, a named list does look a little bit as a python dictionary, it allows to retrieve values inside a list using names (i.e. strings) instead of by numerical index. The code below is also a good example on how it is possible to mix R and python code inside a Jupyter notebook</p> </div></div></div><div><div><div>In [1]:</div><div> <div><div><pre><span></span><span>%</span><span>load_ext</span> rpy2.ipython<br></pre></div> </div></div></div> </div><div><div><div>In [2]:</div><div> <div><div><pre><span></span><span>%</span><span>R</span> RL &lt;- list('one'=1, 'two'=2, 'three'=3) ; RL[['one']] <br></pre></div> </div></div></div> <div><div> <div><div>Out[2]:</div> <div><pre>array([ 1.])</pre></div> </div> </div></div> </div><div><div><div>In [3]:</div><div> <div><div><pre><span></span><span>PD</span> <span>=</span> <span>{</span><span>'one'</span><span>:</span> <span>1</span><span>,</span> <span>'two'</span><span>:</span><span>2</span><span>,</span> <span>'three'</span><span>:</span> <span>3</span><span>}</span> <span>;</span> <span>PD</span><span>[</span><span>'one'</span><span>]</span><br></pre></div> </div></div></div> <div><div> <div><div>Out[3]:</div> <div><pre>1</pre></div> </div> </div></div> </div><div><div></div><div><div><p>But a dictionary in python can do more, the keys can be any immutable type, so you can use numbers and tuples for keys, even in one dictionary</p> </div></div></div><div><div><div>In [4]:</div><div> <div><div><pre><span></span><span>PD</span> <span>=</span> <span>{</span><span>'one'</span><span>:</span> <span>1</span><span>,</span> <span>2</span><span>:</span><span>2</span><span>,</span> <span>(</span><span>3</span><span>,</span><span>'three'</span><span>)</span> <span>:</span>  <span>3</span> <span>}</span> <span>;</span> <span>PD</span><span>[(</span><span>3</span><span>,</span><span>'three'</span><span>)]</span><br></pre></div> </div></div></div> <div><div> <div><div>Out[4]:</div> <div><pre>3</pre></div> </div> </div></div> </div><div><div></div><div><div><p>And you can add by name into an existing dictionary.</p> </div></div></div><div><div><div>In [5]:</div><div> <div><div><pre><span></span><span>PD</span><span>[</span><span>'four'</span><span>]</span> <span>=</span> <span>4</span> <span>;</span> <span>PD</span><br></pre></div> </div></div></div> <div><div> <div><div>Out[5]:</div> <div><pre>{'one': 1, 2: 2, (3, 'three'): 3, 'four': 4}</pre></div> </div> </div></div> </div><div><div></div><div><div><p>You need a two step approach in R, or at least I don't know a single step approach to add a named element</p> </div></div></div><div><div><div>In [6]:</div><div> <div><div><pre><span></span><span>%</span><span>R</span> RL[4] &lt;- 4 ; names(RL)[4] &lt;- 'four' ; names(RL)<br></pre></div> </div></div></div> <div><div> <div><div>Out[6]:</div> <div><pre>array(['one', 'two', 'three', 'four'], <br>      dtype='&lt;U5')</pre></div> </div> </div></div> </div><div><div></div><div><div><p>But there is a more fundamental difference, the python dictionary uses a hash table implementation so that access to a key is independent of the size of the dictionary, while the named list in R seems to use a sorted table of names. This makes access by names (very) slow compared to access by index in R. Here is an example, that also shows how to mix intelligently R and python.</p> </div></div></div><div><div></div><div><div><p>The code builds (large) equivalent structures, and a random subset is defined using R (easier in R) and passed to the python code. Then we measure the access time using different approaches. In python, we use both a list and a dictionary.</p> </div></div></div><div><div><div>In [7]:</div><div> <div><div><pre><span></span><span>import</span> <span>time</span><br><span>import</span> <span>pandas</span><br><span>nBits</span> <span>=</span> <span>[</span><span>18</span><span>,</span><span>19</span><span>,</span><span>20</span><span>,</span><span>21</span><span>]</span><br><span>columns</span> <span>=</span> <span>[</span><span>&quot;Size&quot;</span><span>,</span> <span>&quot;Python dict by key&quot;</span><span>,</span> <span>&quot;Python list by index&quot;</span><span>,</span> <span>&quot;R list by index&quot;</span><span>,</span> <span>&quot;R list by key&quot;</span><span>]</span><br><span>results</span> <span>=</span> <span>pandas</span><span>.</span><span>DataFrame</span><span>(</span><span>columns</span><span>=</span><span>columns</span><span>,</span> <span>index</span><span>=</span><span>nBits</span><span>)</span><br><span>nSamples</span> <span>=</span> <span>1000</span><br><span>for</span> <span>nBit</span> <span>in</span> <span>[</span><span>18</span><span>,</span><span>19</span><span>,</span><span>20</span><span>,</span><span>21</span><span>]:</span><br><span>n</span> <span>=</span> <span>(</span><span>1</span><span>&lt;&lt;</span><span>nBit</span><span>)</span><br><span>results</span><span>[</span><span>&quot;Size&quot;</span><span>][</span><span>nBit</span><span>]</span> <span>=</span> <span>n</span><br><span>%</span><span>R</span> -i nSamples -o samples -i n   samples &lt;- sample(1:(n-1), nSamples, replace = FALSE) <br><span>%</span><span>R</span> -o keys                       keys &lt;- as.character(samples)<br><span>keys</span> <span>=</span> <span>list</span><span>(</span><span>keys</span><span>)</span><br><span>PL</span> <span>=</span> <span>list</span><span>(</span><span>range</span><span>(</span><span>n</span><span>))</span><br><span>start</span> <span>=</span> <span>time</span><span>.</span><span>perf_counter</span><span>()</span><br><span>S</span> <span>=</span> <span>sum</span><span>([</span><span>PL</span><span>[</span><span>i</span><span>]</span> <span>for</span> <span>i</span> <span>in</span> <span>samples</span> <span>])</span><br><span>stop</span> <span>=</span> <span>time</span><span>.</span><span>perf_counter</span><span>()</span><br><span>results</span><span>[</span><span>&quot;Python list by index&quot;</span><span>][</span><span>nBit</span><span>]</span> <span>=</span> <span>(</span><span>stop</span> <span>-</span> <span>start</span><span>)</span> <span>/</span> <span>nSamples</span> <span>*</span> <span>1e6</span><br><span>PD</span> <span>=</span> <span>{</span> <span>str</span><span>(</span><span>x</span><span>):</span><span>x</span> <span>for</span> <span>x</span> <span>in</span> <span>PL</span><span>}</span><br><span>start</span> <span>=</span> <span>time</span><span>.</span><span>perf_counter</span><span>()</span><br><span>S</span> <span>=</span> <span>sum</span><span>([</span><span>PD</span><span>[</span><span>k</span><span>]</span> <span>for</span> <span>k</span> <span>in</span> <span>keys</span><span>])</span><br><span>stop</span> <span>=</span> <span>time</span><span>.</span><span>perf_counter</span><span>()</span><br><span>results</span><span>[</span><span>&quot;Python dict by key&quot;</span><span>][</span><span>nBit</span><span>]</span> <span>=</span> <span>(</span><span>stop</span> <span>-</span> <span>start</span><span>)</span> <span>/</span> <span>nSamples</span> <span>*</span> <span>1e6</span><br><span>%</span><span>R</span> -i n -o RL RL &lt;- as.list(1:n)<br><span>%</span><span>R</span>            start &lt;- Sys.time() ; S &lt;- 0 ; for (i in samples) { S &lt;- S + RL[[i]] } ; stop &lt;- Sys.time()<br><span>%</span><span>R</span> -o idelay  idelay &lt;- stop - start<br><span>%</span><span>R</span>            names(RL) &lt;- as.character(RL)<br><span>%</span><span>R</span>            start &lt;- Sys.time() ; S &lt;- 0 ; for (k in keys ) { S &lt;- S + RL[[k]] } ; stop &lt;- Sys.time()<br><span>%</span><span>R</span> -o kdelay  kdelay &lt;- stop - start  <br><span>results</span><span>[</span><span>&quot;R list by index&quot;</span><span>][</span><span>nBit</span><span>]</span> <span>=</span> <span>idelay</span><span>[</span><span>0</span><span>]</span> <span>/</span> <span>nSamples</span> <span>*</span> <span>1e6</span><br><span>results</span><span>[</span><span>&quot;R list by key&quot;</span><span>][</span><span>nBit</span><span>]</span> <span>=</span> <span>kdelay</span><span>[</span><span>0</span><span>]</span> <span>/</span> <span>nSamples</span> <span>*</span> <span>1e6</span><br><span>results</span><br></pre></div> </div></div></div> <div><div> <div><div>Out[7]:</div> <div><div><table border="1"><tr> Size Python dict by key Python list by index R list by index R list by key </tr><tr>18 <td>262144</td> <td>0.447617</td> <td>0.257533</td> <td>0.998974</td> <td>1298</td> </tr><tr>19 <td>524288</td> <td>0.659162</td> <td>0.341844</td> <td>2.00105</td> <td>3028.03</td> </tr><tr>20 <td>1048576</td> <td>0.758547</td> <td>0.357429</td> <td>8.03399</td> <td>6471.99</td> </tr><tr>21 <td>2097152</td> <td>0.857677</td> <td>0.415936</td> <td>1.02687</td> <td>13096</td> </tr></table></div></div> </div> </div></div> </div><div><div></div><div><div><p>The results have sometimes outliers, but the main result is that access by key in R is both slow and increasing with the list size.</p> </div></div></div><div><div></div><div><div><p>There is more to say, in particular there are data structures in R that are based on hash tables, but they remain less flexible than a python dictionary. I am not the first person making this type of analysis, you can check e.g.</p><ul><li><a href="http://jeffreyhorner.tumblr.com/post/114524915928/hash-table-performance-in-r-part-i">http://jeffreyhorner.tumblr.com/post/114524915928/hash-table-performance-in-r-part-i</a> and sequels</li><li><a href="https://opendatagroup.wordpress.com/2009/07/26/hash-package-for-r/">https://opendatagroup.wordpress.com/2009/07/26/hash-package-for-r/</a> introducing <a href="https://cran.r-project.org/web/packages/hash/">https://cran.r-project.org/web/packages/hash/</a> </li></ul></div></div></div>