<p><div><div><div><div data-show-faces="false" data-height="21" data-width="100" data-layout="button_count" data-send="true" data-href="https://www.r-bloggers.com/stepwise-bayesian-optimization-with-mlrmbo/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="https://mlr-org.github.io/Stepwise-Bayesian-Optimization-with-mlrMBO/"> mlr-org</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>With the release of the new version of <a rel="nofollow" href="https://mlr-org.github.io/mlrMBO/" target="_blank">mlrMBO</a> we added some minor fixes and added a practical feature called <em><a rel="nofollow" href="https://mlr-org.github.io/mlrMBO/articles/supplementary/human_in_the_loop_MBO.html" target="_blank">Human-in-the-loop MBO</a></em>.<br>
It enables you to sequentially</p>
<ul>
<li>visualize the state of the surrogate model,</li>
<li>obtain the suggested parameter configuration for the next iteration and</li>
<li>update the surrogate model with arbitrary evaluations.</li>
</ul>
<p>In the following we will demonstrate this feature on a simple example.<br>
<span></span></p>
<p>First we need an objective function we want to optimize.<br>
For this post a simple function will suffice but note that this function could also be an external process as in this mode <strong>mlrMBO</strong> does not need to access the objective function as you will only have to pass the results of the function to <strong>mlrMBO</strong>.</p>
<figure>
<pre><code data-lang="r"><span>library</span><span>(</span><span>mlrMBO</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>library</span><span>(</span><span>ggplot2</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>set.seed</span><span>(</span><span>1</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""><br data-jekyll-commonmark-ghpages=""></span><span>fun</span><span> </span><span>=</span><span> </span><span>function</span><span>(</span><span>x</span><span>)</span><span> </span><span>{</span><span><br data-jekyll-commonmark-ghpages=""> </span><span>x</span><span>^</span><span>2</span><span> </span><span>+</span><span> </span><span>sin</span><span>(</span><span>2</span><span> </span><span>*</span><span> </span><span>pi</span><span> </span><span>*</span><span> </span><span>x</span><span>)</span><span> </span><span>*</span><span> </span><span>cos</span><span>(</span><span>0.3</span><span> </span><span>*</span><span> </span><span>pi</span><span> </span><span>*</span><span> </span><span>x</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>}</span></code></pre>
</figure>
<p>However we still need to define the our search space.<br>
In this case we look for a real valued value between -3 and 3.<br>
For more hints about how to define ParamSets you can look <a rel="nofollow" href="http://jakob-r.de/mlrHyperopt/articles/working_with_parconfigs_and_paramsets.html#the-basics-of-a-paramset" target="_blank">here</a> or in the <a rel="nofollow" href="https://rdrr.io/cran/ParamHelpers/man/makeParamSet.html" target="_blank">help of ParamHelpers</a>.</p>
<figure>
<pre><code data-lang="r"><span>ps</span><span> </span><span>=</span><span> </span><span>makeParamSet</span><span>(</span><span><br data-jekyll-commonmark-ghpages=""> </span><span>makeNumericParam</span><span>(</span><span>&quot;x&quot;</span><span>,</span><span> </span><span>lower</span><span> </span><span>=</span><span> </span><span>-3</span><span>,</span><span> </span><span>upper</span><span> </span><span>=</span><span> </span><span>3</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>)</span></code></pre>
</figure>
<p>We also need some initial evaluations to start the optimization.<br>
The design has to be passed as a <code>data.frame</code> with one column for each dimension of the search space and one column <code>y</code> for the outcomes of the objective function.</p>
<figure>
<pre><code data-lang="r"><span>des</span><span> </span><span>=</span><span> </span><span>generateDesign</span><span>(</span><span>n</span><span> </span><span>=</span><span> </span><span>3</span><span>,</span><span> </span><span>par.set</span><span> </span><span>=</span><span> </span><span>ps</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>des</span><span>$</span><span>y</span><span> </span><span>=</span><span> </span><span>apply</span><span>(</span><span>des</span><span>,</span><span> </span><span>1</span><span>,</span><span> </span><span>fun</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>des</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">##            x         y<br data-jekyll-commonmark-ghpages="">## 1 -1.1835844 0.9988801<br data-jekyll-commonmark-ghpages="">## 2 -0.5966361 0.8386779<br data-jekyll-commonmark-ghpages="">## 3  2.7967794 8.6592973</code></pre>
</figure>
<p>With these values we can initialize our sequential MBO object.</p>
<figure>
<pre><code data-lang="r"><span>ctrl</span><span> </span><span>=</span><span> </span><span>makeMBOControl</span><span>()</span><span><br data-jekyll-commonmark-ghpages=""></span><span>ctrl</span><span> </span><span>=</span><span> </span><span>setMBOControlInfill</span><span>(</span><span>ctrl</span><span>,</span><span> </span><span>crit</span><span> </span><span>=</span><span> </span><span>crit.ei</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>opt.state</span><span> </span><span>=</span><span> </span><span>initSMBO</span><span>(</span><span><br data-jekyll-commonmark-ghpages=""> </span><span>par.set</span><span> </span><span>=</span><span> </span><span>ps</span><span>,</span><span> <br data-jekyll-commonmark-ghpages=""> </span><span>design</span><span> </span><span>=</span><span> </span><span>des</span><span>,</span><span> <br data-jekyll-commonmark-ghpages=""> </span><span>control</span><span> </span><span>=</span><span> </span><span>ctrl</span><span>,</span><span> <br data-jekyll-commonmark-ghpages=""> </span><span>minimize</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>,</span><span> <br data-jekyll-commonmark-ghpages=""> </span><span>noisy</span><span> </span><span>=</span><span> </span><span>FALSE</span><span>)</span></code></pre>
</figure>
<p>The <code>opt.state</code> now contains all necessary information for the optimization.<br>
We can even plot it to see how the Gaussian process models the objective function.</p>
<figure>
<pre><code data-lang="r"><span>plot</span><span>(</span><span>opt.state</span><span>)</span></code></pre>
</figure>
<p><img alt="plot of chunk optstate1" src="https://mlr-org.github.io/figures/2018-01-10-Stepwise-Bayesian-Optimization-with-mlrMBO/optstate1-1.svg"></p>
<p>In the first panel the <em>expected improvement</em> ($EI = E(y_{min}-\hat{y})$) (see <a rel="nofollow" href="http://www.ressources-actuarielles.net/EXT/ISFA/1226.nsf/0/f84f7ac703bf5862c12576d8002f5259/$FILE/Jones98.pdf" target="_blank">Jones et.al.</a>) is plotted over the search space.<br>
The maximum of the <em>EI</em> indicates the point that we should evaluate next.<br>
The second panel shows the mean prediction of the surrogate model, which is the Gaussian regression model aka <em>Kriging</em> in this example.<br>
The third panel shows the uncertainty prediction of the surrogate.<br>
We can see, that the <em>EI</em> is high at points, where the mean prediction is low and/or the uncertainty is high.</p>
<p>To obtain the specific configuration suggested by mlrMBO for the next evaluation of the objective we can run:</p>
<figure>
<pre><code data-lang="r"><span>prop</span><span> </span><span>=</span><span> </span><span>proposePoints</span><span>(</span><span>opt.state</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>prop</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## $prop.points<br data-jekyll-commonmark-ghpages="">##             x<br data-jekyll-commonmark-ghpages="">## 969 -2.999979<br data-jekyll-commonmark-ghpages="">## <br data-jekyll-commonmark-ghpages="">## $propose.time<br data-jekyll-commonmark-ghpages="">## [1] 0.273<br data-jekyll-commonmark-ghpages="">## <br data-jekyll-commonmark-ghpages="">## $prop.type<br data-jekyll-commonmark-ghpages="">## [1] &quot;infill_ei&quot;<br data-jekyll-commonmark-ghpages="">## <br data-jekyll-commonmark-ghpages="">## $crit.vals<br data-jekyll-commonmark-ghpages="">##            [,1]<br data-jekyll-commonmark-ghpages="">## [1,] -0.3733677<br data-jekyll-commonmark-ghpages="">## <br data-jekyll-commonmark-ghpages="">## $crit.components<br data-jekyll-commonmark-ghpages="">##       se     mean<br data-jekyll-commonmark-ghpages="">## 1 2.8899 3.031364<br data-jekyll-commonmark-ghpages="">## <br data-jekyll-commonmark-ghpages="">## $errors.model<br data-jekyll-commonmark-ghpages="">## [1] NA<br data-jekyll-commonmark-ghpages="">## <br data-jekyll-commonmark-ghpages="">## attr(,&quot;class&quot;)<br data-jekyll-commonmark-ghpages="">## [1] &quot;Proposal&quot; &quot;list&quot;</code></pre>
</figure>
<p>We will execute our objective function with the suggested value for <code>x</code> and feed it back to mlrMBO:</p>
<figure>
<pre><code data-lang="r"><span>y</span><span> </span><span>=</span><span> </span><span>fun</span><span>(</span><span>prop</span><span>$</span><span>prop.points</span><span>$</span><span>x</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>y</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## [1] 8.999752</code></pre>
</figure>
<figure>
<pre><code data-lang="r"><span>updateSMBO</span><span>(</span><span>opt.state</span><span>,</span><span> </span><span>x</span><span> </span><span>=</span><span> </span><span>prop</span><span>$</span><span>prop.points</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>y</span><span>)</span></code></pre>
</figure>
<p>The nice thing about the <em>human-in-the-loop</em> mode is, that you don’t have to stick to the suggestion.<br>
In other words we can feed the model with values without receiving a proposal.<br>
Let’s assume we have an expert who tells us to evaluate the values $x=-1$ and $x=1$ we can easily do so:</p>
<figure>
<pre><code data-lang="r"><span>custom.prop</span><span> </span><span>=</span><span> </span><span>data.frame</span><span>(</span><span>x</span><span> </span><span>=</span><span> </span><span>c</span><span>(</span><span>-1</span><span>,</span><span>1</span><span>))</span><span><br data-jekyll-commonmark-ghpages=""></span><span>ys</span><span> </span><span>=</span><span> </span><span>apply</span><span>(</span><span>custom.prop</span><span>,</span><span> </span><span>1</span><span>,</span><span> </span><span>fun</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>updateSMBO</span><span>(</span><span>opt.state</span><span>,</span><span> </span><span>x</span><span> </span><span>=</span><span> </span><span>custom.prop</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>as.list</span><span>(</span><span>ys</span><span>))</span><span><br data-jekyll-commonmark-ghpages=""></span><span>plot</span><span>(</span><span>opt.state</span><span>,</span><span> </span><span>scale.panels</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span></code></pre>
</figure>
<p><img alt="plot of chunk feedmanual" src="https://mlr-org.github.io/figures/2018-01-10-Stepwise-Bayesian-Optimization-with-mlrMBO/feedmanual-1.svg"></p>
<p>We can also automate the process easily:</p>
<figure>
<pre><code data-lang="r"><span>replicate</span><span>(</span><span>3</span><span>,</span><span> </span><span>{</span><span><br data-jekyll-commonmark-ghpages=""> </span><span>prop</span><span> </span><span>=</span><span> </span><span>proposePoints</span><span>(</span><span>opt.state</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""> </span><span>y</span><span> </span><span>=</span><span> </span><span>fun</span><span>(</span><span>prop</span><span>$</span><span>prop.points</span><span>$</span><span>x</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""> </span><span>updateSMBO</span><span>(</span><span>opt.state</span><span>,</span><span> </span><span>x</span><span> </span><span>=</span><span> </span><span>prop</span><span>$</span><span>prop.points</span><span>,</span><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>y</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>})</span></code></pre>
</figure>
<p><em>Note:</em> We suggest to use the normal mlrMBO if you are only doing this as mlrMBO has more advanced logging, termination and handling of errors etc.</p>
<p>Let’s see how the surrogate models the true objective function after having seen seven configurations:</p>
<figure>
<pre><code data-lang="r"><span>plot</span><span>(</span><span>opt.state</span><span>,</span><span> </span><span>scale.panels</span><span> </span><span>=</span><span> </span><span>TRUE</span><span>)</span></code></pre>
</figure>
<p><img alt="plot of chunk optstate2" src="https://mlr-org.github.io/figures/2018-01-10-Stepwise-Bayesian-Optimization-with-mlrMBO/optstate2-1.svg"></p>
<p>You can convert the <code>opt.state</code> object from this run to a normal mlrMBO result object like this:</p>
<figure>
<pre><code data-lang="r"><span>res</span><span> </span><span>=</span><span> </span><span>finalizeSMBO</span><span>(</span><span>opt.state</span><span>)</span><span><br data-jekyll-commonmark-ghpages=""></span><span>res</span></code></pre>
</figure>
<figure>
<pre><code data-lang="text">## Recommended parameters:<br data-jekyll-commonmark-ghpages="">## x=-0.22<br data-jekyll-commonmark-ghpages="">## Objective: y = -0.913<br data-jekyll-commonmark-ghpages="">## <br data-jekyll-commonmark-ghpages="">## Optimization path<br data-jekyll-commonmark-ghpages="">## 3 + 6 entries in total, displaying last 10 (or less):<br data-jekyll-commonmark-ghpages="">##            x          y dob eol error.message exec.time         ei<br data-jekyll-commonmark-ghpages="">## 1 -1.1835844  0.9988801   0  NA          </code></pre></figure>