<p> </p>
<div data-shortcode="caption" class="wp-caption"><a rel="nofollow" href="https://statisfaction.wordpress.com/?attachment_id=6973" target="_blank"><img data-orig-file="https://statisfaction.files.wordpress.com/2017/12/proof500.gif?w=456" data-orig-size="500,188" data-medium-file="https://statisfaction.files.wordpress.com/2017/12/proof500.gif?w=456?w=400" data-attachment-id="6973" data-image-meta="{" data-image-description="" src="https://statisfaction.files.wordpress.com/2017/12/proof500.gif?w=456" data-permalink="https://statisfaction.wordpress.com/2017/12/20/sub-gaussian-property-for-the-beta-distribution-part-2/proof500/" data-large-file="https://statisfaction.files.wordpress.com/2017/12/proof500.gif?w=456?w=500" alt="" data-image-title="proof500" data-comments-opened="1" class="wp-image-6973" data-recalc-dims="1"></a>
<p class="wp-caption-text"><em>Left</em>: What makes the <span>Beta optimal proxy variance (red)</span> so special? <em>Right</em>: The difference function has a double zero (black dot).</p>
</div>
<p>As a follow-up on my previous <a rel="nofollow" href="https://statisfaction.wordpress.com/2017/05/02/sub-gaussian-property-for-the-beta-distribution-part-1/" target="_blank">post</a> on the sub-Gaussian property for the Beta distribution [1], I’ll give here a visual illustration of the proof.</p>
<p>A random variable <img alt="X" title="X" src="https://s0.wp.com/latex.php?latex=X&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> with finite mean <img alt="mu=mathbb{E}[X]" title="mu=mathbb{E}[X]" src="https://s0.wp.com/latex.php?latex=%5Cmu%3D%5Cmathbb%7BE%7D%5BX%5D&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> is sub-Gaussian if there is a positive number <img alt="sigma" title="sigma" src="https://s0.wp.com/latex.php?latex=%5Csigma&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> such that:</p>
<p><img alt="mathbb{E}[exp(lambda (X-mu))]leexpleft(frac{lambda^2sigma^2}{2}right),,text{for all } lambdainmathbb{R}." title="mathbb{E}[exp(lambda (X-mu))]leexpleft(frac{lambda^2sigma^2}{2}right),,text{for all } lambdainmathbb{R}." src="https://s0.wp.com/latex.php?latex=%5Cmathbb%7BE%7D%5B%5Cexp%28%5Clambda+%28X-%5Cmu%29%29%5D%5Cle%5Cexp%5Cleft%28%5Cfrac%7B%5Clambda%5E2%5Csigma%5E2%7D%7B2%7D%5Cright%29%5C%2C%5C%2C%5Ctext%7Bfor+all+%7D+%5Clambda%5Cin%5Cmathbb%7BR%7D.&bg=fff&%23038;fg=222&%23038;s=0" class="latex"></p>
<p>We focus on <em>X</em> being a Beta<img alt="(alpha,beta)" title="(alpha,beta)" src="https://s0.wp.com/latex.php?latex=%28%5Calpha%2C%5Cbeta%29&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> random variable. Its moment generating function <img alt="mathbb{E}[exp(lambda X)]" title="mathbb{E}[exp(lambda X)]" src="https://s0.wp.com/latex.php?latex=%5Cmathbb%7BE%7D%5B%5Cexp%28%5Clambda+X%29%5D&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> is known as the Kummer function, or <a rel="nofollow" href="https://en.wikipedia.org/wiki/Confluent_hypergeometric_function" target="_blank">confluent hypergeometric function</a> <img alt="_1F_1(alpha,alpha+beta,lambda)" title="_1F_1(alpha,alpha+beta,lambda)" src="https://s0.wp.com/latex.php?latex=_1F_1%28%5Calpha%2C%5Calpha%2B%5Cbeta%2C%5Clambda%29&bg=fff&%23038;fg=222&%23038;s=0" class="latex">. So <em>X </em>is <img alt="sigma^2" title="sigma^2" src="https://s0.wp.com/latex.php?latex=%5Csigma%5E2&bg=fff&%23038;fg=222&%23038;s=0" class="latex">-sub-Gaussian as soon as the difference function</p>
<p><img alt="u_sigma(lambda)=expleft(frac{alpha}{alpha+beta}lambda+frac{sigma^2}{2}lambda^2right)-_1F_1(alpha,alpha+beta,lambda)" title="u_sigma(lambda)=expleft(frac{alpha}{alpha+beta}lambda+frac{sigma^2}{2}lambda^2right)-_1F_1(alpha,alpha+beta,lambda)" src="https://s0.wp.com/latex.php?latex=u_%5Csigma%28%5Clambda%29%3D%5Cexp%5Cleft%28%5Cfrac%7B%5Calpha%7D%7B%5Calpha%2B%5Cbeta%7D%5Clambda%2B%5Cfrac%7B%5Csigma%5E2%7D%7B2%7D%5Clambda%5E2%5Cright%29-_1F_1%28%5Calpha%2C%5Calpha%2B%5Cbeta%2C%5Clambda%29&bg=fff&%23038;fg=222&%23038;s=0" class="latex"></p>
<p>remains positive on <img alt="mathbb{R}" title="mathbb{R}" src="https://s0.wp.com/latex.php?latex=%5Cmathbb%7BR%7D&bg=fff&%23038;fg=222&%23038;s=0" class="latex">. This difference function <img alt="u_sigma(cdot)" title="u_sigma(cdot)" src="https://s0.wp.com/latex.php?latex=u_%5Csigma%28%5Ccdot%29&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> is plotted on the right panel above for parameters <img alt="(alpha,beta)=(1,1.3)" title="(alpha,beta)=(1,1.3)" src="https://s0.wp.com/latex.php?latex=%28%5Calpha%2C%5Cbeta%29%3D%281%2C1.3%29&bg=fff&%23038;fg=222&%23038;s=0" class="latex">. In the plot, <img alt="sigma^2" title="sigma^2" src="https://s0.wp.com/latex.php?latex=%5Csigma%5E2&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> is varying from <span>green for the variance <img alt="text{Var}[X]=frac{alphabeta}{(alpha+beta)^2(alpha+beta+1)}" title="text{Var}[X]=frac{alphabeta}{(alpha+beta)^2(alpha+beta+1)}" src="https://s0.wp.com/latex.php?latex=%5Ctext%7BVar%7D%5BX%5D%3D%5Cfrac%7B%5Calpha%5Cbeta%7D%7B%28%5Calpha%2B%5Cbeta%29%5E2%28%5Calpha%2B%5Cbeta%2B1%29%7D&bg=fff&%23038;fg=222&%23038;s=0" class="latex"></span> (which is a <span>lower bound to the optimal proxy variance</span>) to <span>blue for the value <img alt="frac{1}{4(alpha+beta+1)}" title="frac{1}{4(alpha+beta+1)}" src="https://s0.wp.com/latex.php?latex=%5Cfrac%7B1%7D%7B4%28%5Calpha%2B%5Cbeta%2B1%29%7D&bg=fff&%23038;fg=222&%23038;s=0" class="latex"></span>, <span>a simple upper bound</span> given by Elder (2016), [2]. The idea of the proof is simple: the optimal proxy-variance corresponds to the value of <img alt="sigma^2" title="sigma^2" src="https://s0.wp.com/latex.php?latex=%5Csigma%5E2&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> for which <img alt="u_sigma(cdot)" title="u_sigma(cdot)" src="https://s0.wp.com/latex.php?latex=u_%5Csigma%28%5Ccdot%29&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> admits a <span>double zero, as illustrated with the red curve</span> (black dot). The left panel shows the curves with <img alt="mu = frac{alpha}{alpha+beta}" title="mu = frac{alpha}{alpha+beta}" src="https://s0.wp.com/latex.php?latex=%5Cmu+%3D+%5Cfrac%7B%5Calpha%7D%7B%5Calpha%2B%5Cbeta%7D&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> varying, interpolating from <span>green</span> for <img alt="text{Var}[X]=frac{alphabeta}{(alpha+beta)^2(alpha+beta+1)}" title="text{Var}[X]=frac{alphabeta}{(alpha+beta)^2(alpha+beta+1)}" src="https://s0.wp.com/latex.php?latex=%5Ctext%7BVar%7D%5BX%5D%3D%5Cfrac%7B%5Calpha%5Cbeta%7D%7B%28%5Calpha%2B%5Cbeta%29%5E2%28%5Calpha%2B%5Cbeta%2B1%29%7D&bg=fff&%23038;fg=222&%23038;s=0" class="latex"> to <span>blue</span> for <img alt="frac{1}{4(alpha+beta+1)}" title="frac{1}{4(alpha+beta+1)}" src="https://s0.wp.com/latex.php?latex=%5Cfrac%7B1%7D%7B4%28%5Calpha%2B%5Cbeta%2B1%29%7D&bg=fff&%23038;fg=222&%23038;s=0" class="latex">, with <span>only one curve qualifying as the optimal proxy variance in red</span>.</p>
<h4>References</h4>
<p>[1] Marchal and Arbel (2017), <a rel="nofollow" href="https://projecteuclid.org/euclid.ecp/1507860211" target="_blank">On the sub-Gaussianity of the Beta and Dirichlet distributions</a>. Electronic Communications in Probability, 22:1–14, 2017. Code on <a rel="nofollow" href="https://github.com/jarbel/Beta-sub-Gaussian" target="_blank">GitHub</a>.<br>
[2] Elder (2016), Bayesian Adaptive Data Analysis Guarantees from Subgaussianity, <a rel="nofollow" href="https://arxiv.org/abs/1611.00065" target="_blank">https://arxiv.org/abs/1611.00065</a></p>
<p> <a rel="nofollow" href="https://feeds.wordpress.com/1.0/gocomments/statisfaction.wordpress.com/6972/" target="_blank"><img border="0" alt="" src="https://feeds.wordpress.com/1.0/comments/statisfaction.wordpress.com/6972/"></a> <img border="0" alt="" src="https://i1.wp.com/pixel.wp.com/b.gif?resize=1%2C1&ssl=1" data-recalc-dims="1"></p>
	<img alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/9vf318wxp3Q" height="1">