/usr/share/doc/pyxplot/html/sect0243.html is in pyxplot-doc 0.8.4-3.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta name="generator" content="plasTeX" />
<meta content="text/html; charset=utf-8" http-equiv="content-type" />
<title>PyXPlot Users' Guide: Estimating the Error in </title>
<link href="sect0244.html" title="The Covariance Matrix" rel="next" />
<link href="sec-bayes_pdf.html" title="The Probability Density Function" rel="prev" />
<link href="ch-fit_maths.html" title="The fit Command: Mathematical Details" rel="up" />
<link rel="stylesheet" href="styles/styles.css" />
</head>
<body>
<div class="navigation">
<table cellspacing="2" cellpadding="0" width="100%">
<tr>
<td><a href="sec-bayes_pdf.html" title="The Probability Density Function"><img alt="Previous: The Probability Density Function" border="0" src="icons/previous.gif" width="32" height="32" /></a></td>
<td><a href="ch-fit_maths.html" title="The fit Command: Mathematical Details"><img alt="Up: The fit Command: Mathematical Details" border="0" src="icons/up.gif" width="32" height="32" /></a></td>
<td><a href="sect0244.html" title="The Covariance Matrix"><img alt="Next: The Covariance Matrix" border="0" src="icons/next.gif" width="32" height="32" /></a></td>
<td class="navtitle" align="center">PyXPlot Users' Guide</td>
<td><a href="index.html" title="Table of Contents"><img border="0" alt="" src="icons/contents.gif" width="32" height="32" /></a></td>
<td><a href="sect0255.html" title="Index"><img border="0" alt="" src="icons/index.gif" width="32" height="32" /></a></td>
<td><img border="0" alt="" src="icons/blank.gif" width="32" height="32" /></td>
</tr>
</table>
</div>
<div class="breadcrumbs">
<span>
<span>
<a href="index.html">PyXPlot Users' Guide</a> <b>:</b>
</span>
</span><span>
<span>
<a href="sect0230.html">Appendices</a> <b>:</b>
</span>
</span><span>
<span>
<a href="ch-fit_maths.html">The <tt class="tt">fit</tt> Command: Mathematical Details</a> <b>:</b>
</span>
</span><span>
<span>
<b class="current">Estimating the Error in <img src="images/img-0804.png" alt="$\mathbf{u}^0$" style="vertical-align:0px;
width:19px;
height:16px" class="math gen" /></b>
</span>
</span>
<hr />
</div>
<div><h1 id="a0000000244">C.3 Estimating the Error in <img src="images/img-0804.png" alt="$\mathbf{u}^0$" style="vertical-align:0px;
width:19px;
height:16px" class="math gen" /></h1>
<p>To estimate the error in the best-fitting parameter values that we find, we assume <img src="images/img-0798.png" alt="$\mathrm{P}\left( \mathbf{u} | \left\{ \mathbf{x}_ i, f_ i, \sigma _ i \right\} \right)$" style="vertical-align:-5px;
width:129px;
height:19px" class="math gen" /> to be approximated by an <img src="images/img-0794.png" alt="$n_\mathrm {u}$" style="vertical-align:-2px;
width:19px;
height:10px" class="math gen" />-dimensional Gaussian distribution around <img src="images/img-0804.png" alt="$\mathbf{u}^0$" style="vertical-align:0px;
width:19px;
height:16px" class="math gen" />. Taking a Taylor expansion of <img src="images/img-0805.png" alt="$L(\mathbf{u})$" style="vertical-align:-4px;
width:37px;
height:18px" class="math gen" /> about <img src="images/img-0804.png" alt="$\mathbf{u}^0$" style="vertical-align:0px;
width:19px;
height:16px" class="math gen" />, we can write: </p><table id="<plasTeX.TeXFragment object at 0xb216cec>" cellpadding="7" width="100%" cellspacing="0" class="eqnarray">
<tr id="a0000001670">
<td style="width:40%"> </td>
<td style="vertical-align:middle; text-align:right"><img src="images/img-0806.png" alt="$\displaystyle L(\mathbf{u}) $" style="vertical-align:-4px; width:37px; height:18px" class="math gen" /></td>
<td style="vertical-align:middle; text-align:center"><img src="images/img-0058.png" alt="$\displaystyle = $" style="vertical-align:2px; width:12px; height:4px" class="math gen" /></td>
<td style="vertical-align:middle; text-align:left"><img src="images/img-0807.png" alt="$\displaystyle L(\mathbf{u}^0) + \underbrace{ \sum _{i=0}^{n_\mathrm {u}-1} \left( u_ i - u^0_ i \right) \left.\frac{\partial L}{\partial u_ i}\right|_{\mathbf{u}^0} }_{\textrm{Zero at $\mathbf{u}^0$ by definition}} + \label{eqa:L_ taylor_ expand} $" style="vertical-align:-53px; width:253px; height:85px" class="math gen" /></td>
<td style="width:40%"> </td>
<td style="width:20%" class="eqnnum"><span>(<span>C.4</span>)</span></td>
</tr><tr id="a0000001671">
<td style="width:40%"> </td>
<td style="vertical-align:middle; text-align:right"><img src="images/img-0175.png" alt="$\displaystyle $" style="vertical-align:0px; width:1px; height:1px" class="math gen" /></td>
<td style="vertical-align:middle; text-align:center"><img src="images/img-0175.png" alt="$\displaystyle $" style="vertical-align:0px; width:1px; height:1px" class="math gen" /></td>
<td style="vertical-align:middle; text-align:left"><img src="images/img-0808.png" alt="$\displaystyle \sum _{i=0}^{n_\mathrm {u}-1} \sum _{j=0}^{n_\mathrm {u}-1} \frac{\left( u_ i - u^0_ i \right) \left( u_ j - u^0_ j \right)}{2} \left.\frac{\partial ^2 L}{\partial u_ i \partial u_ j}\right|_{\mathbf{u}^0} + \mathcal{O}\left( \mathbf{u} - \mathbf{u}^0\right)^3 \nonumber $" style="vertical-align:-25px; width:438px; height:65px" class="math gen" /></td>
<td style="width:40%"> </td>
<td style="width:20%" class="eqnnum"> </td>
</tr>
</table><p>Since the logarithm of a Gaussian distribution is a parabola, the quadratic terms in the above expansion encode the Gaussian component of the probability distribution <img src="images/img-0798.png" alt="$\mathrm{P}\left( \mathbf{u} | \left\{ \mathbf{x}_ i, f_ i, \sigma _ i \right\} \right)$" style="vertical-align:-5px;
width:129px;
height:19px" class="math gen" /> about <img src="images/img-0804.png" alt="$\mathbf{u}^0$" style="vertical-align:0px;
width:19px;
height:16px" class="math gen" />.<a href="#a0000001672" class="footnote"><sup class="footnotemark">1</sup></a> We may write the sum of these terms, which we denote <img src="images/img-0809.png" alt="$Q$" style="vertical-align:-4px;
width:14px;
height:16px" class="math gen" />, in matrix form: </p><table id="<plasTeX.TeXFragment object at 0xb22d9bc>" class="equation" width="100%" cellspacing="0" cellpadding="7">
<tr>
<td style="width:40%"> </td>
<td><img src="images/img-0810.png" alt="\begin{equation} Q = \frac{1}{2} \left(\mathbf{u} - \mathbf{u}^0\right)^\mathbf {T} \mathbf{A} \left(\mathbf{u} - \mathbf{u}^0\right) \label{eqn:Q_ vector} \end{equation}" style="width:412px;
height:37px" class="math gen" /></td>
<td style="width:40%"> </td>
<td class="eqnnum" style="width:20%"><span>(<span>C.5</span>)</span></td>
</tr>
</table><p>where the superscript <img src="images/img-0811.png" alt="$^\mathbf {T}$" style="vertical-align:7px;
width:11px;
height:9px" class="math gen" /> represents the transpose of the vector displacement from <img src="images/img-0804.png" alt="$\mathbf{u}^0$" style="vertical-align:0px;
width:19px;
height:16px" class="math gen" />, and <img src="images/img-0812.png" alt="$\mathbf{A}$" style="vertical-align:0px;
width:15px;
height:12px" class="math gen" /> is the Hessian matrix of <img src="images/img-0378.png" alt="$L$" style="vertical-align:0px;
width:12px;
height:12px" class="math gen" />, given by: </p><table id="a0000001673" class="equation" width="100%" cellspacing="0" cellpadding="7">
<tr>
<td style="width:40%"> </td>
<td><img src="images/img-0813.png" alt="\begin{equation} A_{ij} = \nabla \nabla L = \left.\frac{\partial ^2 L}{\partial u_ i \partial u_ j}\right|_{\mathbf{u}^0} \end{equation}" style="width:399px;
height:46px" class="math gen" /></td>
<td style="width:40%"> </td>
<td class="eqnnum" style="width:20%"><span>(<span>C.6</span>)</span></td>
</tr>
</table><p> <a name="a0000001674" id="a0000001674"></a> </p><p>This is the Hessian matrix which is output by the <tt class="tt">fit</tt> command. In general, an <img src="images/img-0794.png" alt="$n_\mathrm {u}$" style="vertical-align:-2px;
width:19px;
height:10px" class="math gen" />-dimensional Gaussian distribution such as that given by Equation (<a></a>) yields elliptical contours of equi-probability in parameter space, whose principal axes need not be aligned with our chosen coordinate axes – the variables <img src="images/img-0814.png" alt="$u_0 ... u_{n_ u-1}$" style="vertical-align:-4px;
width:77px;
height:12px" class="math gen" />. The eigenvectors <img src="images/img-0815.png" alt="$\mathbf{e}_ i$" style="vertical-align:-2px;
width:14px;
height:10px" class="math gen" /> of <img src="images/img-0812.png" alt="$\mathbf{A}$" style="vertical-align:0px;
width:15px;
height:12px" class="math gen" /> are the principal axes of these ellipses, and the corresponding eigenvalues <img src="images/img-0816.png" alt="$\lambda _ i$" style="vertical-align:-2px;
width:14px;
height:15px" class="math gen" /> equal <img src="images/img-0817.png" alt="$1/\sigma _ i^2$" style="vertical-align:-5px;
width:35px;
height:21px" class="math gen" />, where <img src="images/img-0792.png" alt="$\sigma _ i$" style="vertical-align:-2px;
width:14px;
height:10px" class="math gen" /> is the standard deviation of the probability density function along the direction of these axes. </p><p>This can be visualised by imagining that we diagonalise <img src="images/img-0812.png" alt="$\mathbf{A}$" style="vertical-align:0px;
width:15px;
height:12px" class="math gen" />, and expand Equation (<a></a>) in our diagonal basis. The resulting expression for <img src="images/img-0378.png" alt="$L$" style="vertical-align:0px;
width:12px;
height:12px" class="math gen" /> is a sum of square terms; the cross terms vanish in this basis by definition. The equations of the equi-probability contours become the equations of ellipses: </p><table id="a0000001675" class="equation" width="100%" cellspacing="0" cellpadding="7">
<tr>
<td style="width:40%"> </td>
<td><img src="images/img-0818.png" alt="\begin{equation} Q = \frac{1}{2} \sum _{i=0}^{n_\mathrm {u}-1} A_{ii} \left(u_ i - u^0_ i\right)^2 = k \end{equation}" style="width:414px;
height:53px" class="math gen" /></td>
<td style="width:40%"> </td>
<td class="eqnnum" style="width:20%"><span>(<span>C.7</span>)</span></td>
</tr>
</table><p>where <img src="images/img-0651.png" alt="$k$" style="vertical-align:0px;
width:8px;
height:13px" class="math gen" /> is some constant. By comparison with the equation for the logarithm of a Gaussian distribution, we can associate <img src="images/img-0819.png" alt="$A_{ii}$" style="vertical-align:-2px;
width:23px;
height:14px" class="math gen" /> with <img src="images/img-0820.png" alt="$-1/\sigma _ i^2$" style="vertical-align:-5px;
width:49px;
height:21px" class="math gen" /> in our eigenvector basis. </p><p>The problem of evaluating the standard deviations of our variables <img src="images/img-0821.png" alt="$u_ i$" style="vertical-align:-2px;
width:14px;
height:10px" class="math gen" /> is more complicated, however, as we are attempting to evaluate the width of these elliptical equi-probability contours in directions which are, in general, not aligned with their principal axes. To achieve this, we first convert our Hessian matrix into a covariance matrix. </p></div>
<div id="footnotes">
<p><b>Footnotes</b></p>
<ol>
<li id="a0000001672">The use of this is called <i class="itshape">Gauss’ Method</i>. Higher order terms in the expansion represent any non-Gaussianity in the probability distribution, which we neglect. See MacKay, D.J.C., <i class="itshape">Information Theory, Inference and Learning Algorithms</i>, CUP (2003).</li>
</ol>
</div>
<div class="navigation">
<table cellspacing="2" cellpadding="0" width="100%">
<tr>
<td><a href="sec-bayes_pdf.html" title="The Probability Density Function"><img alt="Previous: The Probability Density Function" border="0" src="icons/previous.gif" width="32" height="32" /></a></td>
<td><a href="ch-fit_maths.html" title="The fit Command: Mathematical Details"><img alt="Up: The fit Command: Mathematical Details" border="0" src="icons/up.gif" width="32" height="32" /></a></td>
<td><a href="sect0244.html" title="The Covariance Matrix"><img alt="Next: The Covariance Matrix" border="0" src="icons/next.gif" width="32" height="32" /></a></td>
<td class="navtitle" align="center">PyXPlot Users' Guide</td>
<td><a href="index.html" title="Table of Contents"><img border="0" alt="" src="icons/contents.gif" width="32" height="32" /></a></td>
<td><a href="sect0255.html" title="Index"><img border="0" alt="" src="icons/index.gif" width="32" height="32" /></a></td>
<td><img border="0" alt="" src="icons/blank.gif" width="32" height="32" /></td>
</tr>
</table>
</div>
<script language="javascript" src="icons/imgadjust.js" type="text/javascript"></script>
</body>
</html>
|