JaynesProbabilityTheory/chapter11.html at master · MaksimIM/JaynesProbabilityTheory · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="pandoc" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
  <title>chapter11</title>
  <style>
    code{white-space: pre-wrap;}
    span.smallcaps{font-variant: small-caps;}
    span.underline{text-decoration: underline;}
    div.column{display: inline-block; vertical-align: top; width: 50%;}
    div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
    ul.task-list{list-style: none;}
  </style>
  <script src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js" type="text/javascript"></script>
  <!--[if lt IE 9]>
    <script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
  <![endif]-->
</head>
<body>
<h1 id="discrete-prior-probabilities-the-entropy-principle">Discrete prior probabilities: the entropy principle</h1>
<p><span class="math inline">\(\leftarrow\)</span> <a href="./index.html">Back to Chapters</a></p>
<h3 id="comments-on-11.6">Comments on 11.6</h3>
<p>First of all, Lagrange multipliers method can give rigorous proofs. Of course the presentation in 11.6 is not rigorous, but that is a feature of the particular situation and presentation, not a general comment on Lagrange multipliers method.</p>
<p>Secondly, from “modern” perspective we see that the constraints <span class="math inline">\(E_p[f_k]=F_k\)</span> imply that the cross-entropy</p>
<p><span class="math display">\[H(p, u)=E_p[-\log u]=\]</span> <span class="math display">\[E_p[\log Z(\vec{\lambda})]+E_p[\sum \lambda_i f_i]=\]</span> <span class="math display">\[\log Z(\vec{\lambda})+\vec{\lambda}\cdot \vec{F}\]</span></p>
<p>is fixed — and equal to <span class="math inline">\(H(u)\)</span>.</p>
<p>Then</p>
<p><span class="math display">\[H(p)=H(p, u)-D_{KL}(p,u) =\]</span></p>
<p><span class="math display">\[H(u)-D_{KL}(p,u) \leq H(u)\]</span></p>
<p>with equality precisely when <span class="math inline">\(p=u\)</span>.</p>
<h3 id="comment-on-10.7-and-10.8">Comment on 10.7 and 10.8</h3>
<p>This seems to ignore completely the debate on foundations of quantum mechanics, and such topics as “alternative” formulations of de Broglie–Bohm pilot wave and Everett many world interpretation, as well as the Bell’s theorem. Jaynes’s views on a lot of this are in his paper “<a href="https://bayes.wustl.edu/etj/articles/cmystery.pdf">Clearing up the mysteries – the original goal</a>”.</p>
</body>
</html>